For a 2 tailed test, the alpha is divided by 2. As we look back to the statistical table, we take 0.05, right? Alternatively, can we take 0.025 instead but do not divide the alpha by 2? Will that be the same?

When looking at the critical values for a two-tailed test with 5% level of significance, we look for the columns under "Level of significance for two-tailed test" which have already accounted for the two-tailed nature of the test. However, you may also look for the columns under "level of significance for one-tailed test". However, they do not account for the two-tailed nature, and thus, you need to do it yourself and use 0.025 instead. You will realize you end up using the same column.