Specifically, they can make either Type I or Type II errors. As you analyze your own data and test hypotheses, understanding the difference between Type I and Type II errors is extremely important, because there’s a risk of making each type of error in every analysis, and the amount of risk is in your control.

A type I error (false-positive) occurs if an investigator rejects a null hypothesis that is actually true in the population; a type II error (false-negative) occurs if the investigator fails to reject a null hypothesis that is actually false in the population.

If your statistical test was significant, you would have then committed a Type I error, as the null hypothesis is actually true. To decrease your chance of committing a Type I error, simply make your alpha (p) value more stringent. Chances of committing a Type II error are related to your analyses’ statistical power.

Increasing sample size makes the hypothesis test more sensitive – more likely to reject the null hypothesis when it is, in fact, false. Thus, it increases the power of the test. And the probability of making a Type II error gets smaller, not bigger, as sample size increases.

Type 1 errors have a probability of “α” correlated to the level of confidence that you set. A test with a 95% confidence level means that there is a 5% chance of getting a type 1 error.

Simply put, power is the probability of not making a Type II error, according to Neil Weiss in Introductory Statistics. Mathematically, power is 1 – beta. The power of a hypothesis test is between 0 and 1; if the power is close to 1, the hypothesis test is very good at detecting a false null hypothesis.