Significance level (size) (of a test)

The significance level, also known as the size of a test, is the probability of rejecting the null hypothesis when it is actually true. It represents the threshold for determining whether a statistical result is significant.
Written by
Reviewed by
Updated on Jun 7, 2024
Reading time 4 minutes

3 key takeaways

Copy link to section
  • The significance level, denoted by alpha (α), is the probability of making a Type I error, which is rejecting a true null hypothesis.
  • Common significance levels are 0.05, 0.01, and 0.10, indicating a 5%, 1%, and 10% risk of a Type I error, respectively.
  • The choice of significance level affects the stringency of the test and the balance between Type I and Type II errors.

What is the significance level?

Copy link to section

The significance level of a test is a predetermined threshold used to decide whether to reject the null hypothesis. It is denoted by the Greek letter alpha (α) and typically set at values like 0.05, 0.01, or 0.10. A significance level of 0.05, for example, means there is a 5% risk of rejecting the null hypothesis when it is actually true.

How to interpret the significance level

Copy link to section

The significance level directly influences the decision-making process in hypothesis testing:

  • Type I error (α): This error occurs when the null hypothesis is rejected despite being true. The significance level represents the probability of making this error.
  • Type II error (β): This error happens when the null hypothesis is not rejected despite being false. The complement of the significance level (1 – α) does not directly indicate the probability of a Type II error but affects its likelihood.

A lower significance level (e.g., 0.01) means stricter criteria for rejecting the null hypothesis, reducing the chance of a Type I error but potentially increasing the risk of a Type II error.

Examples of significance level in practice

Copy link to section
  1. Clinical trials: A drug company tests a new medication against a placebo. Setting a significance level of 0.01 means there is only a 1% chance of concluding the drug is effective when it is not, ensuring high confidence in the results.
  2. Quality control: A manufacturer tests the quality of its products. Using a significance level of 0.05, the company can decide whether to reject batches that do not meet quality standards, with a 5% risk of mistakenly rejecting a good batch.

Choosing the significance level

Copy link to section

The choice of significance level depends on the context and consequences of errors in the specific field of study:

  • In scientific research: A 0.05 significance level is commonly used, balancing the risk of Type I errors with the need for practical results.
  • In medical research: A more stringent level (e.g., 0.01) might be chosen to minimize the risk of falsely concluding that a treatment is effective.
  • In exploratory research: A higher significance level (e.g., 0.10) may be acceptable when the emphasis is on identifying potential trends rather than confirming definitive results.

Impact of significance level on hypothesis testing

Copy link to section

The significance level affects the outcome and interpretation of statistical tests:

  • p-value comparison: The p-value obtained from the test is compared to the significance level. If the p-value is less than α, the null hypothesis is rejected.
  • Test power: Lowering the significance level decreases the likelihood of a Type I error but may reduce the power of the test, making it harder to detect a true effect.

Common misconceptions about significance level

Copy link to section
  • p-value interpretation: A p-value less than α does not prove that the null hypothesis is false; it only indicates that the observed data is unlikely under the null hypothesis.
  • Fixed threshold: The significance level is not a fixed standard and should be chosen based on the specific context and goals of the study.

Understanding the significance level is crucial for making informed decisions in hypothesis testing and interpreting the results of statistical analyses. For further exploration, one might study the relationship between significance level and test power, the trade-offs between Type I and Type II errors, and guidelines for setting appropriate significance levels in different research contexts.


Sources & references

James Knight

James Knight

Editor of Education

  • Stock Market
  • Cryptocurrencies
  • Commodities
  • Investing
  • Sport
James is the Editor of Education for Invezz, where he covers topics from across the financial world, from the stock market, to cryptocurrency, to macroeconomic markets. His main focus is on improving financial literacy among casual investors. He has been with Invezz since the start of 2021 and has been...