α and β errors

There are inevitably limitations in estimating the population from a sample survey, and there is always the possibility of incorrect estimation. This incorrect estimation is called error, and there are two types of error: α error and β error.

The α error is "an error in concluding that there is a significant difference when there is really no difference," and the probability of making this error is denoted by α. For example, a 5% probability of an α error is interpreted as "a 5% probability of concluding that there is a significant difference even though there is no real difference.

The β error is "to conclude that there is no significant difference when there really is a difference," and the probability of making this error is denoted by β. For example, a 10% probability of β error is interpreted as "a 10% probability of concluding that there is no significant difference even though there is a real difference.

The α error and β error can be explained in terms of drug efficacy in a clinical trial as follows

• α error: The test is significant even though the drug does not really have an effect.
• β error: The test does not reach significance even though there is actually a drug effect.

Result of the test / true drug effect	Drug effect	No drug effect
Significant	OK	α error
Not significant	β error	OK

Statistical Power

A term that often comes up when discussing β errors is Statistical Power. Statistical Power is "the probability of concluding that there is a difference when there really is a difference. It is defined as 1 - \beta. For example, if the β error is 10%, the power is 90%. A higher power can be interpreted as a stricter test.

Disadvantages of α and β errors

When extending the conclusions obtained from the results of a sample survey to the population, α and β errors occur probabilistically. α and β are probabilities, and the degree to which they are acceptable depends on the trial under study and the circumstances. Below is an example of a clinical trial and anomaly detection.

For clinical trials

In clinical trials, α and β errors are typically set as follows

• α error: 1 - 5%
• β error: 10 - 20%

α errors are treated more severely than β errors. The reason for this is that α errors have a greater social impact.

An α error is "an error that erroneously concludes that a drug has a medicinal effect when it does not. If a drug with no efficacy is distributed in the world, it will result in the loss of a huge amount of public money. In addition, if the drug has a risk of side effects, it will be to the detriment of the patient. Therefore, the α error is a probability that needs to be small in clinical trials.

On the other hand, a β error is "an error in erroneously concluding that a drug has no efficacy when it does. In this case, the drug is not approved by the pharmaceutical affairs bodies despite the fact that it has a medicinal effect, which is a disadvantage to the company. However, the impact on society as a whole is not as great as the α error. Therefore, the tolerance range for β errors is wider than for α errors.

In the case of anomaly detection

Consider the detection of abnormalities in a certain product. α and β errors are as follows.

• α error: Error in which a normal product is mistakenly regarded as abnormal.
• β error: Errors that are mistakenly regarded as normal for products that are not normal.

In such cases, β errors are treated more severely than α errors because β errors are more problematic than α errors.