Hypothesis Tests

Hypothesis testing is used to decide whether an influence or change is significant or not based on samples. The core of the problem is that it is not known how far the sample deviates from the real process. Even representative samples have random scatter.

A complicating factor is that the samples should be relatively small for reasons of time and cost. There is a risk that a sample is obtained that is not typical for the process and thus a false statement is derived. The branch of "inferential statistics" makes it possible, under some assumptions, to determine the probability (risk) of such a deviating sample and thus of a false statement.

Hypothesis tests translate a practical problem into the language of statistics.

The hypotheses to be named always occur in pairs: There is a null hypothesis H₀ and an alternative hypothesis H_A.

The Null Hypothesis H₀ is the hypothesis of equality and contains exactly one case. The process before the change is equal to the process after the change. That is, the change had no influence on the process outcome.

The Alternative Hypothesis H_A is the hypothesis of inequality and includes all other cases. That is, the process before the change is better or worse than the process after the change. This means that the change had an influence on the process outcome.

The hypotheses always refer to the population of all process results.

We have selected new parameter settings for a process and now want to know whether the process has changed (H_A) or not (H₀).

Samples were taken from the process before and after the change. The hypothesis test should now prove which of the statements is true:

1. Formulating the hypotheses: Null hypothesis (H₀) and alternative hypothesis (H_A)
2. Determination of the a-Risk (often 5 %)
3. Carry out the hypothesis test, e.g. T-test, F-test or Chi²-test (depending on the type of data, parameters and question). Each hypothesis test provides a p-value as a result. The p-value is the probability of a random difference and thus the "real" alpha-Risk.
4. Taking the decision:

• If this probability is less than alpha: Reject H₀.  
• p < α = reject H₀, decision for H_A.
• If that probability is greater than alpha: keep H_0.
• p > α = keep H₀, decision against H_A

The alpha-Risk is therefore the probability that a difference is claimed that does not exist.

Of course, there is also the risk of overlooking a difference.
This probability is called beta-Risk.

Therefore, there is no decision without risk! However, the size of the risk is determined with the hypothesis tests and then included in the decision.