Decision making problems based on p-valiue and significance level in applied research

• О.В. Максимова

  ФГБУ «Институт глобального климата и экологии имени академика Ю.А. Израэля», Россия, 107258, Москва, ул. Глебовская,20Б;

  Автор

• V.L. Shper

  University of Science and Technology (MISIS), 4, Leninsky pr., Moscow, 119049, Russian Federation

  Автор

The correct interpretation of p-value-based research results has
been the subject of intense debate in the scientific community in recent decades. This
paper examines two questions: what is the appropriate guideline for choosing a
significance level when testing hypotheses, and how hypothesis testing using a
significance level differs qualitatively from using a p-value. It is demonstrated that
choosing a significance level is the responsibility of the researcher, has nonstatistical
justifications, and depends on the type of the study. It is shown that
decision-making algorithms based on p-value and significance level are different.
Furthermore, the paper examines errors in qualitative inference when making
decisions based on both significance level and p-value. The main conclusion of this
paper is that the idea that a p-value or significance level can reflect both the longterm
results of an experiment and be the proof of an individual sample result is
erroneous. Neither p-value nor significance level verifies or evaluates the
probability of the truthiness or falseness of the hypothesis being tested. To ensure
more reliable conclusions, it is necessary to verify the results every time possible
with additional experiments as well as take into account the results of previous
studies, understand the limitations of the results found, and take into consideration
the variability of all processes.

• PDF (Russian)