!emmp #grapevines analysis frequentism hypothesistesting importanttoknow lean_6sigma math mathematics mba mqg713 nima p-value pvalue pwert quant.research research stat statistica statistical_testing statistics statistik stats t-test words work

My tags:

From Wikipedia, the free encyclopedia

In statistical hypothesis testing, the p-value is the probability of obtaining a result at least as extreme as the one that was actually observed, assuming that the null hypothesis is true. The fact that p-values are based on this assumption is crucial to their correct interpretation.

More technically, a p-value of an experiment is a random variable defined over the sample space of the experiment such that its distribution under the null hypothesis is uniform on the interval [0,1]. Many p-values can be defined for the same experiment.

Contents 1 Coin flipping example 2 Interpretation 3 Frequent misunderstandings 4 See also 5 Additional reading 6 References 7 External links

[edit] Coin flipping example

For example, an experiment is performed to determine whether a coin flip is fair (50% chance of landing heads or tails) or unfairly biased, either toward heads (> 50% chance of landing heads) or toward tails (< 50% chance of landing heads). (A bent coin produces biased results.)

Since we consider both biased alternatives, a two-tailed test is performed. The null hypothesis is that the coin is fair, and that any deviations from the 50% rate can be ascribed to chance alone.

Suppose that the experimental results show the coin turning up heads 14 times out of 20 total flips. The p-value of this result would be the chance of a fair coin landing on heads at least 14 times out of 20 flips plus the chance of a fair coin landing on tails 14 or more times out of 20 flips. In this case the random variable T has a binomial distribution. The probability that 20 flips of a fair coin would result in 14 or more heads is 0.0577. By symmetry, the probability that 20 flips of the coin would result in 14 or more tails (alternatively, 6 or fewer heads) is the same, 0.0577. Thus, the p-value for the coin turning up the same face 14 times out of 20 total flips is 0.0577 + 0.0577 = 0.1154 .

[edit] Interpretation

Generally, one rejects the null hypothesis if the p-value is smaller than or equal to the significance level,^[1] often represented by the Greek letter α (alpha). If the level is 0.05, then results that are only 5% likely or less are deemed extraordinary, given that the null hypothesis is true.

In the above example we have:

• null hypothesis (H₀) — fair coin;
• observation (O) — 14 heads out of 20 flips; and
• probability (p-value) of observation (O) given H₀ — p(O | H₀) = 0.0577 × 2 (two-tailed) = 0.1154 (percentage expressed as 11.54%).

The calculated p-value exceeds 0.05, so the observation is consistent with the null hypothesis — that the observed result of 14 heads out of 20 flips can be ascribed to chance alone — as it falls within the range of what would happen 95% of the time were this in fact the case. In our example, we fail to reject the null hypothesis at the 5% level. Although the coin did not fall evenly, the deviation from expected outcome is just small enough to be reported as being "not statistically significant at the 5% level".

However, had a single extra head been obtained, the resulting p-value (two-tailed) would be 0.0414 (4.14%). This time the null hypothesis - that the observed result of 15 heads out of 20 flips can be ascribed to chance alone - is rejected. Such a finding would be described as being "statistically significant at the 5% level".

Critics of p-values point out that the criterion used to decide "statistical significance" is based on the somewhat arbitrary choice of level (often set at 0.05). A proposed replacement for the p-value is p-rep. It is necessary to use a reasonable null hypothesis to assess the result fairly. The choice of null hypothesis entails assumptions.

[edit] Frequent misunderstandings

The conclusion obtained from comparing the p-value to a significance level yields two results: either the null hypothesis is rejected, or the null hypothesis cannot be rejected at that significance level. You cannot accept the null hypothesis simply by the comparison just made (11% > 5%); there are alternative tests that have to be performed, such as some "goodness of fit" tests. It would be very irresponsible to conclude that the null hypothesis needs to be accepted based on the simple fact that the p-value is larger than the significance level chosen.

The use of p-values is widespread; however, such use has come under heavy criticism due both to its inherent shortcomings and the potential for misinterpretation.

There are several common misunderstandings about p-values.^[2][3]

1. The p-value is not the probability that the null hypothesis is true. (This false conclusion is used to justify the "rule" of considering a result to be significant if its p-value is very small (near zero).)
   In fact, frequentist statistics does not, and cannot, attach probabilities to hypotheses. Comparison of Bayesian and classical approaches shows that a p-value can be very close to zero while the posterior probability of the null is very close to unity. This is the Jeffreys-Lindley paradox.
2. The p-value is not the probability that a finding is "merely a fluke." (Again, this conclusion arises from the "rule" that small p-values indicate significant differences.)
   As the calculation of a p-value is based on the assumption that a finding is the product of chance alone, it patently cannot also be used to gauge the probability of that assumption being true. This is subtly different from the real meaning which is that the p-value is the chance that null hypothesis explains the result: the result might not be "merely a fluke," and be explicable by the null hypothesis with confidence equal to the p-value.
3. The p-value is not the probability of falsely rejecting the null hypothesis. This error is a version of the so-called prosecutor's fallacy.
4. The p-value is not the probability that a replicating experiment would not yield the same conclusion.
5. 1 − (p-value) is not the probability of the alternative hypothesis being true (see (1)).
6. The significance level of the test is not determined by the p-value.
   The significance level of a test is a value that should be decided upon by the agent interpreting the data before the data are viewed, and is compared against the p-value or any other statistic calculated after the test has been performed.
7. The p-value does not indicate the size or importance of the observed effect (compare with effect size).

[edit] See also

• Counternull
• Statistical hypothesis testing
• Binomial test

[edit] Additional reading

• Dallal GE (2007) Historical background to the origins of p-values and the choice of 0.05 as the cut-off for significance
• Hubbard R, Armstrong JS (2005) Historical background on the widespread confusion of the p-value (PDF)
• Fisher's method for combining independent tests of significance using their p-values
• Dallal GE (2007) The Little Handbook of Statistical Practice (A good tutorial)

[edit] References

1. ^ http://economics.about.com/od/termsbeginningwithp/g/pvaluedef.htm
2. ^ Sterne JAC, Smith GD (2001). "Sifting the evidence — what's wrong with significance tests?". BMJ 322 (7280): 226–231. doi:10.1136/bmj.322.7280.226. PMID 11159626. http://www.pubmedcentral.nih.gov/articlerender.fcgi?tool=pubmed&pubmedid=11159626. 
3. ^ Schervish MJ (1996). "P Values: What They Are and What They Are Not". The American Statistician 50 (3): 203-206. http://www.jstor.org/sici?sici=0003-1305(199608)50%3A3%3C203%3APVWTAA%3E2.0.CO%3B2-0. 

[edit] External links

• Free online p-values calculators for various specific tests (chi-square, fisher's F-test, etc).
• Understanding P-values, including a Java applet that illustrates how the numerical values of p-values can give quite misleading impressions about the truth or falsity of the hypothesis under test.

v • d • e Statistics Design of experiments Population • Sampling • Stratified sampling • Replication • Blocking Sample size estimation Null hypothesis • Alternative hypothesis • Type I and Type II errors • Statistical power • Effect size • Standard error Descriptive statistics Continuous data Location Mean (Arithmetic, Geometric, Harmonic) • Median • Mode Dispersion Range • Standard deviation • Coefficient of variation • Percentile Moments Variance • Semivariance • Skewness • Kurtosis Categorical data Frequency • Contingency table Inferential statistics Bayesian inference • Frequentist inference • Hypothesis testing • Significance • P-value • Interval estimation • Confidence interval • Meta-analysis General estimation Bayesian estimator • Maximum likelihood • Method of moments • Minimum distance • Maximum spacing Specific tests Z-test (normal) • Student's t-test • Chi-square test • F-test • Sensitivity and specificity Survival analysis Survival function • Kaplan-Meier • Logrank test • Failure rate • Proportional hazards models Correlation Pearson product-moment correlation coefficient • Rank correlation (Spearman's rho, Kendall's tau) • Confounding variable Linear models General linear model • Generalized linear model • Analysis of variance • Analysis of covariance Regression analysis Linear regression • Nonlinear regression • Nonparametric regression • Semiparametric regression • Logistic regression Statistical graphics Bar chart • Biplot • Box plot • Control chart • Forest plot • Histogram • Q-Q plot • Run chart • Scatter plot • Stemplot History History of statistics • Founders of statistics • Timeline of probability and statistics Publications Journals in statistics • Important publications Category • Portal • Topic outline • List of topics