analysis cs education math mathematics medicalstats metrics power research statistica statistical.power statistics statistics_poweranalysis stats statsterms wp

My tags:

From Wikipedia, the free encyclopedia

The power of a statistical test is the probability that the test will reject a false null hypothesis (that it will not make a Type II error). As power increases, the chances of a Type II error decrease. The probability of a Type II error is referred to as the false negative rate (β). Therefore power is equal to 1 − β.

Power analysis can be used to calculate the minimum sample size required to accept the outcome of a statistical test with a particular level of confidence (power).

Contents 1 A priori vs. post hoc analysis 2 Background 3 Application 4 See also 5 Notes 6 References 7 External links

[edit] A priori vs. post hoc analysis

Power analysis can either be done before (a priori or prospective power analysis) or after (post hoc or retrospective power analysis) data is collected. A priori power analysis is conducted prior to the research study, and is typically used to determine an appropriate sample size to achieve adequate power. Post-hoc power analysis is conducted after a study has been completed, and uses the obtained sample size and effect size to determine what the power was in the study, assuming the effect size in the sample is equal to the effect size in the population. Whereas the utility of prospective power analysis in experimental design is universally accepted, the usefulness of retrospective techniques is controversial ^[1].

[edit] Background

Statistical tests use data from samples to determine if differences or similarities exist in a population. That is, do the criteria for selecting the samples divide the population into statistically distinct sub-populations. For example, to test the null hypothesis that the mean scores of men and women on a test do not differ, samples of men and women are drawn, the test is administered to them, and the mean score of one group is compared to that of the other group using a statistical test. The power of the test is the probability that the test will find a statistically significant difference between men and women, as a function of the size of the true difference between those two populations. Note that power is the probability of finding a difference that does exist, as opposed to the likelihood of declaring a difference that does not exist (which is known as a Type I error).

Statistical power depends on:

• the statistical significance criterion used in the test
• the size of the difference or the strength of the similarity (that is, the effect size) in the population
• the sensitivity of the data.

A significance criterion is a statement of how unlikely a result must be, if the null hypothesis is true, to be considered significant. The most commonly used criteria are probabilities of 0.05 (5%, 1 in 20), 0.01 (1%, 1 in 100), and 0.001 (0.1%, 1 in 1000). If the criterion is 0.05, the probability of the difference must be less than 0.05, and so on. One way to increase the power of a test is to increase (that is, weaken) the significance level. This increases the chance of obtaining a statistically significant result (rejecting the null hypothesis) when the null hypothesis is false, that is, reduces the risk of a Type II error. But it also increases the risk of obtaining a statistically significant result when the null hypothesis is in fact true; that is, it increases the risk of a Type I error.

Calculating the power requires first specifying the effect size that the (non-null) hypothesis expects to detect. The greater the effect size, the greater the power. (Remember that greater power means higher likelihood of getting a statistically significant result, which could still be the null hypothesis.)

Sensitivity can be increased by using statistical controls, by increasing the reliability of measures (as in psychometric reliability), and by increasing the size of the sample. Increasing sample size is the most commonly used method for increasing statistical power.

A common misconception by those new to statistical power is that power is a property of a study or experiment. In reality any statistical result that has a p-value has an associated power. For example, in the context of a single multiple regression, there will be a different level of statistical power associated with the overall r-square and for each of the regression coefficients. When determining an appropriate sample size for a planned study, it is important to consider that power will vary across the different hypotheses.

There are times when the recommendations of power analysis regarding sample size will be inadequate. Power analysis is appropriate when the concern is with the correct acceptance or rejection of a null hypothesis. In many contexts, the issue is less about determining if there is or is not a difference but rather with getting a more refined estimate of the population effect size. For example, if we were expecting a population correlation between intelligence and job performance of around .50, a sample size of 20 will give us approximately 80% power (alpha = .05, two-tail). However, in doing this study we are probably more interested in knowing whether the correlation is .30 or .60 or .50. In this context we would need a much larger sample size in order to reduce the confidence interval of our estimate to a range that is acceptable for our purposes. These and other considerations often result in the recommendation that when it comes to sample size, "More is better!"

[edit] Application

Funding agencies, ethics boards and research review panels frequently request that a researcher perform a power analysis, for example to determine the minimum number of animal test subjects needed for an experiment. If a study is inadequately powered, there is no point in completing the research. Although there are no formal standards for power, most researchers who assess the power of their tests use 0.80 as a standard for adequacy.

[edit] See also

• Effect size
• Sample size
• Neyman-Pearson lemma

[edit] Notes

1. ^ Thomas, L. (1997) Retrospective power analysis. Conservation Biology 11(1):276-280

[edit] References

• Cohen, J.: Statistical Power Analysis for the Behavioral Sciences. (2nd ed.) 1988. ISBN 0-8058-0283-5.

[edit] External links

• Hypothesis Testing and Statistical Power of a Test
• G*Power – A free program for Statistical Power Analysis
• R/Splus package of power analysis functions along the lines of Cohen (1988)
• Free A-priori Sample Size Calculator for Multiple Regression from Daniel Soper's Free Statistics Calculators website. Computes the minimum required sample size for a study, given the alpha level, the number of predictors, the anticipated effect size, and the desired statistical power level.

v • d • e Statistics Design of experiments Population • Sampling • Stratified sampling • Replication • Blocking Sample size estimation Null hypothesis • Alternative hypothesis • Type I and Type II errors • Statistical power • Effect size • Standard error Descriptive statistics Continuous data Location Mean (Arithmetic, Geometric, Harmonic) • Median • Mode Dispersion Range • Standard deviation • Coefficient of variation • Percentile Moments Variance • Semivariance • Skewness • Kurtosis Categorical data Frequency • Contingency table Inferential statistics Bayesian inference • Frequentist inference • Hypothesis testing • Significance • P-value • Interval estimation • Confidence interval • Meta-analysis General estimation Bayesian estimator • Maximum likelihood • Method of moments • Minimum distance • Maximum spacing Specific tests Z-test (normal) • Student's t-test • Chi-square test • F-test • Sensitivity and specificity Survival analysis Survival function • Kaplan-Meier • Logrank test • Failure rate • Proportional hazards models Correlation Pearson product-moment correlation coefficient • Rank correlation (Spearman's rho, Kendall's tau) • Confounding variable Linear models General linear model • Generalized linear model • Analysis of variance • Analysis of covariance Regression analysis Linear regression • Nonlinear regression • Nonparametric regression • Semiparametric regression • Logistic regression Statistical graphics Bar chart • Biplot • Box plot • Control chart • Forest plot • Histogram • Q-Q plot • Run chart • Scatter plot • Stemplot History History of statistics • Founders of statistics • Timeline of probability and statistics Publications Journals in statistics • Important publications Category • Portal • Topic outline • List of topics