Why p-values mislead with big samples By Arjun Guptap-values are one of the most popular measures used in research across different fields “Most scientists employ the p-value as if it were an absolute index of the truth. A low p-value is automatically taken as substantial evidence that the data support a real phenomenon.” Halsey, L. G., Curran-Everett, D., Vowler, S. L., & Drummond, G. B. (2015)But there is a problem we don’t always learn about the p-values. They change as the sample size changes as well! Take this study as an exampleHere is what Miola and Miot (2021) found ot A: Neon =3.8 = ===) HG op p =0.084 p= 0.084 Diff, = 1.0(22a0.1) 7 ot de soris =30 1 co 8 n=3 is SD=1.5 | i p =0.015 G1 (30) (c2(n=30)Even when the mean and SD were same, doubling the sample size changed the p-value obtained n=15 n=30 p =0.084. p =0.015 This is because as n increases, the p-values decreases considerably.So it is quite likely that you get a “statistically signifcant” result even though the difference between your two groups is negligible. Increasing sample size increases statistical power, and thus a progressively greater proportion of p values < 0.05 are obtained. Halsey, L. G., Curran-Everett, D., Vowler, S. L., & Drummond, G. B. (2015)This brings into question the “practical/clinical” significance of the differences we are seeing, not just their statistical significance. Statistical significance a Practical significance| wish there was one number we could use at all times to understand the degree of “truth” in research but the truth is complicated. It requires care and attention.There are multiple cogs in the wheel. The appropriate interpretation of all them gives us the best estimate of “reality”Follow for more research and stats content! Fopolelelotadhe ss adlos. iid