The Meaning and Interpretation of P-values (what the data say?

The P-value, which directly depends on a given sample, attempts to provide a measure of the
strength of the results of a test, in contrast to a simple reject or do not reject. If the null
hypothesis is true and the chance of random variation is the only reason for sample differences,
then the P-value is a quantitative measure to feed into the decision making process as evidence.
The following table provides a reasonable interpretation of P-values:

P-value Interpretation
P< 0.01 very strong evidence against H0
0.01< = P < 0.05 moderate evidence against H0
0.05< = P < 0.10 suggestive evidence against H0
0.10< = P little or no real evidence against H0

This interpretation is widely accepted, and many scientific journals routinely publish papers
using such an interpretation for the result of test of hypothesis.

For the fixed-sample size, when the number of realizations is decided in advance, the distribution
of p is uniform (assuming the null hypothesis). We would express this as P(p < x) = x. That
means the criterion of p <0.05 achieves a of 0.05.

A p-value is a measure of how much evidence you have against the null hypothesis. The smaller
the p-value, the more evidence you have. One may combine the p-value with the significance
level to make decision on a given test of hypothesis. In such a case, if the p-value is less than
some threshold (usually .05, sometimes a bit larger like 0.1 or a bit smaller like .01) then you
reject the null hypothesis.

Understand that the distribution of p-values under null hypothesis H0 is uniform, and thus does
not depend on a particular form of the statistical test. In a statistical hypothesis test, the P value is
the probability of observing a test statistic at least as extreme as the value actually observed,
assuming that the null hypothesis is true. The value of p is defined with respect to a distribution.
Therefore, we could call it "model-distributional hypothesis" rather than "the null hypothesis".

In short, it simply means that if the null had been true, the p value is the probability against the
null in that case. The p-value is determined by the observed value, however, this makes it
difficult to even state the inverse of p.