The fallacy of null hypothesis rejection

I quote from: Jacob Cohen, The Earth Is Round (p < .05), American Psychologist 49, no. 12 (1994), 997-1003:

One problem arises from a misapplication of deductive syllogistic reasoning. Falk and Greenbaum (in press) called this the “illusion of probabilistic proof by contradiction” or the “illusion of attaining improbability.”  […]   It is the widespread belief that the level of significance at which Ho is rejected, say .05, is the probability that it is correct or, at the very least, that it is of low probability.

The following is almost but not quite the reasoning of null hypothesis rejection:

If the null hypothesis is correct, then this datum (D) can not occur.

It has, however, occurred.

Therefore, the null hypothesis is false.

If this were the reasoning of Ho testing, then it would be formally correct. It would be what Aristotle called the modus tollens, denying the antecedent by denying the

consequent. But this is not the reasoning of NHST. Instead, it makes this reasoning probabilistic, as follows:

If the null hypothesis is correct, then these data are highly unlikely.

These data have occurred.

Therefore, the null hypothesis is highly unlikely.

By making it probabilistic, it becomes invalid. Why? Well, consider this: The following syllogism is sensible and also the formally correct modus tollens:

If a person is a Martian, then he is not a member of Congress.

This person is a member of Congress.

Therefore, he is not a Martian.

Sounds reasonable, no? This next syllogism is not sensible because the major premise is wrong, but the reasoning is as before and still a formally correct modus

tollens:

If a person is an American, then he is not a member of Congress.

(WRONG!)

This person is a member of Congress.

Therefore, he is not an American.

If the major premise is made sensible by making it probabilistic, not absolute, the syllogism becomes formally incorrect and leads to a conclusion that is not sensible:

If a person is an American, then he is probably not a member

of Congress. (TRUE, RIGHT?)

This person is a member of Congress.

Therefore, he is probably not an American. (Pollard &

Richardson. 1987)

This is formally exactly the same as

If Ho is true, then this result (statistical significance) would

probably not occur.

This result has occurred.

Then Ho is probably not true and therefore formally invalid.

This formulation appears at least implicitly in article after article in psychological journals and explicitly in some statistics textbooks—”the illusion of attaining improbability.”

(Quoted from: Jacob Cohen, The Earth Is Round (p < .05), American Psychologist 49, no. 12 (1994), 997-1003.)