Lesson 1 · about 15 minutes · free preview

Most problems in research reports and theses have nothing to do with advanced methods. They are the same eight mistakes, again and again. Learn to spot them and your results become both correct and easy to read.

1. The test does not match the question

The research question asks whether three groups differ, and the paper reports three separate t tests. Or the outcome is yes/no and the paper runs a linear regression. A careful reader looks at the question first, then checks whether the test answers it.

WeakWe compared in-person, hybrid and online students with three independent t tests.
BetterWe compared the three formats with a one-way ANOVA, followed by Tukey HSD comparisons.

Lesson 2 gives you a one-page way to choose the test before you open SPSS.

2. Only a p value, no size of the effect

A p value tells the reader whether a result is surprising under the null hypothesis. It does not tell them whether the difference is large enough to matter. APA 7 asks for an effect size and, where possible, a confidence interval with every main test.

WeakExam scores differed by course format (p = .031).
BetterExam scores differed by course format, F(2, 197) = 3.55, p = .031, η2 = .035. The effect was small: format accounted for about 3.5% of the variation in scores.

3. SPSS output pasted straight into the paper

SPSS shows very small p values as .000. A p value is never zero, so "p = .000" shows the table came straight from the output window. Other signs: column labels such as Sig. (2-tailed), five decimal places, and all the default rows left in.

Weakt = 7.672, df = 199, Sig. = .000
Bettert(199) = 7.67, p < .001

4. Assumptions never mentioned

Every test rests on assumptions about the data. You do not need a paragraph on each, but the reader needs one or two sentences saying what you checked and what you did about it. Saying nothing leaves the reader to assume nothing was checked.

Weak(No mention of assumptions.)
BetterQ–Q plots showed no marked departure from normality within groups, and Levene's test did not indicate unequal variances (p = .72).

5. The sample size changes and nobody says why

Table 1 says 200 participants, the regression has 193, and the correlation has 196. Each analysis drops cases with missing values, which is SPSS's default. That is acceptable, but it has to be stated, with the numbers.

Weak(N = 200 in the method section; df of 189 in the regression with three predictors, unexplained.)
BetterSeven students (3.5%) had a missing value on sleep or study hours; analyses used all available cases, so n ranges from 193 to 200.

6. Many tests, one lucky result

Run twenty tests at the .05 level and you should expect about one "significant" result by chance alone. A long list of comparisons with only the one that "worked" discussed is a classic warning sign. State in advance which tests answer your questions, and when you make several comparisons within one question, use a correction (Tukey, Bonferroni or Holm).

7. Causal words for a correlational design

If you did not assign people to conditions, you cannot say one variable "improved", "increased" or "led to" another. This is the most common wording mistake in research reports.

WeakMore sleep improved exam performance.
BetterStudents who reported more sleep tended to have higher exam scores, r(194) = .25, p < .001.

8. Spinning a non-significant result

"Marginally significant", "approaching significance" and "a trend toward" are red flags. So is the opposite mistake: treating a non-significant result as proof that there is no difference. Report it plainly, with the effect size and interval, and let the reader see how precise the estimate is.

WeakWorkshop attendees scored marginally higher (p = .48), suggesting the workshop works.
BetterWe found no clear difference between attendees and non-attendees, t(197.23) = 0.71, p = .478, d = 0.10, 95% CI [−0.18, 0.38]. The interval is consistent with no effect or a small one in either direction.

Try it now

Open the results section of your current paper or thesis chapter. Go through the eight points above and mark each one: fine, needs work, or not applicable. Most people find at least three.

All numbers in this course come from the course practice dataset (200 invented students). You can download it in the full course and reproduce every result in SPSS yourself.

This is Lesson 1 of both statistics courses

The full courses add six or seven more lessons, a practice dataset and the Reporting Guide. Choose the software you use:

With SPSS (US$49)   With jamovi (US$49)