Research Methods and Statistics in Psychology

This chapter answers Section XH5.1 of the Humanities and Social Sciences (XH) paper, “Research Methods and Statistics”, which is Section XH5.1 of XH5 Psychology, one of the six optional sections chosen at the time of application. It teaches the philosophical worldviews behind research; quantitative, qualitative and mixed-methods designs; how a study is designed (problems, variables, operational definitions, hypotheses and sampling); the data-collection methods from interview and survey to experiment, case study and ethnography; research ethics; descriptive statistics (central tendency, dispersion and the normal curve); inferential statistics (parametric and non-parametric tests, effect size and power); correlation, regression and factor analysis; and experimental designs with the ANOVA family. The numericals are correlation coefficients, z-scores, the standard error, degrees of freedom and Cohen’s d.

1. Worldviews and the three research designs

A research worldview is the set of beliefs a researcher brings about what reality is and how it can be known. Four are standard. Postpositivism holds that there is a reality we can approach but never fully capture; it is deterministic, reductionist and verifies theories through objective measurement, and it underlies quantitative research. Constructivism holds that people build multiple subjective meanings of their experience, so the researcher works inductively from the participants’ views; it underlies qualitative research. The transformative worldview foregrounds power, inequality and change for marginalised groups. Pragmatism is problem-centred and pluralistic, choosing whatever works for the research question, and it underlies mixed-methods research.

  • Quantitative research tests theories by measuring variables as numbers and analysing them with statistics; it is deductive, prefers large representative samples, and prizes generalisability, reliability and control.
  • Qualitative research explores meaning through words, images and observation; it is inductive, uses small purposive samples, and builds themes from rich context rather than testing a pre-set hypothesis.
  • Mixed-methods research combines both in one study. The main types are convergent (collect both at once and compare), explanatory sequential (numbers first, then words to explain them) and exploratory sequential (words first to build an instrument, then numbers).
🎯 Worldview drives the design
The choice between counting and understanding is not a matter of taste: a question about how widespread a belief is calls for measurement and a quantitative design, while a question about what a belief means to those who hold it calls for a qualitative design. The worldview, the question and the method must line up, which is why the purpose statement is written before the method is chosen.

2. Designing a study: problems, variables, hypotheses, sampling

A study begins with a research problem (the gap or puzzle), narrowed into a purpose statement and then into testable research questions or hypotheses. A variable is any characteristic that varies. The independent variable (IV) is manipulated or classified, the dependent variable (DV) is measured as the outcome, a confounding (extraneous) variable offers a rival explanation, a mediator carries the effect of the IV on the DV, and a moderator changes the strength or direction of that effect. An operational definition states a construct in terms of the exact operations used to measure it, for example defining anxiety as the score on a named scale.

A hypothesis is a predicted relationship. The null hypothesis (H₀) states no effect or no difference; the alternative (H₁) states that there is one. A directional hypothesis predicts the sign of the effect (tested one-tailed), a non-directional one predicts only that there is an effect (tested two-tailed). Testing is set up to reject H₀, never to prove H₁ true.

  • Probability sampling gives every unit a known non-zero chance of selection, so results generalise: simple random, systematic (every k-th unit), stratified (sample within each subgroup) and cluster (sample whole groups).
  • Non-probability sampling has no such guarantee and limits generalisation: convenience, purposive (chosen for a trait), quota and snowball (referrals). Purposive sampling is the norm in qualitative work.
⚠️ Population and sample, parameter and statistic
A number computed on the whole population is a parameter (written with Greek letters, μ and σ); the same number computed on a sample is a statistic (M and s) and estimates the parameter with some error. Confusing the two, or treating a convenience sample as if it generalised, are the common design slips.

3. Methods of data collection and research ethics

  • Interview (structured, semi-structured or unstructured) and the self-completion questionnaire or survey gather self-report; surveys reach many people cheaply but depend on honest, accurate recall.
  • Observation records behaviour directly — participant or non-participant, in a natural (naturalistic) or controlled setting; it avoids self-report bias but risks observer bias and reactivity.
  • The experiment manipulates an IV, controls other variables and uses random assignment, so it alone can support a causal claim. A quasi-experiment compares pre-existing groups with no random assignment, so causal inference is weaker.
  • Field studies observe behaviour in its real setting; focus group discussions gather group talk on a topic; narratives study life stories; case studies explore one person, group or event in depth; ethnography immerses the researcher in a culture over a long period.

Research ethics protect participants and the integrity of the record. Core duties are informed consent, confidentiality and anonymity, protection from physical and psychological harm, the right to withdraw at any time, and honest reporting. Where mild deception is unavoidable it must be justified and followed by a full debriefing. Proposals are reviewed by an ethics board (an IRB). Reporting ethics forbid fabrication, falsification and plagiarism of data.

ℹ️ Only the experiment licenses ‘causes’
Observation, survey and correlational designs can show that two things go together, but only a true experiment with manipulation and random assignment rules out rival explanations well enough to say one thing causes another. A quasi-experiment is used when random assignment is impossible or unethical, and its conclusions are hedged accordingly.

4. Descriptive statistics and the normal curve

Measures of central tendency summarise the typical value. The mean uses every score and is best for symmetric interval data; the median is the middle value and resists outliers, so it is preferred for skewed data; the mode is the most frequent value and is the only one usable for nominal data. Measures of dispersion summarise spread: the range (max minus min), the variance (mean squared deviation), the standard deviation (SD) (its square root, in the original units) and the inter-quartile range (middle 50%).

The normal probability curve is a symmetric, bell-shaped, asymptotic distribution in which mean, median and mode coincide at the centre. Under it the empirical rule holds: about 68% of scores fall within ±1 SD of the mean, about 95% within ±2 SD, and about 99.7% within ±3 SD. Skewness measures asymmetry (a positive skew has a long right tail, which pulls the mean above the median) and kurtosis measures the heaviness of the tails.

A z-score (standard score) re-expresses a raw score as its distance from the mean in SD units: z = (X − μ) / σ. A positive z is above the mean, a negative z below it; the mean of z-scores is 0 and their SD is 1. Because the standard normal table links each z to an area, z-scores turn any normal score into a percentile.

5. Inferential statistics: tests, effect size and power

Significance testing asks how likely the observed data would be if H₀ were true. That likelihood is the p-value; if it falls below the chosen significance level α (usually 0.05), H₀ is rejected. A one-tailed test places all of α in one direction and is used only for a directional hypothesis; a two-tailed test splits α between both tails. A Type I error is rejecting a true H₀ (its probability is α); a Type II error is failing to reject a false H₀ (its probability is β). Power = 1 − β is the chance of detecting a real effect, and it rises with sample size, effect size and α.

Parametric tests assume the data are interval or ratio, roughly normal and have homogeneous variances; they include the t-test, ANOVA and Pearson r, and are more powerful when their assumptions hold. Non-parametric tests make no distributional assumption and suit ordinal or nominal data: the Mann–Whitney U, Wilcoxon signed-rank, Kruskal–Wallis, chi-square and Spearman’s rho. For an independent-samples t-test the degrees of freedom are n₁ + n₂ − 2; for a chi-square test of independence they are (rows − 1)(columns − 1).

A significant result says an effect is unlikely to be zero, not that it is large, so an effect size is reported alongside. Cohen’s d = (M₁ − M₂) / SD_pooled scales a mean difference by the spread (about 0.2 small, 0.5 medium, 0.8 large). The standard error of the mean, SE = SD / √n, measures how much a sample mean would vary from sample to sample and shrinks as n grows, which is why large samples give tighter estimates.

6. Correlation, regression and factor analysis

Correlation measures how two variables vary together, from −1 (perfect inverse) through 0 (none) to +1 (perfect direct). The Pearson product-moment r suits two continuous, linearly related variables and equals the covariance divided by the product of the SDs: r = cov(X,Y) / (s_x s_y). The Spearman rank-order rho applies the same idea to ranks and captures any monotonic relationship, so it suits ordinal data. Partial correlation measures the link between two variables while statistically holding a third constant; multiple correlation R measures how well several predictors together relate to one outcome.

Special correlation coefficients by the nature of the two variables
CoefficientVariable 1Variable 2
Point-biserialContinuousTrue (natural) dichotomy
BiserialContinuousArtificial dichotomy (really continuous)
Phi coefficientTrue dichotomyTrue dichotomy
TetrachoricArtificial dichotomyArtificial dichotomy

Regression turns a correlation into prediction. Simple linear regression fits the line Ŷ = a + bX by least squares (minimising the squared prediction errors), where b is the slope and a the intercept. Multiple regression predicts one outcome from several predictors and reports each predictor’s unique weight. Factor analysis reduces many correlated variables to a few latent factors; it assumes the variables are inter-correlated and the sample large enough (checked by the KMO measure and Bartlett’s test). Factors are extracted (for example by principal-axis factoring), kept by the eigenvalue-over-1 rule or a scree plot, and then rotated — orthogonal (varimax, uncorrelated factors) or oblique (oblimin, correlated factors) — to make the loadings interpretable.

⚠️ Pick the coefficient by the variable type
The special coefficients differ only in whether each variable is continuous, a true dichotomy or a dichotomy forced on an underlying continuum. Point-biserial pairs a continuous variable with a true two-category one (such as male/female); biserial is used when that two-category variable is really continuous but scored as two groups. Tetrachoric and phi are the all-dichotomous pair, artificial and true respectively.

7. Experimental designs and the ANOVA family

  • A factorial design crosses two or more IVs, so it yields a main effect for each IV and an interaction that shows whether the effect of one IV depends on the level of the other; a 2 × 3 design has six cells.
  • A randomized block design groups subjects into homogeneous blocks on a nuisance variable (such as age) and randomises treatments within each block, removing that variable’s noise.
  • A repeated-measures (within-subjects) design tests the same people under every condition, which controls individual differences but introduces order effects, handled by counterbalancing.
  • A single-subject design studies one participant across phases, as in the A-B-A-B reversal (baseline, treatment, withdrawal, treatment). A cohort study follows a group sharing a characteristic forward over time, a longitudinal rather than experimental design.

The ANOVA family tests mean differences. ANOVA (analysis of variance) compares three or more group means with an F-ratio = variance between groups / variance within groups; a large F signals that the group means differ by more than chance, and a significant F is followed by post-hoc tests to locate the difference. MANOVA extends this to two or more dependent variables tested together. ANCOVA adds a covariate — a continuous nuisance variable statistically removed before comparing the means — which increases power and adjusts for pre-existing differences.

Key takeaways

  • Four worldviews map to the designs: postpositivism to quantitative (deductive, numeric, generalising), constructivism to qualitative (inductive, meaning, purposive samples), pragmatism to mixed methods; the transformative worldview centres power and change.
  • H₀ states no effect; a directional hypothesis is tested one-tailed and a non-directional one two-tailed. Probability sampling (random, systematic, stratified, cluster) generalises; non-probability sampling (convenience, purposive, quota, snowball) does not. Only a true experiment with random assignment licenses a causal claim.
  • Mean suits symmetric data, median resists skew, mode fits nominal data. The normal curve carries the 68–95–99.7 rule and z = (X − μ)/σ. A positive skew pulls the mean above the median.
  • Set α before testing; a Type I error rejects a true H₀ (rate α) and a Type II error misses a real effect (rate β), with power = 1 − β. Parametric tests assume normality; non-parametric tests do not. Report an effect size (Cohen’s d) with every significant result; SE = SD/√n.
  • Pearson r suits linear continuous data, Spearman rho suits ranks; partial correlation controls a third variable. The special coefficients go by variable type: point-biserial (continuous × true dichotomy), biserial (continuous × artificial dichotomy), phi (two true dichotomies), tetrachoric (two artificial). Regression predicts by least squares; factor analysis extracts and rotates latent factors.
  • Factorial designs give main effects and interactions; randomized block controls a nuisance variable; repeated measures controls individual differences but needs counterbalancing; single-subject uses A-B-A-B. ANOVA compares ≥3 means by F = between/within variance, MANOVA adds multiple DVs, ANCOVA removes a covariate.

Practice questions (16)

Attempt each one before opening the answer. Every explanation names the tempting wrong option as well as the right one, because that is where marks are lost.

  1. A researcher wants to understand, in depth, what the experience of chronic pain means to the people who live with it, using long open-ended interviews and building themes from their accounts. Which worldview and design best fit this aim?

    1. A constructivist worldview with a qualitative design, because the aim is to interpret the participants’ own subjective meanings
    2. A postpositivist worldview with a quantitative design, because the data come from structured measurement instruments
    3. A postpositivist worldview with a mixed-methods design, because any set of interviews must include a numeric measurement component in order to be considered scientifically valid
    4. A transformative worldview with an experimental design, because the study manipulates the participants’ level of pain
    Show answer

    Answer: A — A constructivist worldview with a qualitative design, because the aim is to interpret the participants’ own subjective meanings

    Exploring in-depth subjective meaning through open interviews and inductive theme-building is the hallmark of constructivism and qualitative research. The quantitative option is wrong because the study measures nothing numerically; the mixed-methods option invents a false requirement that interviews need numbers; and the experimental option is impossible and unethical here, as pain is not being manipulated.

  2. In a study, the effect of a training programme on performance is stronger for younger employees than for older ones. In this sentence, age is functioning as which kind of variable?

    1. The dependent variable, because it is the outcome that the training programme is designed to change
    2. A mediator, because it is the mechanism through which the training produces its effect on performance
    3. A moderator, because it changes the strength of the relationship between the training and performance
    4. A confounding variable, because it provides a complete rival explanation that invalidates the whole study
    Show answer

    Answer: C — A moderator, because it changes the strength of the relationship between the training and performance

    A variable that changes the strength or direction of an IV–DV relationship is a moderator, which is exactly what age does here. A mediator would be an intervening step that carries the effect, not a trait that strengthens or weakens it; performance, not age, is the dependent variable; and a confound would be an uncontrolled cause of the outcome, not a described interaction.

  3. A team compares reading scores of children in two existing schools, one of which already uses a new curriculum, with no random assignment of children to schools. What is the most accurate label for this design?

    1. A true experiment, because there is a clear comparison between a treatment and a control group
    2. A quasi-experiment, because groups are compared on a treatment but membership was not randomly assigned
    3. An ethnography, because the researchers are studying children within their own school culture
    4. A focus group discussion, because the data are gathered from a defined group of participants together
    Show answer

    Answer: B — A quasi-experiment, because groups are compared on a treatment but membership was not randomly assigned

    There is a manipulation-like contrast (curriculum vs none) but the children were not randomly assigned to conditions, which is the defining absence of a quasi-experiment. It is not a true experiment precisely because random assignment is missing; ethnography is long-term cultural immersion, not a two-group score comparison; and a focus group is a discussion method, not a between-school comparison of test scores.

  4. A small town’s household incomes are strongly positively skewed by a few very wealthy families. Which measure of central tendency best represents the income of a typical household, and why?

    1. The mode, because in income data the most frequently occurring value is always equal to the median
    2. The mean, because it uses every single income value and is therefore always the most representative average
    3. The range, because it captures the full distance from the poorest to the richest household in the town
    4. The median, because it is the middle value and is not dragged upward by the few extreme high incomes
    Show answer

    Answer: D — The median, because it is the middle value and is not dragged upward by the few extreme high incomes

    With strong positive skew the mean is pulled toward the extreme high values, so the median better represents the typical case. The mean option states a false universal claim; the range is a measure of spread, not central tendency; and the mode claim that it always equals the median is untrue, especially in skewed data.

  5. A researcher sets α = 0.05 and runs a two-tailed test. The true effect in the population is real but small, and the sample is also small. Which outcome is made more likely by these conditions specifically?

    1. A Type II error, because low power from a small sample and small effect makes failing to reject a false H₀ more likely
    2. A Type I error, because reducing the sample size automatically raises the probability of wrongly rejecting a true null hypothesis to a value above the chosen α
    3. A guaranteed significant result, because a real effect, however small, always produces p below 0.05
    4. A reduction in β to zero, because using a two-tailed test removes the possibility of missing the effect
    Show answer

    Answer: A — A Type II error, because low power from a small sample and small effect makes failing to reject a false H₀ more likely

    A small sample and a small true effect both lower statistical power (1 − β), which raises the chance of a Type II error: failing to reject a false H₀. The Type I claim is wrong because its rate is fixed at α regardless of sample size; a real effect does not guarantee significance when power is low; and a two-tailed test does not drive β to zero.

  6. An investigator correlates participants’ sex (recorded as male or female, a genuine two-category variable) with their continuous score on an entrance aptitude test. Which correlation coefficient is appropriate?

    1. The phi coefficient, which is used only when both of the variables are genuine two-category variables
    2. The tetrachoric correlation, which is used only when both variables are artificially dichotomised continua
    3. The point-biserial correlation, which pairs a continuous variable with a true (natural) dichotomy
    4. The Spearman rank-order correlation, which is used only when both variables are ranked ordinal measures
    Show answer

    Answer: C — The point-biserial correlation, which pairs a continuous variable with a true (natural) dichotomy

    One variable is continuous and the other is a true dichotomy, which is exactly the point-biserial case. Tetrachoric needs both variables to be artificial dichotomies; phi needs both to be true dichotomies; and Spearman needs both to be ranks — none matches a continuous-with-true-dichotomy pair.

  7. A design tests two independent variables — teaching method (3 levels) and feedback type (2 levels) — on exam score, and the researcher wants to know whether the best teaching method depends on which feedback type is used. Which design and which result answers that question?

    1. A 3 × 2 factorial design, answered by the main effect of teaching method considered entirely on its own
    2. A 3 × 2 factorial design, answered by the interaction effect between teaching method and feedback type
    3. A single-subject A-B-A-B design, answered by comparing the baseline and treatment phases of one learner
    4. A randomized block design, answered by the variance removed by blocking participants on their prior score
    Show answer

    Answer: B — A 3 × 2 factorial design, answered by the interaction effect between teaching method and feedback type

    Two crossed IVs make a 3 × 2 factorial design, and whether the effect of one IV depends on the level of the other is precisely the interaction. A main effect alone tells you the average effect of teaching method ignoring feedback, which does not answer the dependence question; a single-subject design has no second IV; and a randomized block design controls a nuisance variable rather than crossing two IVs.

  8. A psychologist must briefly mislead participants about the true purpose of a study to keep their behaviour natural. Which ethical safeguard specifically addresses the use of such deception?

    1. Using a one-tailed test, which is the recognised statistical correction for any study that involves deception
    2. Random assignment of participants, which by itself removes any ethical concern raised by deception
    3. Increasing the sample size, so that the harm done by deception is statistically averaged away to nothing
    4. A full debriefing afterwards, in which the real purpose and the reason for the deception are explained
    Show answer

    Answer: D — A full debriefing afterwards, in which the real purpose and the reason for the deception are explained

    Where mild deception is justified, a thorough debriefing that reveals the true purpose is the standard safeguard. Random assignment and sample size are design features that do nothing to undo deception, and a one-tailed test is a statistical choice with no bearing on ethics at all.

  9. Which of the following statements about significance testing and effect size are correct?

    1. The p-value is the probability of data at least as extreme as observed, computed assuming the null hypothesis is true
    2. Statistical power increases as the sample size, the true effect size and the chosen α increase
    3. A statistically significant result with a very large sample is automatically a practically important one
    4. Cohen’s d expresses a difference between two means in units of the pooled standard deviation
    Show answer

    Answer: A — The p-value is the probability of data at least as extreme as observed, computed assuming the null hypothesis is true; B — Statistical power increases as the sample size, the true effect size and the chosen α increase; D — Cohen’s d expresses a difference between two means in units of the pooled standard deviation

    The definition of the p-value, the determinants of power, and the definition of Cohen’s d are all correct. The claim that significance implies practical importance is false: a huge sample can make a trivial effect significant, which is exactly why an effect size is reported alongside the p-value.

  10. Which of the following statements about correlation, regression and factor analysis are correct?

    1. Partial correlation measures the association between two variables while statistically holding a third variable constant
    2. Simple linear regression fits its line by minimising the sum of the squared prediction errors
    3. Orthogonal rotation (such as varimax) produces factors that are allowed to correlate with one another
    4. A Pearson correlation of zero guarantees that the two variables have no relationship of any kind
    Show answer

    Answer: A — Partial correlation measures the association between two variables while statistically holding a third variable constant; B — Simple linear regression fits its line by minimising the sum of the squared prediction errors

    The definitions of partial correlation and of least-squares regression are correct. Orthogonal rotation forces factors to be uncorrelated (it is oblique rotation that allows correlation), and a Pearson r of zero only rules out a linear relationship — a strong curved relationship can still give r ≈ 0.

  11. Which of the following statements correctly distinguish members of the ANOVA family and related designs?

    1. ANCOVA statistically removes a continuous covariate before comparing group means, which can increase power
    2. MANOVA is used when there are two or more dependent variables to be tested together
    3. The F-ratio in a one-way ANOVA is the variance between groups divided by the variance within groups
    4. A repeated-measures design removes order effects automatically and therefore never needs counterbalancing
    Show answer

    Answer: A — ANCOVA statistically removes a continuous covariate before comparing group means, which can increase power; B — MANOVA is used when there are two or more dependent variables to be tested together; C — The F-ratio in a one-way ANOVA is the variance between groups divided by the variance within groups

    ANCOVA’s covariate adjustment, MANOVA’s multiple DVs, and the between-over-within F-ratio are all stated correctly. The repeated-measures claim is false and is the reverse of the truth: testing the same people in every condition creates order effects, which is exactly why counterbalancing is needed.

  12. On a normally distributed intelligence test the population mean is μ = 100 and the standard deviation is σ = 15. A person scores X = 130. Compute the person’s z-score. Give the answer as a plain number.

    Numerical answer — type the value.

    Show answer

    Answer: 2

    z = (X − μ) / σ = (130 − 100) / 15 = 30 / 15 = 2. The score is two standard deviations above the mean.

  13. A sample of n = 25 scores has a standard deviation of SD = 15. Compute the standard error of the mean using SE = SD / √n. Give the answer as a plain number.

    Numerical answer — type the value.

    Show answer

    Answer: 3

    SE = SD / √n = 15 / √25 = 15 / 5 = 3. Dividing by n instead of √n (giving 0.6) is the usual slip.

  14. Two independent groups are compared with a treatment group mean M₁ = 108, a control group mean M₂ = 100, and a pooled standard deviation of 16. Compute Cohen’s d = (M₁ − M₂) / SD_pooled. Give the answer to one decimal place.

    Numerical answer — type the value.

    Show answer

    Answer: 0.5

    d = (108 − 100) / 16 = 8 / 16 = 0.5, a medium effect by Cohen’s benchmarks.

  15. For two continuous variables, the covariance is cov(X,Y) = 12, the standard deviation of X is s_x = 4 and the standard deviation of Y is s_y = 5. Compute the Pearson product-moment correlation r = cov(X,Y) / (s_x s_y). Give the answer to one decimal place.

    Numerical answer — type the value.

    Show answer

    Answer: 0.6

    r = 12 / (4 × 5) = 12 / 20 = 0.6, a strong positive linear relationship within the valid range of −1 to +1.

  16. An independent-samples t-test compares a group of n₁ = 12 participants with a group of n₂ = 10 participants. Compute the degrees of freedom, df = n₁ + n₂ − 2. Give the answer as a plain number.

    Numerical answer — type the value.

    Show answer

    Answer: 20

    df = n₁ + n₂ − 2 = 12 + 10 − 2 = 20. The two means consume one degree of freedom each, so 2 are subtracted from the total of 22 scores.