Methodological Advances to Enhance the Extraction of Policy-relevant Information in TIMSS and TIMSS-L

Studies 1–3 · Status and Findings

Henry I. Braun

Lynch School of Education, Boston College

Damian W. Betebenner

National Center for the Improvement of Educational Assessment

with Jihang Chen · Qi Meng · Kiara Sanchez

Research team

IEA · 18 August 2026

What Stakeholders Actually Ask

TIMSS reporting answers “How are our students doing?” with great authority. But the questions that drive policy are almost always about change:

Are our students learning more from Grade 4 to Grade 8 than a decade ago?
Is the gap between subgroups widening or closing as students progress?
Did a curriculum or tracking reform actually accelerate learning?
Which systems turn a given starting point into the strongest growth — and what might we learn from them?

Every one of these is a question about growth, not status.

A single snapshot — however well measured — cannot answer them. Growth is a statement about the same cohort across time.

Our charge: extract far more of the growth information policymakers need — from the assessments the IEA already fields.

What TIMSS Answers Well — and the One Thing It Cannot

TIMSS assesses Grade 4 and Grade 8 each cycle — but on independent samples. No student is measured at both grades.

We observe the Grade 4 distribution and the Grade 8 distribution.
We do not observe the link between them — how a given student’s standing at Grade 4 relates to their standing at Grade 8.
Cross-sectional design gives us two snapshots; growth lives in the connection we can’t see.

This is not a flaw to apologize for — it is a design choice with a real cost. Our aim is to recover the growth signal the design leaves implicit.

Why Growth Is the Missing Piece

Methodologists agree that cohort-longitudinal evidence is the strongest foundation for policy-relevant inference. Growth measures let us:

Move from description to diagnosis — not just whether achievement differs, but where in the distribution progress is strong or weak.
Surface equity patterns that averages hide — who is accelerating, who is falling behind, and at what starting points.
Support (cautious) causal reasoning about reforms, by comparing the growth of well-matched systems or subgroups.

More growth information raises the value of every TIMSS cycle.

For the IEA, that strengthens the benefit case for participation in both Grade 4 and Grade 8 — and for TIMSS-L.

These methods are exploratory instruments — designed to generate strong hypotheses for triangulation, not to settle causal questions on their own.

Three Studies, One Goal: Cohort Growth

One goal — cohort growth — pursued three ways. Study 1 (M/Q-Q) is complete, with an interim report delivered. Studies 2 and 3 are in progress; Study 3 unifies the other two through copula theory. The remainder of this session takes each in turn.

STUDY 1

M/Q-Q Plots

Matched-percentile growth comparisons from existing cross-sectional TIMSS

✓ Completed · interim report submitted 15 July

Study 1 — Statement of Work

Apply the M/Q-Q (matched Q-Q) plot methodology to a set of country case studies. The deliverables were expanded beyond the original proposal:

Three pairs of countries rather than two: Sweden / Finland, Lithuania / Hungary, Japan / Korea.
For one pair, growth comparisons overall and for all three cognitive subdomains — Knowing, Applying, Reasoning — in both Math and Science.
For two countries, growth comparisons (overall proficiency) between cohorts: [Grade 4 (2015) → Grade 8 (2019)] vs. [Grade 4 (2019) → Grade 8 (2023)].

This directly complements TIMSS’s own reporting. TIMSS reports compare cohorts within a grade; the M/Q-Q analysis adds the within-cohort, across-grade growth dimension the reports cannot.

How to read an M/Q-Q plot: each curve maps a country’s Grade-4 quantile to the matched-percentile Grade-8 quantile — an emulated cohort trajectory. When one country’s curve lies entirely above another’s, its students show uniformly accelerated progress across the whole distribution.

Study 1 — Status and Findings

Status: Completed

Work complete; interim report submitted 15 July 2026.

Across the case studies, the analyses show that in many instances one population exhibits “uniformly accelerated progress” relative to another — a clear, distribution-wide signal with direct implications for policymakers and educators.

The comparison is exploratory: it identifies contrasts worth investigating, to be triangulated with paradata and policy context.
The next two panels show Japan vs. Korea, overall proficiency in Science.

Why it matters: a single number (“average gain”) can hide who accelerated. The M/Q-Q curve shows the whole pattern — where in the distribution one system pulls ahead, and by how much.

Illustrative reading (next slide): Japan’s curve sits above Korea’s across nearly the entire Grade-4 score range — in both cohort pairs — indicating Japan’s uniformly accelerated progress in Science relative to Korea.

Study 1 — Japan vs. Korea: Uniformly Accelerated Progress (Science)

Japan (red) lies above Korea (green) across almost the entire score range — and the pattern replicates across both cohort pairs. Japan demonstrates uniformly accelerated progress relative to Korea in Science; the consistency across independent cohorts strengthens the signal.

STUDY 2

Quantile Regression on TIMSS-L

Modeling the full conditional distribution of growth where students are linked

● In progress

Study 2 — TIMSS-L: Growth We Can Actually See

TIMSS-L is the genuine longitudinal advance. It follows cohorts across Grades 4 → 5 and 8 → 9 — the same students measured twice — so growth is observed directly, not inferred.

For the first time in the TIMSS family, we have linked pairs: prior score and later score for each student.
This makes direct country-by-country growth estimates possible — overall and by subgroup.

TIMSS-L gives us the one thing cross-sectional TIMSS never could: the connection between two time points for a real cohort.

But it is new, covers short spans (one year), and — for now — a limited set of cycles and countries.
The vast majority of TIMSS evidence ever collected remains cross-sectional.

Study 2 — Statement of Work

Fit flexible quantile regressions to model the full conditional distribution of Time-2 scores given Time-1 scores, across the entire Time-1 score range.

Goes beyond average growth to a percentile-by-percentile description of cohort progress.
Sheds light on differential year-on-year growth across the baseline score scale — where growth is strong, where it stalls, and for whom.
Implemented via the Student Growth Percentile (Betebenner, 2009) spline approach — non-parametric, with no assumption of a common vertical scale.

Status: In progress

Estimation framework in place; country and subgroup analyses of TIMSS-L cohorts underway.

From “is the country growing?” to “which students are growing, by how much, and under what conditions?” — the granularity where equity-relevant, actionable insights live.

STUDY 3

Copula Integration

Sklar’s theorem to unify Studies 1 and 2 — growth on a probability scale, even without linkage

● In progress

Study 3 — Statement of Work

Study 3 exploits Sklar’s theorem, which uses copulas to model the dependence structure in multivariate distributions.

Task: employ selected copulas to integrate the M/Q-Q approach for pseudo-cohort studies (Study 1) with conditional growth modeling for longitudinal studies (Study 2) — yielding richer, policy-relevant insights from TIMSS and TIMSS-L.

Copula theory, combined with extensive empirical analysis, enables the direct estimation of growth on a probability scale — even in the absence of actual longitudinal data.

Status: In progress

Inference machinery built and validated on linked data; deployment to TIMSS underway.

The unifying idea: Study 1’s matched-percentile mapping and Study 2’s conditional distributions are both statements about the same object — the copula. Naming it explicitly lets us generalize both, and carry the result to data that was never linked.

Study 3 — A Joint Picture Is Margins + Coupling

Any relationship between two measurements decomposes cleanly into two separable parts:

Sklar’s Theorem (1959), in words

The margins — how scores are distributed at each grade — plus the copula — how the ranks at the two grades move together — fully determine the joint distribution.

Cross-sectional TIMSS gives us the margins (both grades observed).
The copula — the coupling — is the missing center panel.

Cross-sectional data give us the two margins; the joint that connects them (center) is unobserved.

The copula speaks only in ranks — percentiles, not scale points. That makes it stable across content areas, grade spans, and countries, and appropriate for scores whose meaning is fundamentally ordinal.

Study 3 — The Bridge

Study 3 — Unlocking the Archive

No new data collection required. A dependence structure established on linked longitudinal data — large state systems, benchmarked against TIMSS-L — is carried to every past cross-sectional cycle, turning thirty years of snapshots into cohort-growth evidence.

Study 3 — Early Empirical Validation

Before touching TIMSS, the machinery was tested on millions of linked records from large U.S. state longitudinal systems, where the true growth is known — then re-estimated as if the link had been hidden.

Inferred vs. true group growth: r = 0.995
Median error under half a percentile point (MAE ≈ 0.40)
Across 966 conditions, four datasets, four content areas

A single t-copula family fits ~64% of conditions and, with one alternate, covers >94% — the “coupling” really is remarkably stable.
The honest trade-off: without student links, confidence intervals widen ~2.5× — quantifiable, and still narrow enough for country- and subgroup-level comparison.

Inferred vs. true group-level growth across hundreds of subgroups — points on the diagonal indicate near-perfect recovery.

What This Means for the IEA

A stronger value proposition for participation:

Every past and future cross-sectional cycle becomes a source of cohort-growth insight — raising the return on data already collected.
TIMSS-L gains a second role: beyond its own longitudinal findings, it validates in the international setting the dependence structure drawn from state longitudinal data — the check that justifies applying it across the entire TIMSS archive.
New, equity-focused comparisons — across countries, subgroups, and performance levels — that current reporting cannot support.

The benefit/cost argument

More policy-relevant information per assessment dollar — strengthening the case for participation in Grade 4, Grade 8, and TIMSS-L together.

Offered as exploratory instruments: their outputs generate hypotheses for triangulation with paradata, technical documentation, and country-specific policy context — not stand-alone causal verdicts.

Where Things Stand — and What’s Next

Study 1 · M/Q-Q

✓ COMPLETED

Interim report delivered (15 July).
Finalize reporting across the three country pairs, subdomains, and cohort comparisons.

Study 2 · Quantile Regression

● IN PROGRESS

Complete SGP / quantile-regression analyses on TIMSS-L cohorts.
Country and subgroup growth comparisons; both content areas.

Study 3 · Copula

● IN PROGRESS

Deploy validated machinery on TIMSS 2019 & 2023 (Grade 4 → Grade 8).
Country growth regimes and rankings; benchmark against Study 1.

Studies 2 and 3 advance together: TIMSS-L results are compared with the state longitudinal analyses to settle on a sound, well-benchmarked dependence model for the archive.

Methodological Advances in TIMSS and TIMSS-L

Growth measurement no longer requires longitudinal data everywhere — it requires the margins we already have, and a dependence structure we can learn once.

Henry I. Braun · Lynch School of Education, Boston College · braunh@bc.edu

Damian W. Betebenner · NCIEA · dbetebenner@nciea.org

with the assistance of Jihang Chen · Qi Meng · Kiara Sanchez

Thank you

Foundational references: Sklar (1959); Braun (1988, 2025); Betebenner (2009); Nelsen (2006). This program of work is developed with AI-based systems integrated across analysis, formalization, and exposition; the authors retain full responsibility for all claims.