Define ‘meaningfully different’ per incidence group

The advice. “You could have a different definition of meaningfully different for the four groups.” And: “With the higher fields, the percentage of acceptable overlap could be less (less overlap for higher incidence).”

Why. OEE’s working definition is a single overlap rule — one ceiling for overall items, a slightly lower one for the scorable set — applied across four groups that differ by an order of magnitude in annual volume. (The ceilings themselves are in the restricted layer.) The rule exists because low-incidence fields cannot fill an item pool fast enough, but it is being applied to high-incidence fields that can. The constraint that justifies the rule does not apply where the rule does the most damage.

Higher-incidence fields accumulate examinees quickly, so they can support more distinct items and should be held to a lower overlap ceiling. Low-incidence fields need the tolerance.

What it replaces. The rejected definition — see MTTC Meaningfully Different Forms — together with the committee’s judgement that a handful of forms produced over several years, each sharing most of its items with the others, cannot be called meaningfully different.

Status. Open. OEE would need to re-open the working definition with Pearson / Evaluation Systems.