Assessment/standards transition has become the norm.
Some states have transitioned assessments 3 times in the last several years.
In most cases these states have also transitioned standards at least once.
In many cases states allow dual mode administration (P&P or Online).
The ESSA Era: Ch-Ch-Changes
ESSA has given states increased flexibility in creating accountability systems.
Most states now include student growth in their accountability systems.
Even those states with experience using calculating growth encounter challenges when transitioning between assessments/standards.
Knowing what to look for beforehand can minimize problems that can emerge.
Ch-Ch-Changes & Growth
Assessment/Standards transition can impact growth calculations, sometimes in significant ways.
Some states have transitioned assessments 3 times in the last several years.
In most cases these states have also transitioned standards at least once.
In many cases states allow dual mode administration (P&P or Online).
In some cases states have had issues with ceiling/floor effects on their tests.
Vertical Scales
Some states have transitioned from assessments without a vertical-scale to assessments with vertical-scales (e.g., many SBAC states)
Some states have transitioned from assessments with a vertical scale to assessments without a vertical scale.
Some states have never had a vertical scale.
As should be apparent, you don't need a vertical scale to calculate some form of student growth.
Vertical Scales
Despite not needing a vertical scale, some states use their vertical scale for growth calculations
A vertical scale allows for subtraction which is intuitive to users.
Depending upon how the scale is used, it can be critical that the scale be strongly equated from year-to-year.
If difference scores are calculated for students, consistent interpretation from year-to-year requires strong equating.
Norm-Referenced Growth
Most states do some form of norm-referenced growth (SGP or VAM) even if they have a vertical-scale.
Norm-referenced growth is very forgiving with regard to assessment transitions.
As long as students take the same assessment in any given year, we can norm their progress.
However, norms by their nature are limited to stating where one is in a larger distribution.
Criterion-Referenced Growth
Several states have shown interest in supplementing their norm-referenced analyses with growth-to-proficiency (criterion-referenced) growth.
Growth-to-standard is particularly prominent in English Language Proficiency testing.
Growth-to-standard calculation requires at least two years of data on the same scale.
To perform growth-to-standard across an assessment transition, some states put the old test scores on the new scale and move forward
as though they have two years of data.
Transitioning to new standards doesn't impede growth-to-standard but will result in more difficult/lenient targets depending upon
whether the standards are more difficult or lenient.
Value tables
Value-tables, which use the states prior and current performance levels to cross-tabulate student growth and award points based upon the transition observed.
In an of themself, a value-table is neither norm- or criterion-referenced. Similar to gain scores.
Use of value-tables requires the performance levels to mean the same thing from year to year.
Assessment or standards transition require resetting values in the value-table.
Mode Effects
Many states, transition to online assessments, have allowed dual mode administration.
Often obscured in status results, growth is a magnifying glass that shows these differences.
Unless explicity adjusted for, on every assessment and in every state I've seen significant mode effects -- large enough to impact growth results used for accountability.
Ceiling/Floor Effects
Tests with significant ceilings/floors can impact some growth analyses.
Students at the HOSS/LOSS will receive inaccurate growth scores
These results, when aggregated, can change summary scores used for accountability ratings.
Due-Dilligence
Investigate correlations between growth and prior-achievement at both the individual and summary level. Make sure the results make sense.
Investigate summary growth results for specific demographic subgroups or school types. Make sure the results make sense.
If allowing dual-mode administration, look at growth results associated with each testing mode. Make sure there are only minor differences.
If using growth-to-standard, calculate summary results and make sure the results conform to your vision for the accountability system.
Look at growth associated with lowest and highest performers (prior score) and make sure growth scores are accurate.
Center for Assessment