In my work and reading while carrying out research for our college’s self-study, I’ve encountered an unexpected reality. People are collecting a lot more data than they used to do. Between the emphasis on accountability from federal and state governments, the need to demonstrate effectiveness and achievement, and the growing “culture of assessment” it seems we have, at the very least, agreed on the need to gather data to know what’s happening in our educational system. In fact, many people are drowning in data.
Assessment: The unclosed loop
The increase in empirically based practices lags behind the amount of data we collect. People gather the data, look at it, and don’t know what to do next. If we don’t like the story it tells, we may moan and groan – and then work harder in hopes the data will be better next time. But the effort isn’t guided by details from the data. Sometimes it’s even guided by off-the-cuff hypotheses about what might have caused those findings.
Assessment, the buzzword of the decade, is supposed to be the first part of a process:
- gather the data and find out what is going on.
- Plumb its depths to make any causal inferences possible.
- Design changes that address the specific low-points identified by the data
- Design brief measures that can be made as the program progresses, to see if it is having an effect in the desired direction
- Implement the program and the measures
- If changes seem to be occurring, continue the program and assess its overall impact at the end.
- If changes are not occurring, or they are in the wrong direction, look for data indicating why the plans didn’t work – then tweak or overhaul the program.
What often happens, though, is a short-circuited version:
- gather the data to find out what’s going on
- if it’s pretty good, breathe a sigh of relief
- if it’s not so good, pitch in to work harder and implement some good ideas you’ve heard about in the news or on journals
- hope and pray that the figures will be better at the end of the year
Struck by Data
This usual trajectory creates a situation where we haven’t used the data as the basis for our change – so we aren’t able to predict or affect the results of the next assessment. When the new data are collected, they come up and hit us in the face.
Invesco put out a video to illustrate how unattended-data can wreak havoc in our life:
- Data Collection Cycles LucidChart (wootang01.wordpress.com)
- What Makes Salespeople Stand Out from the Crowd? (customerthink.com)
- Visual.ly – a new tool to create data visualisations (blogs.journalism.co.uk)