Assessment Literacy Leaders in Indiana (ALLIN)
Understanding and Using Standardized Assessment
Understanding and Using Standardized Assessment
Problem:
A new standardized test was introduced in Indiana. Educators and potential educators needed help understanding how to interpret and appropriately use assessment data.
Overview:
We developed an in-person workshop with hands-on interactive activities. I led the curriculum design in collaboration with three other team members (see our website here). I also led the facilitation of all the interactive activities in the workshop.
We held several workshops at local school and district locations in Indiana as well as at the School of Education at Indiana University. We had more than 100+ educators participate in these workshops.
I also redesigned the workshop content for an online version that educators could participate in through the learning management system Canvas.
Below are examples from our workshop content.
Workshop Content
Outline for Assessment Literacy Workshop
Introductions
Assessment Literacy: Why?
How a test comes to life
Validity
Reliability
Standard Error of Measurement
Fairness
Computer-adaptive testing (CAT)
Purposes of your interim assessments
Understanding your interim assessment score reports
Interpreting your interim assessment score reports
Based on data collected from educators, a review of assessment curriculum, and our own expertise, we developed a list of assessment literacy standards.
After polling educators, we created an outline for the workshop that catered to their specific educational needs. Each workshop had a slightly different outline, but the general outline for the workshop is provided.
We began with a hook, helping educators see how better understanding assessment would help them in their jobs. Then, we taught various aspects of assessment based on their needs. Finally, we ended by helping educators apply what they learned to their own context.
Activity - Card Sorting
Goal: Prime educators for a discussion about the test creation process.
We separated the educators into groups of 3-4, handed them a group of cards with each step of the typical test creation process, and asked them to put the cards in order from first to last.
Then we shared the actual order for the test creation process, walked them through the process, and discussed differences between their expected order and the actual order.
Activity - Workshop Application
Goal: Help educators apply the concepts from the workshop to the assessments their schools were using.
We had small application activities throughout the workshop, but the last half of the workshop was a series of activities and prompts where educators looked at their own assessments to apply what they learned.
Below are the handouts that accompanied these activities.
Workshop_WorkshopApplications.pdfOnline Workshop - The Case for Assessment Literacy (presentation and script)
The online workshop was designed in the Canvas Learning Management System and was split into three modules.
The image to the right shows an example of the organization of the first module on Standardized Assessment Processes.
Each section of the module started with a brief introduction to what learners should be able to do upon completing the module and what activities were part of the module. For example, below is the introduction to "The Case for Assessment Literacy" online module.
After completing this section, you should be able to:
Describe what being assessment literate means
Productively discuss the impact of assessment literacy on education
For this section you will:
Read a short article about the importance of assessment literacy (~10 minutes).
Watch our video giving a brief description of our workshop and why we believe assessment literacy is important (~15 minutes).
Rank the reasons for becoming more assessment literate (as discussed in the reading and presentation) in order of most relevance to you, given your role and setting (~15 minutes).
(Optional) Post questions to your peers in this workshop (and the facilitators) about this section.
The first activity asks learners to read and collaboratively annotate a short blog post:
Read Jim Popham - The Most Cost-Effective Way to Improve Schools.
Read this short article by Jim Popham who has written a lot about assessment and assessment literacy. This is a short read, so it shouldn't take you very long.
(OPTIONAL) Annotate the article with comments or questions. If you have questions or comments about the content of the article, use the annotation function of Google Drive to annotate the article. To annotate a PDF in Google Drive:
Select text in the PDF.
Click the "Comment" button on the right.
Type your comment or question.
Click the "Comment" button to post your annotation.
Video Presentation
In the online version, we recorded short presentations on the different topics covered in the workshop. Those recording were posted on YouTube and then embedded within the learning management system. This is the video presentation that introduces one of the topics covered in the workshop.
01_CaseForAssessmentLiteracy.pdfPresentation Script and PDF
These are the presentation slides with the script below. We asked workshop participants to use the "comment" feature in Google Drive to annotate the script with their comments and questions. This turned a static presentation into a collaborative process because other participants could see and respond to their peers' comments.
To help educators apply the principles, they are asked to consider the context they work in and their job responsibilities and then write a reflection responding to a prompt. See an example below.
Rank the reasons
Rank the reasons for becoming more assessment literate (as discussed in the reading and presentation) in order of most relevance to you, given your role and setting.
The reasons are:
Give educators a voice to be able to speak back when assessments are not being used appropriately.
Choose or create more appropriate benchmark/interim assessments.
Be able to make better educational decisions in schools and classrooms.
Better understand the assessment on which school and teacher evaluations are based.
Add a brief justification for the reasons you chose as the most relevant and the one you chose as the least relevant, keeping in mind your role and context. Make sure to provide enough detail so that those not familiar with your context can understand your thought process. If you have not thought about these things before, this might be challenging.
(OPTIONAL) Add additional reasons why assessment literacy is important that are not listed that you feel are important to consider.