Collaborative Annotation Technology in Online University Courses
In Fall of 2020, the English Department at Indiana University began using the collaborative annotation software, Hypothesis, in all the online freshman reading and writing courses. This implementation provided an opportunity to study how students and teachers were using this technology towards learning.
I led an end-to-end generative research study exploring how university instructors used the Hypothesis collaborative annotation tool in their classroom.
Problem and Research Questions
Collaborative annotation software is not new in higher education, but most of the design and research has been focused on the students as the users. University instructors also use the software and often design how the students use collaborative annotation through instructions and assignments requirements.
I sought understand the user perspective of the instructors. I asked:
How did the English Department university instructors use the collaborative annotation technology, Hypothesis, in their courses?
How did instructors use the technology features?
What problems arose while using the technology?
This project was part of my dissertation research and I had to move at a much slower pace than I would have liked due to educational institution requirements and administrative hurdles.
To speed up the impact of the research for Hypothesis, I shared out early results at a conference hosted by Hypothesis called iAnnotate. I also met with organization leaders at Hypothesis to share some of the specific issues that instructors faced.
I chose to use a focus group approach for collecting user insights and I structured the focus groups similar to a contextual inquiry. There were four focus group sessions that took place over the course of 4 months. The instructors shared their screen and showed how they were using collaborative annotation and how they were asking their students to use it.
I deliberately chose to use focus groups for a few reasons.
First, I wanted the sessions to feel comfortable for the users (instructors) and the focus group format is similar to professional development meetings that they were already familiar with.
Second, conversations among users can lead to naunced and rich insights because users will ask each other questions or build off of each other's responses in ways that may not happen in in-depth interviews.
Finally, focus groups can save time and money by combining multiple individual interviews into one session.
What I Did
As the lead researcher on this project, my responsibilities included:
Managing study budget and participant compensation
Facilitating 7 focus group and user testing sessions with 6 instructors who were using collaborative annotation
Collecting and managing research data (50+ course documents, 1500+ annotations, 25 survey responses, over 7 hours of focus group data)
Transcribing focus group data
Analyzing data using qualitative coding (using thematic analysis and content analysis)
Using data collection and analysis software: Microsoft Excel, MaxQDA, Zoom, and Google Forms
Conducting data audits with research team to ensure data analysis is consistent
Presenting results to stakeholders