Challenge

There lacked research on how real-time information of student learning may augment and complement K-8 teachers' pedagogical practices when teaching computer science using approachable coding environments such as Scratch. Before proposing any new technological interventions to address the research gap, we are gathering direct insights from K-8 teachers to explore:

(1) how teachers typically teach K-8 students using approachable coding environments (we focus on Scratch in this project), and
(2) what preferences teachers may have for a new technology design aimed to augment their computer science teaching capabilities.

Process

Initial needs analysis

Interviews

I met up and conducted semi-structured interviews with 10 DE and MD K-8 teachers who were currently teaching computer science using Scratch in formal class settings. Through these meetings, our team had a better understanding of their Scratch teaching practices and what they may look for in an augmentation design as educators.

SURVEY

To complement and validate our findings from the interviews, I deployed a survey designed on Qualtrics based on themes from the interviewed teachers' responses. Using snowball sampling, we received responses from 37 teachers who have taught using Scratch in at least one formal K-8 class session, some of whom previously partnered with our university via our annual computer science education professional development program. 

INSIGHTS

The teaching practices both interview and survey participants wanted augmentation for were:
(1) to maintain and enhance students' levels of motivation and engagement, and
(2) adapt Scratch lessons according to students' expressed learning needs.

Both studies' participants most preferred augmentation form was a learning analytics dashboard. The remaining examples of forms augmentation has previously taken in prior research to assist teachers received mixed reactions. 

Should a technology be designed to augment the aforementioned teaching practices using data on student learning, the interviewees specifically expressed that they would prefer to take control over the task of interpreting the data instead of allowing automation to do so (e.g., technology provides recommendations on best Scratch teaching practices according to data on student learning).

I definitely think [interpretation] needs to be differentiated per child, whether you have somebody who’s [in] special needs or ESL [English as a second language]. That all needs to be taken into factor and I don’t know if a computer can automate that. You might have a student who has mental health issues who needs extra support in that area. So, I think the empathy part of the teacher needs to be involved in that. (Teacher 7)

We found that the insights gained from our initial needs analysis aligned with past empirical findings on common pedagogical needs K-12 computer science teachers have, one of which was to receive guidance on how to implement student-centered teaching practices. 

Design recommendations

We put together design recommendations based on the obtained insights to guide and inform future works on augmenting K-8 teachers' Scratch teaching practices. They include: 

• Augment teachers' awareness of classroom happenings rather than recommending specific teaching actions to teachers. 
• First explore learning analytics dashboard as an augmentation form; explore impacts of other forms such as ambient lighting displays informed by prior research or its combination with dashboard. 
• Place human interpretation of data on student learning at a priority, but consciously present data and information in a way that would be useful to teachers on-the-go in classrooms. 

Based on what we heard from teachers during the interviews, I also made a recommendation for future augmentation designers to minimize potentials of bias fusing into their designs and broadly, teaching practices and students' learning experience impacted by augmentation. 

Additional details on the initial needs analysis phase and results are available in our conference publication. 

Concept validation

Speed dating & Storyboarding

To further understand and validate insights and design recommendations from the initial needs analysis phase, I adopted the "speed dating" design method to observe their gut reactions to imagined scenarios—I represented ours with storyboards—and identify expected and new design opportunities. From this phase onwards, we decided to redefine our explorations of augmentation designs' targeted teaching audience to 3rd-5th grade teachers; according to Digital Promise's 2021 Computational Thinking for an Inclusive World report, grades 3-5 used coding environments like Scratch to learn computer science. 

Our 13 storyboards represented futuristic classroom scenarios where students are learning to program using Scratch and their teacher is using an augmentation technology to be more aware of classroom happenings—the augmentation ideas were inspired by the needs and preferences past teacher participants expressed. The storyboards also imagined meaningful reflective prompting of students during class as a teaching practice that may augment teachers' awareness of classroom happenings (e.g., checking in on students and asking what they found interesting the most so far, what they want to learn next, and so on). Our literature review revealed that existing Scratch-based elementary-level curricula lacked recommendations on how students can be guided to reflect on their progress towards successfully completing a Scratch coding activity.

I met with 7 DE and MD teachers, 4 of whom were from our previous interviews or survey, and presented them with the imagined scenarios. Here are some of the scenario storyboards I illustrated: 

Order ranking

After the speed dating session, our 7 teachers also completed an order ranking activity. The main objective of the activity was to validate the teachers' reactions to the presented storyboards with additional artifacts. Using a Google Jamboard I created for the activity, teachers individually completed three ranks, each rank of storyboards that make up a main category of scenarios (i.e., monitor student learning using Scratch, initiating reflective moments, level of data detail and automation). They were also welcome to add new ideas for augmentation to the ranks. 

INSIGHTS

Overall, in alignment to the initial needs analysis, teachers had a clear preference for augmentation to come in the form of a learning analytics dashboard over ambient lighting displays, a different example form of teaching augmentation from past works, when monitoring students' Scratch learning. Many were concerned that the classroom-wide display of information on students' learning experience with a lighting display would make students competitive, distracted, and/or self-conscious. However, some teachers expressed interest in exploring the use of both augmentation forms simultaneously. 

Teachers preferred that their awareness of classroom happenings involving Scratch be augmented with detailed information on student learning (e.g., more specific information on students' coding decisions and explorations in Scratch), but the group as a whole was torn on how much they wanted to take on the task of interpreting the data as they teach in formal class settings. 

Teachers were also interested in prompting meaningful reflective learning moments amongst their students with the help of an augmentation design when they code using Scratch, but they have never tried implementing such teaching practice when teaching using Scratch. Many teachers described that they have prompted reflective learning amongst students in other subjects or units before, but typically with the use of existing educational technologies, many of which were introduced during the transition to remote learning due to the COVID-19 pandemic. 

What was common amongst all teacher participants was that teachers preferred to monitor student learning when using Scratch by walking around the class and providing individualized guidance to each student. Public elementary school teachers especially lacked instructional time to teach using Scratch as is that they may find it more time-consuming to check a learning analytics dashboard mid-class for information on student learning; most instructional time in their schools are devoted to tested subjects (i.e., English language arts and mathematics) and teachers teaching computer science only have approx. 45 minutes per class each week. 

Outcomes

We are currently reevaluating our initial research plan to conduct prototyping of augmentation forms at multiple levels of fidelity. Our concept validation phase revealed in much greater detail that crucial barriers still exist in elementary-level computer science education (e.g., lack of classroom instructional time), which were in alignment with Google and Gallup's report on K-12 computer science education in 2016. Continuing with the initial research plan would introduce interventions that may impact our targeted users (elementary school teachers) without considering on-the-ground realities and practice, in addition to using internal development resources and time on artifacts that may not have as great of a return on investment as first predicted. Further case studies, interviews, direct observations, and participatory research using methods such as photo-voicing may be required to craft a typology of elementary school teachers' computer science teaching practices, which can then inform more meaningful interventions and designs.