Workspace Awareness Support for Bring-Your-Own-Device Classrooms
Multi-device Classrooms & Problem of Collaborator Awareness
In a multi-device classroom, students are in close proximity and are connected through a large shared virtual canvas via their personal devices. In this environment, students are often times very focused on their individual devices and have difficulties observing other students due to the physical distance among them and the private nature of personal devices. As the result, maintaining awareness of peers in the virtual workspace was difficult for the students.
A group of students is working on a math problem together using their tablets
During our field observations and discussions with teachers, students would yell out to the entire class to find out who was mysteriously moving their objects in the workspace (e.g., pictures, videos, and annotations). This created distractions for other students, and teachers had to use social means to compensate for what technology was lacking. We observed that one teacher posted up signs in the classroom advising students to “practice patience with peers and technology”.
This project investigated ways to improve students’ awareness of their peers while minimizing distractions to their learning.
Context: SMART amp Software
This project was in collaboration with SMART Technologies in Calgary, Alberta, Canada, and we decided to use the SMART amp software as our case study. It provided a web-based shared canvas, similar to Google Docs. The primary usage scenario was that each student used a laptop or tablet in the class, and SMART amp was a shared virtual canvas that connected all students. The shared canvas allowed multiple students to work together at the same time.
A multi-device classroom making use of the SMART amp software. (Image owned by SMART Technologies)
In SMART amp, a teacher might ask students to collaboratively brainstorm ideas, solve group questions, correct each other’s writing, and construct diagrams. The digital functionality provided by the devices can enhance the students’ learning experiences. For example, as a teacher asks the students to collaboratively construct a diagram for our solar system in the virtual canvas, students can research on the characteristics of different planets online, insert videos, add their own drawings, and annotate their pictures
Sample usage of SMART ampTM. Teachers asked students to do a variety of activities in the shared canvas. One example was to collaboratively brainstorm. In this case, a teacher pasted a Frayer model for students to brainstorm on mammals (as shown in the centre circle). Students had to answer the question in each quadrant: definition, characteristics, examples, and non-examples.
Design Requirements and Proposed Solution
Through field observations and discussions with teachers and company stakeholders, I derived the following requirements to guide the design process.
Based on the design requirements, I propose a transient awareness cue to show the current states of objects that were being manipulated.
- Balancing Awareness and Distractions: As students’ main objective in the workspace is learning, the awareness cue design should not distract students from their tasks at hand.
- Appropriate for Classrooms: The awareness cue needs to be understandable and appropriate for a wide age range, study subjects, and activity formats (e.g., individual work, small group, and large group, whole class brainstorming). The design also needs to consider students’ unique work patterns. My in-class observations revealed that, unlike adults, students are easily distracted and they rapidly and repeatedly switch between working and socializing with other students.
- Applicable to a Wide Variety of Devices: The awareness cue design should consider only the basic support given the wide variety of devices may be used. The cue also needed to be efficient enough in terms of data communication and performance to account for the slower low-cost devices.
Based on the design requirements, I propose a transient awareness cue to show the current states of objects that were being manipulated.
Conceptual design of the student awareness cue. A) Student John is interacting with a star on his tablet. B) Another student saw that the star begins to move. C) With the awareness feature, she finds out that John was moving it. D) The awareness cue fades out and disappears after some time.
Iterative Design Process
Low-fidelity sketches
I approached the challenge of designing a workspace awareness cue that balanced awareness and distractions for students by employing a highly iterative approach. I started with hand-drawn sketches to quickly iterated through a wide range of design concepts. The sketches were presented to the teachers and company stakeholders to get their feedback. The figure below shows some of the early sketches.
I approached the challenge of designing a workspace awareness cue that balanced awareness and distractions for students by employing a highly iterative approach. I started with hand-drawn sketches to quickly iterated through a wide range of design concepts. The sketches were presented to the teachers and company stakeholders to get their feedback. The figure below shows some of the early sketches.
Some of the initial sketches showing varying
amounts of identity, interaction, and history information as
well as different locations for awareness information.
Medium-fidelity mock-ups
Based on the feedback on the initial sketches, we decided to go with a light-weight persistent display next to the objects that students are interacting with. The mock-ups were presented to teachers and company stakeholders to get their feedback. The figure below shows some of the medium-fidelity mock-ups.
Based on the feedback on the initial sketches, we decided to go with a light-weight persistent display next to the objects that students are interacting with. The mock-ups were presented to teachers and company stakeholders to get their feedback. The figure below shows some of the medium-fidelity mock-ups.
Some of the digital mock-ups showing different
designs of awareness cue. The top versions (A, B, C, and D) depict when one student is interacting with the object. The bottom versions (E, F, G, and H) depict when more than one student has interacted
with the object recently.
High-fidelity Implementation & Redesign
Based on the feedback on the digital mock-ups, the Balloon design was selected and implemented (see A and E in the figure above). It displayed a collaborator’s initials and faded out in a fixed amount of time. After the design was tested in an informal testing session with company employees, a classroom evaluation was conducted with a class with experience teacher and student users of SMART amp. Although students did notice the cue, they were confused about who the initials belonged to.
The redesigned cue incorporated students’ first names, and the visual design resembled a Callout Bubble pointing towards its associated object, see figure below. I further improved the design through two rounds of informal testing with brainstorming and small-group collaboration tasks.
Based on the feedback on the digital mock-ups, the Balloon design was selected and implemented (see A and E in the figure above). It displayed a collaborator’s initials and faded out in a fixed amount of time. After the design was tested in an informal testing session with company employees, a classroom evaluation was conducted with a class with experience teacher and student users of SMART amp. Although students did notice the cue, they were confused about who the initials belonged to.
The redesigned cue incorporated students’ first names, and the visual design resembled a Callout Bubble pointing towards its associated object, see figure below. I further improved the design through two rounds of informal testing with brainstorming and small-group collaboration tasks.
Callout Bubble design. (A) Sophia K. is interacting with the “BYOD” text; (B) cue fades out; (C) cue re-adjusts its position when there is no sufficient space at the top.
Classroom Evaluation: Field Study
Data collection and analysis
The final Callout Bubble design was tested in a field study with 4 teachers and 71 students to understand whether and how the Callout Bubble impacted the students’ awareness of each other. I chose to do a field study to know whether the cue was appropriate for such a real-world application. The teachers conducted their daily lesson as usual, but used the version of the application with Callout Bubbles.
At the end of their lessons, the students and teachers filled out an online survey on Google Forms. The teacher survey aimed to gather information about the lesson, group size, divergent of lesson plan, and their feedback on the Callout Bubble. The student survey focused on students’ experience using the Callout Bubble, how they resolved conflicts, and their behaviour change. I conducted statistical analysis on Likert-scale questions on the survey and conducted text-coding on the free form answers.
Key questions and findings
In the data analysis, one particular student comment really stood out to the team and showed the the extent of the problem and the impact having the Callout Bubble can make.
The final Callout Bubble design was tested in a field study with 4 teachers and 71 students to understand whether and how the Callout Bubble impacted the students’ awareness of each other. I chose to do a field study to know whether the cue was appropriate for such a real-world application. The teachers conducted their daily lesson as usual, but used the version of the application with Callout Bubbles.
At the end of their lessons, the students and teachers filled out an online survey on Google Forms. The teacher survey aimed to gather information about the lesson, group size, divergent of lesson plan, and their feedback on the Callout Bubble. The student survey focused on students’ experience using the Callout Bubble, how they resolved conflicts, and their behaviour change. I conducted statistical analysis on Likert-scale questions on the survey and conducted text-coding on the free form answers.
Key questions and findings
In the data analysis, one particular student comment really stood out to the team and showed the the extent of the problem and the impact having the Callout Bubble can make.
“This Callout Bubble has saved my life because this can tell me who has been touching my stuff and been doing stuff.”
- A 10-year-old student
- A 10-year-old student
Some of the key questions include:
The results showed that Callout Bubble improved student’ coordination within themselves and allowed for self-monitor. One student’s comment showed improve coordination of physical space:
- Does Callout Bubble distract students from their work?
- How does Callout Bubble change student behaviour?
The results showed that Callout Bubble improved student’ coordination within themselves and allowed for self-monitor. One student’s comment showed improve coordination of physical space:
“Now it's easier to work so you know who's working on something so you don't get in their way.” – Student 40
Students also showed the ability to coordinate their learning:
“If I [see] someone working on that part [of the project] I would go to a different problem.” – Student 3
One teacher also commented on the benefits of Callout Bubble in keeping students accountable of their actions:
“It kept them more on task and they kept each other more accountable for what was happening in the work space …” – Teacher 3
Although the Callout Bubble was designed for students, it also helped the teachers with their monitoring of student activities. Considering a group of students using tablet, due to the private nature of personal devices, it can be difficult for teachers to get a sense of the overall class progress and which group needed help. One teacher commented on how Callout Bubble helped in maintaining awareness of student activities:
“I could keep tabs easier on who was doing what without having to check attribution [a feature in SMART amp], and it was real-time.” – Teacher 2
Attribution is a feature that teachers can see who created and who last edited an object.
Insights & Lesson Learned
Some of the key lessons learned through this study include the following:
- Support awareness of collaborators even in co-located environments: Given the distance and private nature of the personal devices, it was difficult for students to maintain awareness of each other. Although the students were co-located in the same classroom, there was still a need to support their awareness of each other.
- Support teacher’s awareness of classroom activities: Similarly, it was difficult for the teachers to moderate the classroom activities in real-time. While Callout Bubble also helped teachers in keep track of student activities, more understanding into the teachers’ information needs in multi-device classrooms is needed.
- Fine tune the notification/distraction balance: While the design process tried to minimize distraction, overly minimalistic designs created confusions. Designs need to balance between notifying students and creating distraction and confusion.
My Roles
- Planned and executed research milestones including ideation, requirement gathering, competitive analysis, scoping, iterative design, implementation, and field study
- Improved student collaboration and reduced teacher workload by releasing the Callout Bubble feature and collaborating with a cross-functional team in an agile environment
- Presented research progress and actionable insights to stakeholders at the end of each sprint
Collaborators
- Erica Arnoldin – SMART Technologies
- Colin Dere – SMART Technologies
- Cresencia Fong – University of Toronto, SMART Technologies
- Mark Hancock – University of Waterloo
- Stacey D. Scott – University of Waterloo
- Edward Tse – SMART Technologies
- Min Xin – SMART Technologies
Publication & Presentation
- Chang, Y.-L.B., Fong, C., Tse, E., Hancock, M., Scott, S.D. (2015). "Callout Bubble Saved My Life": Workspace Awareness Support in BYOD Classrooms. Interactive Tabletop and Surfaces, pp. 73-82.
- Supporting Awareness in Large Collaborative Virtual Workspaces for BYOD Classrooms. Presented to University of Waterloo CS 349 students (User Interfaces), Waterloo, ON, April 2, 2015