Remark: A pen for student feedback

What is Remark?

In recent times, we have seen the rise of the use of educational technology in classrooms. It is evident that its use is to improve overall efficiency of classroom management and communication. However, little has been done in the area of improving live communication between teachers and students during a lecture. Remark is a system that strives to increase the efficiency of teaching by aiding two way communication between students and teachers live in the classroom by the means of a constant stream of low level feedback from the students to the teachers that can be quickly interpreted by the teacher and thereby affect their manner of instruction positively.

Users: Our users were High School students and Teachers

Problem Space Context: High school classroom environments

Skills used: Contextual Inquiry, Interviews, Surveys, Focus Group, Ethnographic observation, Sketch (for UI), Arduino, Affinity Mapping

Role in Project: We had a team of 4 students from the MS-HCI program, jointly responsible for Research, Brainstorming, Conceptualization and Evaluation.. Individually, I was responsible for Arduino Programming and set up while my other team mates worked on Sketch, Laser cutting and Processing for prototyping. This project was done as a part of the requirements of one of our courses, but we have continued to work on it after the course, and submitted a paper to CHI-2016 as well.

Affinity mapping after nitial user research

User Research

To explore current roles of educational technology in classrooms and existing problems, we visited two high schools in greater Atlanta area and observed 10 classes, including STEM and non-STEM classes. Our research was done with:

  • Ethnographic observations of 8 live classroom sessions

  • Face to face interviews with teachers

  • Telephonic Interviews of teachers outside Atlanta (to verify observed findings)

  • Surveys of teachers inside and outside Atlanta (to verify observed and interpreted findings before and during our design phase)

What did we find?

Initial Design brainstorming after affinity mapping themes were found

  • Teachers want to know whether students really understand the lecture or not, often asking students the same, and expecting an honest response.

  • Teachers sometimes do not know if something has been taught to the student previously and hence can spend time over-explaining it

  • Students often are hesitant to ask questions in class and resort to asking each other or approach the teacher privately.

  • Teachers were overburdened with different kinds of eduTech apps and services, most of them that dealt with only out of the class course management systems.

  • Teachers did not have the time to scavenge for technology to use on their own, and complained about the amount of time and effort put into learning to use new technologies every year.

Brainstorming and alternate designs:

After a few sessions of brainstorming and with the help of affinity mapping techniques, we were able to come up with alternate designs to fill our problem space. We decided to work with fixing the 2-way communication between students and teachers to improve the overall efficiency of teacher instruction in a classroom in a way that students could make the most of the lecture in terms of time and content.

Initial System Design:

Our Initial design involved a virtual hand raise system where a student could ask a question to the teacher or to the rest of the class through a discussion forum using an on desk tablet/device. This eventually would that could be displayed to the class on a public display board for the teacher and other students to read. This would allow students to ask questions instantly and essentially queue them up for the teacher to answer. This would also allow students to discuss or help each other with any questions posted.

One of the initial design iterations that was later discarded in favor of the Remark system

So what went was wrong with this design?

  • Expensive to deploy

  • On-desk interface can be distracting

  • “Bring Your Own Device” concept may widen the gap between students who do and do not have smart phones or personal devices

  • Opens window for students to use personal devices for other reasons

  • The system might compromise the privacy of the student wanting to ask a question. This is probably now, no different from simply raising your hand and asking the question.

  • If we allow students to remain “anonymous” on a public classwide forum, they might misuse it making it difficult for the teacher to monitor, thus making it more counterproductive than productive.

  • Why use such a system that tries to “create” a new, an unnatural channel of communication when the most natural manner of communication is to simply put your hand up and ask?

Final System design

Design Brainstorming whiteboard that led to the birth of Remark as we know

We took a step back and looked at the weaknesses from our initial design proposal. Our main issues with this design were the important ones highlighted above. In light of this, we had another few sessions of brainstorming

Finally, we decided to design a non-screen headless interface for the students. Remark was design to be a student initiated system where students could imply indicate a binary value to the teacher of whether they understand the lecture at any instant or not. They do this by swiping on a specialized pen attachment that was unique to each student and contained their ID. The collective responses of the class is collected and presented live to the teacher as an easy to read pie chart of the percentage of the class who can understand the lecture at that given time. The teacher can adapt her lecture to suit the needs of the students accordingly. All these collected responses are also stored for the teacher to reflect on later.

The current prototype of the Remark pen attachment includes a soft potentiometer to provide swiping control and a NeoPixel LED to give feedback to the user. Casings for the electronics are custom 3D-printed for the types of pens used. Each pen plugs into an Arduino R3 microcontroller, programmed with custom code to receive input from all pens and send data output to a custom Processing script on an attached computer. <pic>

Remark Pen prototype used by students

STUDENT INTERACTIONS:

Students can swipe on the soft potentiometer on their pen attachment upwards (to indicate they understand the lecture) or downwards (to indicate they do not understand). They can swipe multiple times through the course of the lecture.

TEACHER INTERACTIONS:

The teacher side of the system is mainly designed for teachers to collect feedback from their students. Teachers interact with the system on two conceptually different occasions, live mode and offline mode.

 

LIVE MODE:

  Teacher &nbsp;live visualization during lecture, dynamically updated in response to student inputs via   t  he Remark Pen

Teacher  live visualization during lecture, dynamically updated in response to student inputs via the Remark Pen

 

In live mode, real time visualization of the feedback from students is displayed for the teacher. We assume that the teacher will need this information while teaching, hence, our visualization needs to be comprehensible from just a quick glance at their screen. This display is extremely minimalistic and can be read even from a distance, as it simply shows a pie chart graphing the number of students who are understanding class material versus the number of students who are not for that given instant. This is a dynamic visualization that reflects the current average status of understanding in the classroom, along with the numeric representation of that percentage of understanding.

OFFLINE MODE:

In offline mode, teachers can interact with the system when they are not teaching, by logging into their account on the Remark System. They can:

  • View the new performance of the class on a line chart over the entire time of the lecture.
  • View the performance of each student in the class over a lecture.
  • Listen back with an associated audio clip of the teacher speaking at the points where class comprehension reaches a minima, around 30 seconds before or after (configurable) the minima was reached. The configurable limit will be set beforehand and recorded via a buffer that continually discards unneccessary recordings.

A diagrammatic representation of the entire Remark System

 

SO WHAT ARE THE BENEFITS OF OUR DESIGN?

  • Augments natural interactions by simply adding additional "power" to them. It does not try to alter any normal kind of interaction between teachers and students in the classroom.
  • Pens on the student re portable between classes
  • Minimalistic, non distracting, headless interface
  • No repeated login required, since students own the device.
  • Students can control pace of a lecture with their responses, thereby increasing teaching efficiency.
  • Feedback for each student can be submitted privately without other students knowing what feedback was submitted. 
  • Records and trends from previous lectures can be used by the teacher to improve lecture structure and content, increasing classroom efficiency.
 User Evaluations with High School students

User Evaluations with High School students

Evaluations:

We conducted user evaluations with a focus group of 6 high school students who were asked to use the device and later discuss their opinions on it. We conducted the following kinds of evaluations methods:

  • Focus Group discussions
  • Observation in a Lab setting with a mock lecture.
  • Heuristic Evaluation with expert evaluators (fellow students and faculty)

Our detailed evaluation report and findings, along with conclusions and future directions are documented in this PDF

Future Directions:

While we are working to incorporate some of the suggestions form our evaluation into our prototype, our Paper on Remark is under review for acceptance at CHI 2016 under Late Breaking works.