Gauging student sentiment using IBM Watson

Durruty, D. & Joyner, D. A. (2016). Gauging student sentiment using IBM Watson. In Proceedings of Learning with MOOCs III. Philadelphia, PA.

Managing large classes can pose unique challenges to the social and affective dimensions of learning. In small- to moderately-sized classes, enough interactions exist to give a sense of the dynamics, personalities, preferences, and sensibilities of the participants. Knowing these subtle yet crucial nuances can make the difference between a sterile, lifeless class and a compelling, personalized learning environment, leading to significant impacts on student outcomes (Swan & Shih 2005).

This problem is exacerbated in current online classrooms: there are few visual or audial cues to distinguish a frustrated student from an enthusiastic one, and textual content can often be difficult to interpret. This in turn frequently results in inaccurate judgements or hidden grievances (Murphy & Coleman 2004). Moreover, in a large class there are so many submissions that it can be difficult for an instructor to get a sense of the student’s unique experience. Thus, instructors are forced to cater only to the hypothetical, non-existent “average” student.

Tone Analyzer from IBM Watson

To resolve this challenge, we examined Tone Analyzer, from the NLP-centric IBM Watson ecosystem (IBM 2016). The Tone Analyzer service is a suite of closed-source, proprietary algorithms for automatically processing textual content in any form (e-mails, forum posts, assignments) and generating a sentiment and personality profile. Tone Analyzer breaks the emotional tone of a segment down by categories such as emotional tone, language style, and big five personality markers; thus, its results provide useful information for instructors (Busato, Prins, Elshout, & Hamaker 1998).

A Solution: Foobazzle

Using Tone Analyzer, we created Foobazzle, a web-based service for gauging student sentiment. Foobazzle serves three primary purposes:

  • Foobazzle allows instructors to get an overall feel for how the class is progressing. If class sentiment is declining (for instance, if students are becoming relatively more frustrated or angry with time), the instructor knows that something is going awry.
  • Foobazzle gives instructors the ability to identify which students are struggling the most and consequently which students require the greatest amount of attention. Frustrated, disappointed, angry or sad students will likely require more assistance than enthusiastic, self-motivated learners.
  • Foobazzle permits educators to identify the type of personalized help each student needs. A shy, introverted and insecure student may require a different approach than an energetic and confident learner.

These functions are aimed at helping instructors replicate the nuanced impressions they would obtain from students in-person based on facial expression, vocal tone, and attendance patterns. Foobazzle will be tested in classes this summer in Georgia Tech’s online Master of Science in Computer Science program, where class enrollment can top 500 students (Joyner, Goel, & Isbell 2016). Future improvements to tools like Foobazzle may extend these analyses to judging facial expressions during office hours, digitally proctored exams, or synchronous class sessions or activities. Foobazzle was implemented using HTML, CSS, JavaScript, and Node.js, and is available at Foobazzle.com.

References

Busato, V. V., Prins, F. J., Elshout, J. J., & Hamaker, C. (1998). The relation between learning styles, the Big Five personality traits and achievement motivation in higher education. Personality and Individual Differences, 26(1), 129-140.

IBM. (2016). Tone Analyzer. Retrieved from https://www.ibm.com/smarterplanet/us/en/ibmwatson/developercloud/tone-analyzer.html

Joyner, D. A., Goel, A., & Isbell, C. (2016). The Unexpected Pedagogical Benefits of Making Higher Education Accessible. In Proceedings of the Third Annual ACM Conference on Learning at Scale. Edinburgh, Scotland.

Murphy, E., & Coleman, E. (2004). Graduate Students’ Experiences of Challenges in Online Asynchronous Discussions. Canadian Journal of Learning and Technology 30(2).

Swan, K., & Shih, L. F. (2005). On the nature and development of social presence in online course discussions. Journal of Asynchronous Learning Networks, 9(3), 115-136.

Full Version

Leave a Reply

Your email address will not be published. Required fields are marked *