Durruty, D. & Joyner, D. A. (2016). Gauging student sentiment using IBM Watson. In Proceedings of Learning with MOOCs III. Philadelphia, PA.
Managing large classes can pose unique challenges to the social and affective dimensions of learning. In small- to moderately-sized classes, enough interactions exist to give a sense of the dynamics, personalities, preferences, and sensibilities of the participants. Knowing these subtle yet crucial nuances can make the difference between a sterile, lifeless class and a compelling, personalized learning environment, leading to significant impacts on student outcomes (Swan & Shih 2005).
This problem is exacerbated in current online classrooms: there are few visual or audial cues to distinguish a frustrated student from an enthusiastic one, and textual content can often be difficult to interpret. This in turn frequently results in inaccurate judgements or hidden grievances (Murphy & Coleman 2004). Moreover, in a large class there are so many submissions that it can be difficult for an instructor to get a sense of the student’s unique experience. Thus, instructors are forced to cater only to the hypothetical, non-existent “average” student.
Tone Analyzer from IBM Watson
To resolve this challenge, we examined Tone Analyzer, from the NLP-centric IBM Watson ecosystem (IBM 2016). The Tone Analyzer service is a suite of closed-source, proprietary algorithms for automatically processing textual content in any form (e-mails, forum posts, assignments) and generating a sentiment and personality profile. Tone Analyzer breaks the emotional tone of a segment down by categories such as emotional tone, language style, and big five personality markers; thus, its results provide useful information for instructors (Busato, Prins, Elshout, & Hamaker 1998).
A Solution: Foobazzle
Using Tone Analyzer, we created Foobazzle, a web-based service for gauging student sentiment. Foobazzle serves three primary purposes:
- Foobazzle allows instructors to get an overall feel for how the class is progressing. If class sentiment is declining (for instance, if students are becoming relatively more frustrated or angry with time), the instructor knows that something is going awry.
- Foobazzle gives instructors the ability to identify which students are struggling the most and consequently which students require the greatest amount of attention. Frustrated, disappointed, angry or sad students will likely require more assistance than enthusiastic, self-motivated learners.
- Foobazzle permits educators to identify the type of personalized help each student needs. A shy, introverted and insecure student may require a different approach than an energetic and confident learner.
Busato, V. V., Prins, F. J., Elshout, J. J., & Hamaker, C. (1998). The relation between learning styles, the Big Five personality traits and achievement motivation in higher education. Personality and Individual Differences, 26(1), 129-140.
IBM. (2016). Tone Analyzer. Retrieved from https://www.ibm.com/smarterplanet/us/en/ibmwatson/developercloud/tone-analyzer.html
Joyner, D. A., Goel, A., & Isbell, C. (2016). The Unexpected Pedagogical Benefits of Making Higher Education Accessible. In Proceedings of the Third Annual ACM Conference on Learning at Scale. Edinburgh, Scotland.
Murphy, E., & Coleman, E. (2004). Graduate Students’ Experiences of Challenges in Online Asynchronous Discussions. Canadian Journal of Learning and Technology 30(2).
Swan, K., & Shih, L. F. (2005). On the nature and development of social presence in online course discussions. Journal of Asynchronous Learning Networks, 9(3), 115-136.