- Presenting at PALE 2018
- Presenting at Learning @ Scale 2018 and AI in Education 2018
- Paper accepted for the PALE Workshop at AIED 2018
- I’ve joined the Advisory Board for Code: Out
- Chapter accepted for volume on user experience in learning systems
- Two full papers accepted to Learning @ Scale 2018
- Presenting at OLC 2018
- Measuring Learner Tone and Sentiment at Scale via Text Analysis of Forum Posts
- Sentiment Analysis of Student Evaluations of Teaching
- Squeezing the Limeade: Policies and Workflows for Scalable Online Degrees
- Toward CS1 at Scale: Building and Testing a MOOC-for-Credit Candidate
- Intelligent Evaluation and Feedback in Support of a Credit-Bearing MOOC
- Harnessing professionals’ motivations and experiences to improve accredited MOOC-based education
- A holistic approach to academic integrity in a CS1 MOOC
CS1301x: Introduction to Computing in Python
Read more about CS1301: Introduction to Computing in Python
In Spring 2017, we launched CS1301x: Introduction to Computing in Python. The course is an experiment in attempting to develop material that can be delivered at scale and be used for a credit-bearing class.
Built on design principles like congruency, adaptivity, and modularity, the course tightly integrates short videos with live practice problems. Throughout the course, students complete over 400 live coding problems with immediate feedback and assessment provided by Vocareum, as well as hundreds more multiple choice and fill-in-the-blank problems provided by edX.
The course is unique in that in addition to the video material one would expect to find in a MOOC, it also provides an adaptive textbook. The book, identical in content and structure to the course, provides an alternate mechanism for consuming the course content.
To date, over 500 Georgia Tech students have enrolled in the course for credit, while over 70,000 students have enrolled in the MOOC. Although Georgia Tech students receive additional support mechanisms like office hours and recitations, the assessments and material are identical across both. Thus, CS1301x shows it is possible to develop a scalable, credit-worthy course.
The course is now available as an Xseries on edX, complete with exams and rigorous assessments, and continues to be offered for credit at Georgia Tech. It has been popular in both the Summer Online Undergraduate Program and the Move On When Ready program.
Read more about OMS CS6750: Human-Computer Interaction
Launched in Fall 2016, OMS CS6750: Human-Computer Interaction focuses on the principles and methods of HCI. It covers the design principles pioneered by visionaries like Don Norman and Jakob Nielsen; theories like distributed cognition and situated action; perspectives from philosophers like Langdon Winner and Ruth Cowan; and the stages of the design life cycle.
In addition to introducing the perspectives and ideals of HCI into the OMSCS program, OMS CS6750 was an experiment in course development. The course focuses on a highly modular structure, where short videos frequently shift setting. This approach allows for simpler maintenance of the course while also keeping the student engaged through interesting shifts in perspective and presentation style.
Read more about OMS CS6460: Educational Technology
In Fall 2015, I launched OMS CS6460: Educational Technology. CS6460 is taught on-campus by Betsy DiSalvo, Barbara Ericson, and others, and it represnted a challenge to convert to the online format. Unlike most classes in the OMSCS program, CS6460 involves very little traditional lecturing when taught in person: most of the class time is taken up with class discussions, project presentations, small group work, and other interactive activities that cannot be pre-produced. However, our earlier experiences in CS7637: Knowledge-Based AI taught us that class participation and interaction can actually be better in the online program, and so CS6460 was built to thrive on that level of class contribution. The class has been a huge success; students report high satisfaction, but more importantly, the class has led to published research, start-up companies, open source apps, and more projects that persist after the semester has ended.
CS6460 is modeled after a miniature PhD dissertation process. Students start the class exploring various fields of Educational Technology before selecting an area in which they are most interested. From there, students spend a month delving into the literature surrounding their chosen topic, getting to know the current researchers and projects in the field. Approximately one-third of the way through the semester, students write a proposal for a project, then spend the remainder of the semester executing their project. In the end, students submit the project as well as a video presentation and a publication-ready paper.
The class is built very heavily on a mentorship model. Each student is partnered with a mentor who evaluates all their assignments, gives guidance on expanding their understanding, and approves their proposal and final project. Through this close relationship, the mentor achieves a greater understanding of the individual student's trajectory, and the student has a go-to person for feedback and questions. Additionally, the class heavily leverages peer review opportunities, not for peer grading but rather to receive feedback from their highly qualified classmates.
All materials for CS6460 are available publicly online, with or without a Udacity account. The general course syllabus and the syllabi for the Fall 2015 and Spring 2016 are available, with links to all course assignments. Most importantly, the course library, a detailed list of resources related to several topics in Educational Technology, is available as well, and can be used by any other classes or groups.
Read more about OMS CS7637: Knowledge-Based Artificial Intelligence
Beginning in February of 2014, I began co-developing a course for Udacity and the Georgia Tech Online Master's of Science in Computer Science with my thesis adviser, Ashok Goel. The course, Knowledge-Based AI: Cognitive Systems, is organized around three primary learning goals. First, this class teaches the concepts, methods, and prominent issues in knowledge-based artificial intelligence. Second, it teaches the specific skills and abilities needed to apply those concepts to the design of knowledge-based AI agents. Third, it teaches the relationship between knowledge-based artificial intelligence and the study of human cognition. After completing development of the course, I transitioned into co-instructing the course in Fall 2014.
From the beginning, my involvement in developing this course was intended as an exploration of and experiment in online learning. How do we leverage what we know from the learning sciences in an online course? What are the biggest obstacles to creating an engaging and effective learning experience online? And, perhaps most interestingly to me, what are the opportunities in online education to not only replicate traditional educational experiences, but improve them? What can we do online that we have difficulty doing in person?
Toward this end, we use this online course to explore several issues in online education. First, I have developed a collection of nanotutors -- AI agents that tutor small, specific skills -- that are deployed directly in the context of interactive exercises and activities within the lessons of the class to provide individualized, embedded feedback to students. Second, we have leveraged peer-to-peer feedback in order to explore scaling a rapid feedback cycle up to a large number of students. Third, we have explored project-based learning in the context of an online classroom as a way of providing engaging, personalizable, and measurable assessments. Fourth, we have investigated key issues in online learning, such as ways to overcome the feeling of isolation in an online class, the specific opportunities provided by online discussions, and the unique demographics, interests, and priorities of online students.
For more information on my experience with the Georgia Tech OMS generally and the Knowledge-Based AI class more specifically, take a look at my blog.
LucyLabs was founded with two goals: to research new initiatives in affordable, accessible education, and to provide research opportunities for online students. Since its inception, over two dozen students have participated in research through LucyLabs, and its work has been featured in conferences like Learning @ Scale, Learning with MOOCs, and AI in Education. For more on LucyLabs, check out its dedicated web site at LucyLabs.gatech.edu.
Our acronym "Limeade" stands for Large Internet-Mediated Accredited Degree. Georgia Tech's OMSCS program stands as the original Limeade, but the paradigm applies to new online degrees at Coursera and programs like edX's MicroMasters. Limeades are distinct from MOOCs in that they attach real college credit to online course material and typically include human evaluation, but differ from traditional distance learning programs in their emphasis on scalable programs and lower costs.
Our research on Limeades focuses on three primary components: the student experience within these programs, the policies and technologies used to deliver these programs, and the learning outcomes of courses in these programs. Our research on Limeades has been featured in the 2016, 2017, and 2018 ACM Conferences on Learning @ Scale, the 2016 Symposium on Educational Advances in AI, the 2018 Conference on AI in Education, and the 2016 and 2017 Learning with MOOCs conferences, as well as in the International Journal for Scholarship of Technology-Enhanced Learning in 2016 and Designing for the User Experience in Learning Systems in 2018.
CS1 at Scale
In 2017, we launched an online version of CS1301, Georgia Tech's Introduction to Computing class. Dubbed CS1301x due to its presence on edX, The class strives to operate at MOOC scale while simultaneously offering a credit-worthy experience to on-campus students.
Our research focuses on the demographics, outcomes, and experiences of students in the course, especially compared to the traditional version of CS1301. We found that students in CS1301x learn no less than students in the traditional version; that they report needing to invest less time to achieve those learning outcomes; and that they report higher satisfaction with the course. The class has drawn more under-represented minorities and non-technical majors, suggesting the online version is preferred by those that do not consider themselves "computer science" people.
Our research on CS1 at scale has been presented at the 2017 and 2018 ACM Conferences on Learning @ Scale, the 2017 Learning with MOOCs conference, and the 2018 International Conference on AI in Education, as well as the forthcoming book Blended Learning in Practice. It also has spawned a textbook, Introduction to Computing, published by McGraw-Hill.
MILA: Modeling & Inquiry Learning Application
For my dissertation research, I designed, implemented, deployed, and evaluated a collection of tools teaching an authentic process of scientific modeling and inquiry to middle school students. The suite of tools consists of three parts: an exploratory learning environment (MILA, Modeling & Inquiry Learning Application), a metacognitive tutoring system (MILA–Tutoring), and a simulation compilation agent (MILA–Simulation).
Results of this research have been featured at several conferences, including the 2010 and 2015 meetings of the Cognitive Science Society, the 2011 International Conference on Advanced Learning Technologies, the 2013 Annual Conference on Systems Engineering Research, the 2014 and 2015 International Conferences on Intelligent User Interfaces, the 2014 International Conference on Intelligent Tutoring Systems, and the 2015 Conference on Computational Creativity, as well as the International Handbook on Meta-Cognition and Self-Regulated Learning in 2013.
- Ph.D. in Human-Centered Computing | Georgia Tech | Specialized in Learning Sciences & Technology | Thesis titled "Metacognitive Tutoring for Inquiry-Driven Modeling" | 2009-2015
- M.S. in Human-Computer Interaction | Georgia Tech | Specialized in Learning Sciences & Computing Education | 2008 - 2009
- B.S. in Computer Science with High Honor | Georgia Tech | Certificate in Social & Personality Psychology | Specialized in People & Media | 2005 - 2008
If parents want to give their children a gift, the best thing they can do is to teach their children to love challenges, be intrigued by mistakes, enjoy effort, and keep on learning. –Carol Dweck, Mindset: The New Psychology of Success