- Graders as Meta-Reviewers: Simultaneously Scaling and Improving Expert Evaluation for Large Online Classrooms
- The Unexpected Pedagogical Benefits of Making Higher Education Accessible
- Expert Evaluation of 300 Projects per Day
- TAPS: A MOSS Extension for Detecting Software Plagiarism at Scale
- Designing Videos with Pedagogical Strategies: Online Students’ Perceptions of Their Effectiveness
- C21U Seminar Series Talk Now Online
- GVU Brown Bag Talk: Impact of Students in the OMSCS
- I can’t say anything good about most MOOCs.
- Course Review: Developing Innovative Ideas for New Companies: The First Step in Entrepreneurship
- Course Review: Emerging Trends & Technologies in the Virtual K-12 Classroom
- I’ve won the College of Computing Outstanding Graduate Teaching Assistant Award
OMS CS6460: Educational Technology
Read more about OMS CS6460: Educational Technology
In Fall 2015, I launched OMS CS6460: Educational Technology. CS6460 is taught on-campus by Betsy DiSalvo, Barbara Ericson, and others, and it represnted a challenge to convert to the online format. Unlike most classes in the OMSCS program, CS6460 involves very little traditional lecturing when taught in person: most of the class time is taken up with class discussions, project presentations, small group work, and other interactive activities that cannot be pre-produced. However, our earlier experiences in CS7637: Knowledge-Based AI taught us that class participation and interaction can actually be better in the online program, and so CS6460 was built to thrive on that level of class contribution. The class has been a huge success; students report high satisfaction, but more importantly, the class has led to published research, start-up companies, open source apps, and more projects that persist after the semester has ended.
CS6460 is modeled after a miniature PhD dissertation process. Students start the class exploring various fields of Educational Technology before selecting an area in which they are most interested. From there, students spend a month delving into the literature surrounding their chosen topic, getting to know the current researchers and projects in the field. Approximately one-third of the way through the semester, students write a proposal for a project, then spend the remainder of the semester executing their project. In the end, students submit the project as well as a video presentation and a publication-ready paper.
The class is built very heavily on a mentorship model. Each student is partnered with a mentor who evaluates all their assignments, gives guidance on expanding their understanding, and approves their proposal and final project. Through this close relationship, the mentor achieves a greater understanding of the individual student's trajectory, and the student has a go-to person for feedback and questions. Additionally, the class heavily leverages peer review opportunities, not for peer grading but rather to receive feedback from their highly qualified classmates.
All materials for CS6460 are available publicly online, with or without a Udacity account. The general course syllabus and the syllabi for the Fall 2015 and Spring 2016 are available, with links to all course assignments. Most importantly, the course library, a detailed list of resources related to several topics in Educational Technology, is available as well, and can be used by any other classes or groups.
Machine Learning Engineer Nanodegree Program
Read more about the Machine Learning Engineer Nanodegree Program
In Fall 2015, together with several other teammates at Udacity, I helped launch the Machine Learning Engineer Nanodegree Program, or MLND. MLND is unique among Udacity's Nanodegree programs in that it heavily leverages the existing Georgia Tech material for its content. Several Georgia Tech courses, including Charles Isbell's and Michael Littman's Machine Learning and Reinforcement Learning, Ashok Goel's and my Knowledge-Based AI, Tucker Balch's Machine Learning for Trading, and Jimeng Sun's Big Data Analytics for Healthcare are all used in the Nanodegree program. These, together with some Udacity Machine Learning and Data Science courses, create a program that takes its students from no knowledge of Machine Learning to enough understanding to seek jobs in the field. Along the way, I serve as the guide to students, with short videos transitioning them between different courses and topics.
Like all Nanodegree programs, MLND is heavily project-focused. Students complete a series of projects moving from model building to supervised, unsupervised, and reinforcement learning, and then finally complete a capstone project in a domain of their choosing like education or healthcare. For each project, students receive individual feedback from an expert grader, and are encouraged to revise and resubmit as many times as they need until they have completed the project satisfactorily. This feedback cycle, accompanied by the rapid feedback enabled by the presence of on-demand graders, has enormous positive pedagogical implications.
In terms of accompany courseware, students draw from the Georgia Tech Machine Learning classes and the Udacity Machine Learning and Data Science classes. Students begin with a Foundations lesson, covering the general goals of Machine Learning in the context of the broader field of Artificial Intelligence. They then move on to building models themselves before proceeding to design systems that can build models of data using supervised, unsupservised, and reinforcement learning.
At the conclusion of the Nanodegree program, students complete a capstone project in an area of their choosing, like education, healthcare, or finance. Students also have the option of delving into core Machine Learning more deeply with a Deep Learning course. The Machine Learning Nanodegree program is also part of the Udacity Nanodegree Plus job guarantee, where graduating students are guaranteed to get a job in the field of their choosing within 6 months or receive a full tuition refund.
OMS CS7637: Knowledge-Based AI: Cognitive Systems
Read more about OMS CS7637: Knowledge-Based AI: Cognitive Systems
Beginning in February of 2014, I began co-developing a course for Udacity and the Georgia Tech Online Master's of Science in Computer Science with my thesis adviser, Ashok Goel. The course, Knowledge-Based AI: Cognitive Systems, is organized around three primary learning goals. First, this class teaches the concepts, methods, and prominent issues in knowledge-based artificial intelligence. Second, it teaches the specific skills and abilities needed to apply those concepts to the design of knowledge-based AI agents. Third, it teaches the relationship between knowledge-based artificial intelligence and the study of human cognition. After completing development of the course, I transitioned into co-instructing the course in Fall 2014.
From the beginning, my involvement in developing this course was intended as an exploration of and experiment in online learning. How do we leverage what we know from the learning sciences in an online course? What are the biggest obstacles to creating an engaging and effective learning experience online? And, perhaps most interestingly to me, what are the opportunities in online education to not only replicate traditional educational experiences, but improve them? What can we do online that we have difficulty doing in person?
Toward this end, we use this online course to explore several issues in online education. First, I have developed a collection of nanotutors -- AI agents that tutor small, specific skills -- that are deployed directly in the context of interactive exercises and activities within the lessons of the class to provide individualized, embedded feedback to students. Second, we have leveraged peer-to-peer feedback in order to explore scaling a rapid feedback cycle up to a large number of students. Third, we have explored project-based learning in the context of an online classroom as a way of providing engaging, personalizable, and measurable assessments. Fourth, we have investigated key issues in online learning, such as ways to overcome the feeling of isolation in an online class, the specific opportunities provided by online discussions, and the unique demographics, interests, and priorities of online students.
For more information on my experience with the Georgia Tech OMS generally and the Knowledge-Based AI class more specifically, take a look at my blog.
MILA: Modeling & Inquiry Learning Application
Science is more than a body of knowledge. It is a way of thinking; a way of skeptically interrogating the universe with a fine understanding of human fallibility. –Carl Sagan
Read more about MILA: the Modeling & Inquiry Learning Application
For my dissertation research, I designed, implemented, deployed, and evaluated a collection of tools teaching an authentic process of scientific modeling and inquiry to middle school students. The suite of tools consists of three parts: an exploratory learning environment (MILA, Modeling & Inquiry Learning Application), a metacognitive tutoring system (MILA–Tutoring), and a simulation compilation agent (MILA–Simulation).
MILA is an exploratory learning environment implemented in Java. In MILA, students describe an ecological phenomenon, propose hypotheses for what might cause that phenomenon to occur, construct explanatory conceptual models for how that hypothesis might explain that phenomenon, and provide evidence in support of their explanation. Toward this end, MILA also allows students to participate in a variety of inquiry activities, like gathering data, experimenting with simulations, and exploring related systems.
Within MILA, my main research has been on a metacognitive tutoring system, MILA–T. MILA–T is comprised of five intelligent agents that monitor student behavior within MILA and provide mentorship, guidance, critiques, and support within the modeling and inquiry activity. MILA–T was tested in a controlled study with over 200 students, and analysis showed that engagement with MILA–T improved students' attitudes towards science, and also improved the explanations of ecological phenomena that students produced.
MILA, MILA–T, and MILA–S have spawned multiple ongoing collaborations striving to support the entire life cycle of a scientist, from secondary school to college to amateur to professional. Upcoming proposed work with the Georgia State University College of Education aims to scale MILA and MILA–T into a more comprehensive curriculum for inquiry-driven modeling in middle school science. Upcoming work with the Georgia Tech School of Biology explores how the same principles and tools can support college-level students. Proposed work with the Smithsonian Institute aims to connect MILA with amateur and professional scientists.
In November 2015, I delivered a talk to Georgia Tech's Center for 21st Century Universities (C21U) titled, "The Unexpected Pedagogical Benefits of Making Higher Education Accessible". The talk traces through some of the efforts initially intended to make the Georgia Tech OMSCS program more accessible, and how it is exactly those efforts that led to an improved educational experience for the students. This talk is also the basis for an upcoming paper at Learning @ Scale.
For CS6460: Educational Technology, I've filmed a series of interviews with professionals in EdTech. At present these professionals primarily come from Udacity, but more will be added in the coming months. The individuals interviewed for the course include Sebastian Thrun on Online Education (also on the left), Stuart Frye on the business of educational technology, Kathleen Mullaney on Gender, Technology, and Education, and Cameron Pittman on game- and simulation-based learning.
|On September 24th, 2015, I delivered a talk to the GVU Brown Bag officially titled, "Impact of Students in the OMSCS", and unofficially titled, "The OMSCS program attracts amazing students and empowers them to do amazing things." In the talk, I discuss the how the flexibility and accessibility of the OMSCS program have allowed it to attract incredible students; how the structure of the program gives those students great control over the classes; and how those two factors together have led to amazing results.|
|In Fall 2015, I launched my own OMSCS course, CS6460: Educational Technology. The class is made possible by the incredible work we've seen OMSCS students produce in the past, and the desire to give them the opportunity to pursue their ideas and contribute their work to the field. Rather than lectures, it is comprised of a library of course materials for students to navigate in their own way, culminating in an ambitious project proposal and, ultimately, a deliverable contribution to the EdTech community.|
On December 4th, 2014, Prof. Ashok Goel and I delivered a talk to the GVU Brown Bag. The talk, titled "Putting Online Learning and Learning Sciences Together", covered our experience with bringing a learning science perspective to the online course development. We focus on a number of design principles surrounding online learning, and especially on the unique opportunities of the medium. To view the talk in its entirety, please visit talk's page on the GVU Brown Bag Archive, or watch the talk on YouTube on the left.
On August 19th, 2014, Ashok Goel and I launched CS7637: Knowledge-Based AI: Cognitive Systems as part of the Georgia Tech Online Master's in Computer Science. To the left is the first video in the course, where we introduce ourselves and our course to the students in the class. To date, the course has been offered four times: co-instructed by us both in Fall 2014 and Spring 2015, by myself alone in Summer 2015, and by Ashok alone in Fall 2015. The course is publicly available free at Udacity.
- Ph.D. in Human-Centered Computing ♦ Georgia Tech ♦ Specialized in Learning Sciences & Technology ♦ Thesis titled "Metacognitive Tutoring for Inquiry-Driven Modeling" ♦ 2009-2015
- M.S. in Human-Computer Interaction ♦ Georgia Tech ♦ Specialized in Learning Sciences & Computing Education ♦ 2008 - 2009
- B.S. in Computer Science with High Honor ♦ Georgia Tech ♦ Certificate in Social & Personality Psychology ♦ Specializing in People & Media ♦ 2005 - 2008
If parents want to give their children a gift, the best thing they can do is to teach their children to love challenges, be intrigued by mistakes, enjoy effort, and keep on learning. –Carol Dweck, Mindset: The New Psychology of Success