- Graders as Meta-Reviewers: Simultaneously Scaling and Improving Expert Evaluation for Large Online Classrooms
- The Unexpected Pedagogical Benefits of Making Higher Education Accessible
- Expert Evaluation of 300 Projects per Day
- TAPS: A MOSS Extension for Detecting Software Plagiarism at Scale
- Designing Videos with Pedagogical Strategies: Online Students’ Perceptions of Their Effectiveness
- C21U Seminar Series Talk Now Online
- GVU Brown Bag Talk: Impact of Students in the OMSCS
- I can’t say anything good about most MOOCs.
- Course Review: Developing Innovative Ideas for New Companies: The First Step in Entrepreneurship
- Course Review: Emerging Trends & Technologies in the Virtual K-12 Classroom
- I’ve won the College of Computing Outstanding Graduate Teaching Assistant Award
- Paper accepted to Learning with MOOCs 2016
- Paper accepted to Qualitative Reasoning 2016
- In Edinburgh presenting at Learning @ Scale
- Learning @ Scale Flipped Presentations Now Available
- I’ve won the 2016 College of Computing Dissertation Award!
- I’ve won the Lockheed Excellence in Teaching Award!
- Joining C21U!
OMS CS6460: Educational Technology
Read more about OMS CS6460: Educational Technology
In Fall 2015, I launched OMS CS6460: Educational Technology. CS6460 is taught on-campus by Betsy DiSalvo, Barbara Ericson, and others, and it represnted a challenge to convert to the online format. Unlike most classes in the OMSCS program, CS6460 involves very little traditional lecturing when taught in person: most of the class time is taken up with class discussions, project presentations, small group work, and other interactive activities that cannot be pre-produced. However, our earlier experiences in CS7637: Knowledge-Based AI taught us that class participation and interaction can actually be better in the online program, and so CS6460 was built to thrive on that level of class contribution. The class has been a huge success; students report high satisfaction, but more importantly, the class has led to published research, start-up companies, open source apps, and more projects that persist after the semester has ended.
CS6460 is modeled after a miniature PhD dissertation process. Students start the class exploring various fields of Educational Technology before selecting an area in which they are most interested. From there, students spend a month delving into the literature surrounding their chosen topic, getting to know the current researchers and projects in the field. Approximately one-third of the way through the semester, students write a proposal for a project, then spend the remainder of the semester executing their project. In the end, students submit the project as well as a video presentation and a publication-ready paper.
The class is built very heavily on a mentorship model. Each student is partnered with a mentor who evaluates all their assignments, gives guidance on expanding their understanding, and approves their proposal and final project. Through this close relationship, the mentor achieves a greater understanding of the individual student's trajectory, and the student has a go-to person for feedback and questions. Additionally, the class heavily leverages peer review opportunities, not for peer grading but rather to receive feedback from their highly qualified classmates.
All materials for CS6460 are available publicly online, with or without a Udacity account. The general course syllabus and the syllabi for the Fall 2015 and Spring 2016 are available, with links to all course assignments. Most importantly, the course library, a detailed list of resources related to several topics in Educational Technology, is available as well, and can be used by any other classes or groups.
LucyLabs: An Online Research Lab for Educational Technology
LucyLabs: An Online Research Lab for Educational Technology
LucyLabs was founded in 2016 in response to two demands: the academic community’s demand for research on the enormous amount of data produced by new efforts in online education, and online students’ demand for opportunities to engage in academic research. Thus, LucyLabs was founded with two goals:
- To research new initiatives in affordable, accessible education, especially within online education.
- To provide the opportunity for online students to participate in academic research.
LucyLabs is presently comprised of a faculty member, myself, and several Masters students, primarily students in the Georgia Tech OMSCS program. We typically collaborate with other faculty members as well. Our current projects focus on: data-based methods for predicting student success in online courses; technology-based initiatives to scale high-quality low-cost assessment and feedback; development of student communities in online education; and efforts to scale the administration of large online classes. Our research focuses especially, but not exclusively, on the Georgia Tech OMSCS program.
LucyLabs was named after my daughter. After entertaining a number of arcane acronyms attempting to unify the parallel themes of affordable education, quality education, online education, and online research opportunities, I, through conversations with colleagues and early lab members, took a step back to look at the ultimate motivation behind providing affordable, excellent higher education. For me, it’s about the next generation. Many of my friends, and more generally many from the Millennial generation, are saddled with enormous amounts of student loan debt. They are forced to put off families, home ownership, and adulthood while paying back their expensive college education. The goal of LucyLabs is to provide a better educational climate for the next — for Lucy’s — generation.
Georgia Tech has long been at the forefront of the push for affordability in higher education. From the HOPE scholarship to the $6,600 Master’s in Computer Science program, it gives a fantastic foundation for exploring efforts to increase college affordability while preserving rigor and excellence for the next generation.
OMS CS7637: Knowledge-Based AI: Cognitive Systems
Read more about OMS CS7637: Knowledge-Based AI: Cognitive Systems
Beginning in February of 2014, I began co-developing a course for Udacity and the Georgia Tech Online Master's of Science in Computer Science with my thesis adviser, Ashok Goel. The course, Knowledge-Based AI: Cognitive Systems, is organized around three primary learning goals. First, this class teaches the concepts, methods, and prominent issues in knowledge-based artificial intelligence. Second, it teaches the specific skills and abilities needed to apply those concepts to the design of knowledge-based AI agents. Third, it teaches the relationship between knowledge-based artificial intelligence and the study of human cognition. After completing development of the course, I transitioned into co-instructing the course in Fall 2014.
From the beginning, my involvement in developing this course was intended as an exploration of and experiment in online learning. How do we leverage what we know from the learning sciences in an online course? What are the biggest obstacles to creating an engaging and effective learning experience online? And, perhaps most interestingly to me, what are the opportunities in online education to not only replicate traditional educational experiences, but improve them? What can we do online that we have difficulty doing in person?
Toward this end, we use this online course to explore several issues in online education. First, I have developed a collection of nanotutors -- AI agents that tutor small, specific skills -- that are deployed directly in the context of interactive exercises and activities within the lessons of the class to provide individualized, embedded feedback to students. Second, we have leveraged peer-to-peer feedback in order to explore scaling a rapid feedback cycle up to a large number of students. Third, we have explored project-based learning in the context of an online classroom as a way of providing engaging, personalizable, and measurable assessments. Fourth, we have investigated key issues in online learning, such as ways to overcome the feeling of isolation in an online class, the specific opportunities provided by online discussions, and the unique demographics, interests, and priorities of online students.
For more information on my experience with the Georgia Tech OMS generally and the Knowledge-Based AI class more specifically, take a look at my blog.
MILA: Modeling & Inquiry Learning Application
Science is more than a body of knowledge. It is a way of thinking; a way of skeptically interrogating the universe with a fine understanding of human fallibility. –Carl Sagan
Read more about MILA: the Modeling & Inquiry Learning Application
For my dissertation research, I designed, implemented, deployed, and evaluated a collection of tools teaching an authentic process of scientific modeling and inquiry to middle school students. The suite of tools consists of three parts: an exploratory learning environment (MILA, Modeling & Inquiry Learning Application), a metacognitive tutoring system (MILA–Tutoring), and a simulation compilation agent (MILA–Simulation).
MILA is an exploratory learning environment implemented in Java. In MILA, students describe an ecological phenomenon, propose hypotheses for what might cause that phenomenon to occur, construct explanatory conceptual models for how that hypothesis might explain that phenomenon, and provide evidence in support of their explanation. Toward this end, MILA also allows students to participate in a variety of inquiry activities, like gathering data, experimenting with simulations, and exploring related systems.
Within MILA, my main research has been on a metacognitive tutoring system, MILA–T. MILA–T is comprised of five intelligent agents that monitor student behavior within MILA and provide mentorship, guidance, critiques, and support within the modeling and inquiry activity. MILA–T was tested in a controlled study with over 200 students, and analysis showed that engagement with MILA–T improved students' attitudes towards science, and also improved the explanations of ecological phenomena that students produced.
MILA, MILA–T, and MILA–S have spawned multiple ongoing collaborations striving to support the entire life cycle of a scientist, from secondary school to college to amateur to professional. Upcoming proposed work with the Georgia State University College of Education aims to scale MILA and MILA–T into a more comprehensive curriculum for inquiry-driven modeling in middle school science. Upcoming work with the Georgia Tech School of Biology explores how the same principles and tools can support college-level students. Proposed work with the Smithsonian Institute aims to connect MILA with amateur and professional scientists.
In November 2015, I delivered a talk to Georgia Tech's Center for 21st Century Universities (C21U) titled, "The Unexpected Pedagogical Benefits of Making Higher Education Accessible". The talk traces through some of the efforts initially intended to make the Georgia Tech OMSCS program more accessible, and how it is exactly those efforts that led to an improved educational experience for the students. This talk is also the basis for an a similar paper at Learning @ Scale 2016.
For CS6460: Educational Technology, I've filmed a series of interviews with professionals in EdTech. At present these professionals primarily come from Udacity, but more will be added in the coming months. The individuals interviewed for the course include Sebastian Thrun on Online Education (also on the left), Stuart Frye on the business of educational technology, Kathleen Mullaney on Gender, Technology, and Education, and Cameron Pittman on game- and simulation-based learning.
|At the 2016 Learning @ Scale conference in Edinburgh, Scotland, an alternative "flipped" track was available for which presenters would film their paper presentations in advance and use their allotted time for interactive activities. I and other collaborators with LucyLabs had five such flipped presentations at the conference. One presentation, "Expert Evaluation of 300 Projects per Day" is available on the left, and the remaining flipped presentations can be found on the LucyLabs media page.|
|On September 24th, 2015, I delivered a talk to the GVU Brown Bag officially titled, "Impact of Students in the OMSCS", and unofficially titled, "The OMSCS program attracts amazing students and empowers them to do amazing things." In the talk, I discuss the how the flexibility and accessibility of the OMSCS program have allowed it to attract incredible students; how the structure of the program gives those students great control over the classes; and how those two factors together have led to amazing results.|
|In Fall 2015, I launched my own OMSCS course, CS6460: Educational Technology. The class is made possible by the incredible work we've seen OMSCS students produce in the past, and the desire to give them the opportunity to pursue their ideas and contribute their work to the field. Rather than lectures, it is comprised of a library of course materials for students to navigate in their own way, culminating in an ambitious project proposal and, ultimately, a deliverable contribution to the EdTech community.|
On December 4th, 2014, Prof. Ashok Goel and I delivered a talk to the GVU Brown Bag titled "Putting Online Learning and Learning Sciences Together" covering our experience with bringing a learning science perspective to the online course development. We focus on a number of design principles surrounding online learning and the unique opportunities of the medium. To view the talk in its entirety, please visit the talk's page, or watch the talk on YouTube on the left.
On August 19th, 2014, Ashok Goel and I launched CS7637: Knowledge-Based AI: Cognitive Systems as part of the Georgia Tech Online Master's in Computer Science. To the left is the first video in the course, where we introduce ourselves and our course to the students in the class. The course is offered three semesters a year, taught by Ashok in Fall and Spring and by myself in Summer. The course's videos are also publicly available free at Udacity.com.
- Ph.D. in Human-Centered Computing ♦ Georgia Tech ♦ Specialized in Learning Sciences & Technology ♦ Thesis titled "Metacognitive Tutoring for Inquiry-Driven Modeling" ♦ 2009-2015
- M.S. in Human-Computer Interaction ♦ Georgia Tech ♦ Specialized in Learning Sciences & Computing Education ♦ 2008 - 2009
- B.S. in Computer Science with High Honor ♦ Georgia Tech ♦ Certificate in Social & Personality Psychology ♦ Specializing in People & Media ♦ 2005 - 2008
If parents want to give their children a gift, the best thing they can do is to teach their children to love challenges, be intrigued by mistakes, enjoy effort, and keep on learning. –Carol Dweck, Mindset: The New Psychology of Success