The MOOC Certificate Paradox

As I’m starting to review some of the courses that I’ve taken, one of my goals is to let prospective employers and admissions offices what they should think when they see a particular MOOC certificate on an applicant’s resume or application. That assessment generally consists of three parts: identity verification, assessment rigor, and content coverage.

Generally, few current MOOC providers feature sufficient verification to make the certificate all that impactful. Coursera’s identity verification, for example, provides no evidence that the student completed the work instead of receiving it from a friend or a database online. Solutions exist to both the problems of identity verification and plagiarism, but thus far, I have not seen many (well, any) MOOC platforms use them.

In the short term, that’s not a major problem. These certificates aren’t yet powerful enough to bother cheating on them. If I’m not going to get significant credit on a job or school application for completing a course, why bother cheating to complete it? This sets up what I call the MOOC Certificate Paradox: the value of a MOOC’s certificate is inversely proportional to the value of the MOOC’s certificate.

How can that be possible? So long as the certificate has no value, we can infer that students only complete it for personal improvement and learning. So long as we can believe they only complete the MOOC for personal improvement and learning, we can attach value to their completion of the MOOC: we know they were not simply earning the credit to improve their resume or application because earning the credit wouldn’t improve their resume or application. Thus, the certificate has value. But if the certificate has value, there emerges an incentive to earn it specifically to obtain that value. We no longer know that they were not simply earning the credit to improve their resume or application because now earning the credit does improve their resume or application. Thus, the value of the certificate diminishes when its value increases. The value of a MOOC’s certificate is contingent on it not having inherent value.

However, this stands in opposition to what I believe to be the function of Verified Certificates and other similar certification programs: to make the course mean something.  Verified Certificates attempt to attach value to MOOC completion, but in the absence of firmer measures to verify students’ identities and prevent plagiarism, that value cannot arise: the moment it arises, it diminishes as an incentive to cheat (as well as an absence of any reliable way to identify cheaters) is introduced.

Of course, there is a way to break out of this paradox: there must be more rigorous methods to verify identities. Quizzes and exams in MOOCs need to be legitimately proctored. All submissions need to be automatically checked for plagiarism. The presence of these measures will provide inherent value to MOOCs rather than value that is dependent on the student’s motives.

Are open online education and quality online education mutually exclusive?

In the past, I’ve touched on a distinction I see in the landscape of higher education. It is this distinction that leads me to say that programs like Coursera and edX and programs like Udacity and the Georgia Tech OMS are not competitors, but rather represent two largely different goals of education: openness and quality.

Of course, I hate using the word ‘quality’ because it implies that open education cannot be high-quality, which is not what I mean to suggest. Rather, what I mean to suggest is that openness and quality often get in the way of one another. Developing open courses for a platform like Coursera almost inherently dictates that costs must be extremely limited. Offering a course through Coursera does not bring in a tremendous amount of money; even the Verified Signature track, I would speculate, barely pays for the human effort required to grade assignments and verify identities. Developing open courses can be an act of either marketing or altruism, but in either case, there is a natural impetus to keep costs low. The outcome, of course, is nonetheless fantastic: the world’s knowledge presented by the world’s experts on that knowledge in a venue that everyone can access. Even if the cost pressure demands that this information can only be presented in the traditional lecture model, the outcome is nonetheless incredibly desirable.

That openness is largely driven by the internet’s ability to deliver content to massive audiences for low costs. However, that’s not the only thing that the internet can do in service of education. The internet also has features and frameworks that can create educational experiences that go beyond what we can do in traditional classrooms. Many traditional college classes are delivered in the same lecture model as the aforementioned Coursera courses, but pedagogically we know that this model is largely ineffective. It is not chosen because it is effective, however; it is chosen because professors’ time is valuable, professors are very often experts in the subject matter rather than in teaching itself, and the lecture model is arguably the easiest way to present material. There are exceptions, of course, but I don’t think I’m being controversial in suggesting these ideas as generally true.

What the internet gives us, however, is a mechanism by which content can be produced once to be consumed by millions. This is part of the reason the openness initiatives work: professors can film the course once and make it available to the masses rather than having to reteach it semester to semester. But while in some places that is an impetus for openness, we may also use that as an impetus for quality. Let’s invent some numbers to make it clearer. Let’s imagine that a class of 50 students are each paying $100 to take a class; this means that the class must cost no more than $5,000 to deliver each semester. However, if the class could be developed once and re-used ten semesters in a row, that means that the same class now can cost up to $50,000 to develop, allowing for much more investment into the quality of the class.

This, of course, is a gross simplification, but it is intended to portray an elegant truth: when we use the internet to deliver content to a much larger population with the same amount of work, we can either pass on the savings to the students (the openness route), or we can reinvest the money into the development of the courses themselves (the quality route). We can ask less investment of the students, or we can give the students more for the same price.

Coursera, edX, and the traditional MOOC community take the former, providing content for a fraction of the cost because it can be delivered to so many people. Udacity, the Georgia Tech OMS, and other more expensive programs take the latter approach, reinvesting that money into creating higher-quality programs in the first place. Both these sides are critical. I don’t like living in a world where education is gated by such a massive monetary investment, and MOOC services are doing a world of good to reduce the barriers to education. At the same time, I love education itself, and I recognize that there are phenomenal things that the internet can do to improve education — but they come with a significant development cost.

Of course, this hasn’t actually answered the question: I’ve shown how openness and quality are distinct and often conflicting goals in online education, but can we accomplish both? Is it possible to create high-quality education that is also openly available for little to no monetary cost? It may be. At present, this is in some ways what the Georgia Tech OMS is doing: nine Georgia Tech courses are available for free to the world, and they are infused with a more significant initial investment that pays significant dividends in the quality of the instruction. This is accomplished because, in some ways, this free offering is “subsidized” by the students taking the actual Masters. This model is incomplete, however, as there is still valuable education locked within the for-cost program. OMS students are not paying for the videos; they are paying for the access to professors and TAs, the access to projects and assignments, and the ultimate “verified certificate”: the Masters diploma at the end of the program. However, this direction at least illustrates that it may be possible to use one offering in service of the other and improve both openness and quality at the same time. For now, however, I regard the two as distinct, exclusive, and desirable goals.

What’s the difference between online learning and distance learning?

At the Georgia Tech OMSCS, we talk a lot about how our program is the first of its kind. The homepage for the OMSCS states, “the first accredited Master of Science in Computer Science that students can earn exclusively through the Massive Open Online Course (MOOC) delivery format and for a fraction of the cost of traditional, on-campus programs.”

However, that description might not be entirely accurate. Georgia Tech OMS courses are neither massive (although they’re getting there) nor open (admission is still required). At the same time, the OMSCS is far from the only online Masters offered at Georgia Tech. So is all the hype just hype?

In my opinion, this speaks to the difference between distance learning and online learning, and the difference is critical. Distance learning has been around for ages through correspondence programs and other similar structures. Some of them are very good. Most online Masters programs today are simple extensions of distance learning programs. In previous years, one would receive course materials in the mail, and mail completed schoolwork back; now, students receive course materials over the internet, and upload completed assignments back. The internet makes distance learning easier, and at times can improve the experience through features like forums, but it does not inherently fundamentally change its structure.

The majority of online Masters programs are distance learning programs of this kind. With Georgia Tech’s online Masters programs, students in the distance learning sections view live or filmed lectures, upload the same assignments, and are graded by the same TAs. The only major difference is geographic: rather than being physically in the room of the lecture, the students are distributed. This is, in my mind, the heart of the distinction between distance learning and online learning: distance learning as nearly as possible identically recreates the in-person process. It may use the internet to do so, but the fundamental structure between distance learning and in-person learning remains the same.

Online learning, on the other hand, aims to leverage the internet not to duplicate the in-person experience, but rather to improve it. Improvement, of course, can come in many ways. Online education can be developed to reduce costs by leveraging MOOC principles, and in fact, this is one of the general guiding principles of the OMSCS: leveraging the internet to deliver an experience that is just as good as the in-person experience at a fraction of the cost. Online learning does not stop there, though. Automated feedback, communities of practice, and several other pedagogical techniques find unique places in the online medium. I’ve talked about a few of these unique benefits in the past, like the ability to transfer course ownership to the students and the natural emphasis on positive activity rather than negative, and I believe we’re only scratching the surface of the ways in which online education can actually improve on the in-person classroom experience.

I, of course, can be accused of bias in that, as an instructor and developer of the Georgia Tech OMS, I want to see it succeed. However, the inverse is true: I work on the Georgia Tech OMS because I believe it will succeed. I’m excited to work on it because while most programs out there are using the internet to improve on distance learning, the Georgia Tech OMS is about using the internet to create new and improved educational experiences altogether. Don’t get me wrong: there’s nothing wrong with distance learning, and it presents some very rich opportunities of its own. Distance learning is all about increasing access to the same quality education, and that is an incredibly important. I’m excited, though, to work on online learning and find ways to use the internet to make higher education more affordable, more accessible, and more effective.

So, if you’re ever asked why we ballyhoo the Georgia Tech OMS so much when online programs, even from highly reputable universities, are becoming common, the reason is that the OMS is about online learning, not distance learning. It’s very different, and it may lead to great things.

Should there exist accreditation for independent online courses?

One of the earliest takeaways from the handful of online courses I’m taking at the moment (Emerging Trends & Technologies in the Virtual K-12 Classroom, Learning How to Learn, and Astronomy: Exploring Space & Time, as well as a few days in Planet Earth and You) is that there exists a radical difference in the scope of different courses. Emerging Trends can be completed in a day if desired. Astronomy: Exploring Space & Time requires a much more significant time investment for the videos, but the interactive elements are relatively light, restricted to short quizzes and writing assignments. Planet Earth and You was much more significant, and decently approximated the amount of time I recall dedicating to traditional on-campus courses.

On the one hand, this is fantastic. Online learning has previously had the strength of not having to arbitrarily fit lessons to pre-determined time slot: if a topic takes more than a class period to instruct, it’s not necessary to arbitrarily break it up halfway through, and if it takes less than a class period to instruct, it’s not necessary to pad it out or randomly combine it with another topic. These courses reflect how that same idea can be expanded to an entire course. Not every course needs to be a three-hours-per-week 16-weeks-per-semester course. If a topic can be learned in 10 to 20 hours total (as Emerging Trends’ “2-4 hours per week, 5 weeks of study” guideline indicates), then let it be learned in that time frame.

On the other side, however, what does that say about how the world interprets these courses? A Bachelors is a Bachelors, a Masters is a Masters, and there exists some general understanding of what those degrees mean. The school from which a degree came has some influence, but universities have spent decades of time building up reputations to differentiate a Stanford Bachelors from a Samford Bachelors. Moreover, there are around 2500 four-year universities in the United States, and while that’s a large number, it isn’t intractable as far as developing an understanding of the different equivalence classes of universities.

There are currently around 1500 Udacity, Coursera, and edX courses combined, just to take three of the biggest organizations as examples. In the three years since these three organizations launched, they’ve developed almost as many courses as there are four-year institutions in the country. If the scope, depth, and rigor of individual open courses on these platforms is going to vary this much, how then will the world learn to interpret what it means to have a credential or certificate from these courses?

To put myself in the position of an employer, if a prospective employee had a verified course certificate from Coursera on their resume that I had not yet heard of, that would presently be somewhat meaningless to me; this is not because the certificate has no value, but just because I have no hope of knowing the certificate’s value at a glance, nor is it feasible to maintain a comprehensive knowledge of all the open courses I might see. This is a tremendous challenge to the value of these programs. Paying for a certified certificate is, for many, based on the belief that the ability to prove you completed a course is powerful. But if the people to whom you would offer that proof have no knowledge of what kind of knowledge and achievement that certificate represents, it remains somewhat meaningless.

This discussion comes dangerously close to the general discussion of accreditation. How does an employer know a certain college degree is valuable? Because it has been accredited by an independent organization. That knowledge of the program’s value and rigor has been offloaded to an external group for assessment. Just like a bank checking with a credit agency before deciding whether to give a person a loan, so also a business implicitly checks with an accreditation group before offering a graduate a job.

I wouldn’t argue that we need accreditation in the traditional sense for online courses; after all, many courses that wouldn’t pass a pass/fail accreditation process are nonetheless very valuable, even if they don’t necessarily demonstrate anything reliable about the students themselves. Based on my first impressions, I can’t say that hearing that a teacher has taken Emerging Trends & Technologies would mean much to my impression of them, but there is still lots they may have learned in the course. Rather, I feel what might be necessary is just a somewhat standardized classification system. Just like on-campus classes are assigned a number of credit hours based on the amount of work they require, so also online classes could be assigned a number or classification of virtual credits based on the rigor, reliability, and scope of the course itself. That, in turn, might lead to even greater programs: instead of a single university building up a Coursera specialization, it could instead be assembled from multiple universities’ courses on the basis of their virtual credits.

But apart from the solution, I feel the problem nonetheless exists and is waiting to be addressed: accreditation fulfills a function in traditional education; should online education have something to fulfill the same function?

MOOCs and online agendas

During my first week in one of the Coursera MOOCs I’m taking, University of Arizona’s Astronomy: Exploring Space and Time, I observed something rather interesting in the community portion of the course. At first I was going to simply include it in my first week impressions, but it brings up a bigger issue that I think is worth examining more thoroughly.

I don’t want to go into the details of it, but early in the course, a thread sprung up started by someone claiming the earth is only 6,000 years old. Obviously, this claim has no place in a science class. While the claim was brought informally by saying, effectively, “Here’s what I think, how about everyone else?”, the responses that followed took, in my opinion, an evangelical turn.

The broader issue at play is this: when we teach a course in person, we tend to assume that the majority of people in the class are students whose primary “agenda” for the class is to either learn the material or get a good grade. In my ten years in academia, I can’t think of an instance of ever worrying about whether additional agendas were at play. This is largely because of the major obstacle to participation in an in-person college degree program: you’re paying a ton of money and dedicating an enormous amount of time, which would deter people with more subtle agendas.

In Coursera and other MOOCs, however, anyone can join, and for free. What is to deter competing agendas from entering the fray? In these MOOCs, there will be people who sign up and participate in these courses with agendas rather than to learn from the course’s stated purpose. There will be churches that consider it an act of evangelism to try to educate these people studying the origins of the universe on the earth’s “true” origin. There will be special interest groups that see classes on climate change or other environmental issues as opportunities to change the conversation in their favor or sow doubt when no doubt ought to exist. There will be political organizations wanting to present a biased view of history that is favorable to their platform. These MOOCs risk becoming battlegrounds rather than learning opportunities. We see this already on other social media platforms, where teams of people are paid by particular groups to try to sway the conversation in one way or the other; there is no obstacle that I can see to prevent open online courses from being subjected to the same kind of astroturfing.

When we think of college courses (or at least when I do), we think of them as opportunities for groups of learners to explore a topic together, typically guided by an expert in the field. When people are entering these courses with their own agendas that directly conflict with the material of the course, that opportunity is compromised.

So what do we do about this? I’m not sure. One could argue in favor of a heavy-handed approach, that anyone arguing for pseudoscience or false science in a college science course is simply kicked out; but the targets of that heavyhandedness would likely take this as a sign that science fears their alternate viewpoints and would sooner silence them than engage them in discussion. Those political groups would insist that it is we scientists who are concealing an agenda in our MOOCs. That is, of course, not the case; but an introductory science course is not the place to get into debates with people that have no intention of ever changing their viewpoint. Complicating matters further, though, is that for some of these people, their participation in these MOOCs is earnest: for many, this may be the first time they’re exposed to the scientific process, and they are willing to learn, and if we merely dismiss them as another fundamentalist with an agenda we miss the opportunity to introduce them to a broader view of the world.

So, a heavyhanded response is likely not the best option. Do we engage them in authentic discussion? That is a risky endeavor. For one, engaging in a discussion with completely unsubstantiated viewpoints risks giving them undeserved legitimacy. For another, any authentic discussion depends on both parties discussing on the same grounds; if we disagree on fundamental truths, then we may not ever meet a resolution. If we debate with someone who does not understand or believe the nature of evidence and theory-building, there is no shared understanding that can arise. It is a waste of our time and a needless distraction for the class.

If both shutting them down and engaging them in discussion are undesirable, what then can we do? Right now, my only idea is a third option: attempt to prevent them from entering the class in the first place. Of course, we can’t and shouldn’t do “interviews” for admission to a class for the same reasons stated above, but there are other ways that we can limit the participation of people with little desire to actually authentically learn. The first to come to mind for me is to introduce a barrier to entry. Attach a cost to pushing for your agenda. When anyone can “walk” in and start spouting off with their views as if they are as qualified as the professor, it is unsurprising that we get people promoting their views rather than engaging with course content. If a cost is associated with doing so, I would argue you limit the number of people who enter with an agenda rather than the desire to learn. Of course, this introduces the side effect of potentially preventing authentic learners from joining the class as well; however, we know that paying students are already far, far more likely to complete courses anyway. Depending on how you explain this data, this would mean that either those students who no longer join when there exists a price were not likely to complete the course anyway, or that forcing those students to pay will increase their likelihood of completing the course. Either option is desirable, though it is up for debate whether those would be the actual effects.

This isn’t all or nothing, of course. One could imagine a hybrid model where the payment associated simply gets you access to the community features, but that the content is available for free. This would preserve Coursera’s MOOCs’ desired openness while also attempting to safeguard its courses from outside influences. And, as a happy side effect, the money raised through this mechanism could go toward funding the classroom experience, paying for graders, compensating professors for their time, and building up the platform. I know I’d be very curious to see how the community for this class operated if I had the option of only interacting with those students also paying to participate in the course.

The conflicting functions of universities

I alluded to this in my previous blog post, but I think it deserves some explicit attention. In some of my recent conversations about the direction of the Online Masters of Science in Computer Science at Georgia Tech, I’ve come to believe that this question is at the core of many of the questions we wrestle with when developing courses in higher education.

What is the function of college? In other words, what goal does college accomplish? That’s a big nebulous question, so let me narrow it down a bit: what is the function of undergraduate education at research universities?

I see two competing functions. First, I see the function that higher education sees in itself. As seen by many professors and administrators,  I would argue that the function many see in higher education is the creation, maintenance, and dissemination of knowledge. Research institutions are called ‘research’ institutions for a reason, after all, but research is not merely about uncovering knowledge; it’s also about communicating knowledge so that a new generation can grow that knowledge. Thus, I would argue that teaching is well within the function of universities, as seen by universities themselves (despite data indicating the contrary).

That’s the function I would argue universities see in themselves. However, I would argue that there is a conflict between that function and the function that students derive from college. For the most part, students attend college to get a better job. To be somewhat unscientific, a quick Google search on “why go to college” corroborates this: the majority of the top results focus on the increased earning potential of individuals who attended college. Other benefits come up as well, but given the massive investment in getting an undergraduate degree, it’s certainly reasonable to expect a sizable return on investment.

So, to summarize that, on the one hand we have universities that are focused on the creation and dissemination of knowledge, and on the other we have students focused on job training and placement[1]. That Venn diagram may sometimes have an intersection, but I would argue that most often, it does not. The things we learn in undergraduate classrooms often have very little applicability to the real world. Getting an undergraduate education is less often about what you learn and more often about proving that you can do so; a person that can graduate from Georgia Tech with a degree in Computer Science can learn the skills necessary to do a particular job, but simply having that degree does not suggest they already have those skills.

This is often a point of contention for students in my experience. Students often ask when they will use something in the real world. In our Knowledge-Based AI class, we do often receive feedback from students that they would like to see examples that are more applicable to their job. While this feedback is understandable, it doesn’t fall into the function most of us attribute to the Masters degree. Others will disagree with me, of course, but I would argue that few research universities view their degree programs as job training, and thus, few would be eager to revise their curricula to bring them more in line with the demands of the job market.

So, we have students treating higher education as job training, while higher education does not regard itself as job training. Is this a problem? I would argue yes; the massive cost of higher education is associated with the cost of the creation and dissemination of knowledge, not with cost of job training. Getting a Bachelors degree to get a good job is like buying house just to have a front lawn. If job training is the goal, we can accomplish it more cheaply and efficiently, while also allowing research universities to focus on the types of knowledge creation that they were built to do.

Ultimately, I would argue that the goals of research universities and the goals of students at research universities are remarkably misaligned. Moreover, I would argue that universities shouldn’t modify themselves to come more into alignment with what students are using them for; alternative, more affordable solutions are necessary to allow job seekers to get job training and knowledge seekers to get knowledge.

Full disclaimer: I work for Udacity, and Udacity’s Nanodegree credentials are a step in the direction of streamlining the function that I claim students are deriving from higher education. You wouldn’t be off-base to believe I’m simply biased in favor of the company I work for. In actuality, however, initiatives like the Nanodegree credentials are why I work for Udacity; I wholeheartedly believe in its mission to make it easier for students to get what they’ve wanted all along.

[1] I don’t mean to claim this distinction is purely black-and-white; there are certainly students that go to college to learn and grow knowledge, and there are certainly departments and individuals at the college level that are concerned with their graduates’ job placement. When thinking about the primary goals these groups have in mind, though, I’d argue my distinction holds: students primarily use college as a step toward a career, and universities generally consider themselves to be bastions of knowledge.

Graduation rates: Look to your left. Now look to your right. Now look in two other random directions.

I feel I should start this post by saying I’m not advocating a return to the old way of doing things. The purpose of this post is just to note a change and ponder a bit what the change means.

An article last week talked about Georgia Tech’s moves to make student retention and graduation among its top priorities. The goal is to keep and graduate more students. That’s a good thing, right?

Many years ago, it wouldn’t have been considered a good thing. Georgia Tech used to be famous in part for its “look to your left” speech at convocation. In Thomas Friedman’s The World is Flat, Friedman includes an excerpt from former Georgia Tech President G. Wayne Clough. Clough recalls, “When I came to Tech as an awestruck freshman back in the sixties, they had this drill for the incoming students. They would tell us: ‘Look to your left. Look to your right. Only one of you will graduate.'”

Back then, a low graduation rate was a point of pride. And in some ways, doesn’t that make sense? If fewer people are able to accomplish something, doesn’t that mean the task is harder? And if the task is harder, doesn’t that make it more of an accomplishment if you do succeed at it? To take an analogy, isn’t completing a marathon more prestigious than completing a 5K because fewer people can do it, because it’s more challenging?

We’ve gone from bragging about our low graduation rate to treating it as a problem to be repaired. What happened? Perhaps we started caring about our students more. Perhaps we recognized that it was the wrong 33% of students graduating, that the reasons students were failing out weren’t correlated with long-term success. Perhaps we recognized that when students leave, they take their tuition checks with them.

Today, Georgia Tech graduates 82% of its students within six years of matriculation. Today, the speech would read, “Look to your left. Now look to your right. Now look at two other random people in the audience. Four of the five of you will graduate within six years.” Which begs the question: is a Georgia Tech diploma as prestigious today as it was fifty years ago? If 80% of students that attempt one can complete it, is it easier than if only 33% of students can complete it? If the ranks of marathon runners expanded 250%, would we not regard the accomplishment as less prestigious than previously?

Grade inflation would lead to higher graduation rates. Easier classes would lead to higher graduation rates. There are many ways we can increase graduation rates at the expense of the education. Just as we could increase the ranks of marathon runners by shortening the length we call a ‘marathon’, we also can increase our graduation rate simply by decreasing the challenge. There are lots of ways to increase graduation rates without negatively impacting the education, though. We can get better at choosing students who are likely to succeed in the first place. We can provide better structures and support environments to maximize the likelihood of student success. We can improve the quality of the instruction to ensure students learn more. We can only admit the best runners, we can make sure the route is fair and the weather is good, and we can provide good hydration along the course to maximize our marathon runners’ chances at success.

But at a certain level, don’t we want to know which students could succeed without that support? College serves many functions: one is to teach a body of knowledge in a certain field, and toward that end, improving the instruction and environment is a highly desirable move because it increases the number of students that attain that body of knowledge. But college is also a test: it examines whether one can succeed at a rigorous course of study. By improving the instruction, do we not inherently lower the rigor? Do we not want to know which students would have succeeded under more rigorous conditions? Aren’t we curious which of those marathon runners could have also completed the race on a hilly course on a rainy day without food or water?

I don’t have an answer here. This comes down, in part, to the function of college: is it to obtain a corpus of knowledge, or is it to prove a level of ability? Who even gets to decide the function? I would speculate universities would favor the former function, but for the vast majority of students, the reason for earning the diploma is to get a job, and those jobs often involve few of the skills learned in the classroom. A Georgia Tech diploma, in my experience, is less an indicator of what you know and more an indicator of what you can do, which would tend to lean toward the latter. If we can design college programs that are more closely tied to the requirements of the job market, we may be able to move toward the former function of higher education, but I don’t believe higher education is interested in reinventing itself as job training — even though that may be one of the major functions it plays in present society.

There are many complicated relationships at play here, but I can’t get over a simple interesting fact: in a span of only a few decades, Georgia Tech went from bragging about its low graduation rate to bragging about its high graduation rate. That’s a fascinating about-face. For my part, I agree with the new focus: we should do everything in our power to equip our students to succeed, as well as to ensure we’re only giving the opportunity to the students who have the potential to succeed in the first place. But I can’t deny that as an undergraduate at Tech, I took great pride at succeeding at a school where success wasn’t guaranteed.

The vocabulary of mindsets

I’ve mentioned in the past that I’m a strong believer in Carol Dweck’s work on Mindsets. Our daughter is only four weeks old now, but even at this age I’ve already become aware of how easy it is to slip into fixed mindset dialog with her. We noticed after a single week she was holding her head up, and our natural instinct was to refer to her as smart, yet this compliment is exactly the opposite of what the growth mindset would tell us to give.

I buy in strongly to the mindset idea, but even I find it extremely easy to slip into fixed mindset compliments and praise. Why is that? I’ve thought about it a good bit over the past couple weeks and even more historically with my tutoring and instructor responsibilities. Growth-oriented compliments tend to have a negative connotation to them for many students: “You really tried hard on that!” is heard as “You really had to try hard to succeed at that!” Next to a classmate that completed a task easily, “You really persevered despite the difficulty!” can be heard as “You encountered difficulties that others didn’t encounter.”

Part of this is a self-fulfilling prophecy: it is precisely the stigma that the fixed mindset gives to effort that causes the negative connotation students perceive in these compliments. However, I believe there is more to it than that. The fixed mindset gives us simple, accessible, popular compliments: you’re smart, you’re brilliant, you’re artistic, you’re athletic, you’re amazing. The growth mindset cautions us against these positive labels because they teach students to protect their identity rather than take risks and grow; yet, it remains difficult to avoid using these labels simply because of their availability and ease. Thus, even without intending to, we may easily introduce our kids to the fixed mindset. I’m thankful I have at least a few more weeks of Lucy hearing my speech as incoherent babbling to practice avoiding such praise!

The simplest way to avoid these tendencies, of course, would be replacement rather than simply avoidance: rather than avoiding fixed mindset praise, I ought to replace it with growth-oriented praise. It is when a student struggles and eventually conquers a hard-fought challenge that they should be praised, and in those situations it is their effort and perseverance that should be praised rather than some innate ability. In Dweck’s experiments, this was often the experimental condition: not simply withholding fixed praise, but providing praise for the right element of the task.

In Dweck’s work, this took the form of things like, “That’s a really good score. You must have worked really hard.” The target of the praise is the work involved, not some inherent personal characteristic. But the fixed mindset compliments that we aim to avoid don’t simply come up in well-articulated and firmly-structured tasks: they arise in the more mundane day-to-day bits of cleverness or insight we provide. A student notices a deeper connection between a problem in math and a problem in physics. A child realizes that her father always comes home early on Fridays. A teacher recognizes a deeper misconception in a student’s understanding.

In these instances, it is natural to say, “Wow, you’re so smart”, but it is not desirable. Yet, praise is not the villain: the villain is what we praise. ‘Smart’ gives us a quick and easy way to praise the wrong target, but what word would we use to praise the right target?

I have yet to find a suitable vocabulary for growth-oriented praise. In the right situations, compliments like “You must have worked really hard” work fine, but in a great many contexts where positive reinforcement is desirable, that compliment doesn’t work. In fact, many of the words we would use to describe someone who puts forth a lot of effort and perseveres despite challenges have a strongly negative connotation: stubborn, headstrong, obstinate. Even the more neutral words still err on the negative side: strong-willed and persistent are neutral at best. Thus far, the only suitable word I’ve found is tenacious, and even that isn’t precisely matched to the growth mindset; its connotation is at least more positive, but it is not hard to imagine a growth mindset without tenacity.

It is easy to tell my daughter that she’s smart: “you’re smart!” But how do I tell my daughter that she perseveres despite challenges, that she always works her hardest, that she puts forth a lot of effort? The vocabulary simply isn’t as clear. I would argue this is one of the reasons the fixed mindset is so pervasive: it’s simply easier. It’s easier to tell my daughter she’s smart than it is to tell her she’s… I can’t even finish that sentence because the vocabulary isn’t available!

Perhaps this is intentional, of course. The accessibility of vocabulary is so we can label individuals, and the growth mindset generally pushes against labels as a whole. We could shove mindsets down their own throat and discuss how a person has a mindset about their mindset, and being able to praise their current mindset might undermine their effort to improve their mindset. I’d reject this notion, personally: if a person has adopted a growth-oriented mindset, then they will properly frame any praise for their growth-oriented mindset. But I digress.

What is particularly interesting is that while there is a vocabulary for fixed-oriented praise that is absent for growth-oriented praise, both fixed and growth mindsets have a vocabulary available for criticism. You can be dumb, stupid, unimaginative, boring, slow: fixed mindset criticisms are easy. But you can also be a quitter, a pushover, a slacker, a loafer. These criticisms target growth mindset characteristics: how well one perseveres, how much effort one puts forth, and how firmly one pushes for their ideas.

So what are the opposites of these growth-oriented criticisms? The closest thing I can find in the dictionary is ‘stalwart’, a word with a very different connotation. ‘Go-getter’ has an entrepreneurial connotation. ‘Self-starter’ focuses on the phase before obstacles may arise. That’s all I can find that even comes close.

It’s simply difficult to describe a person with a growth mindset as easily as we label people as ‘genius’, ‘idiot’, or ‘quitter’. Even tenacious, the only word that even partially appeals to me, lacks a ‘label’ form of the word that can be used. To more easily encourage people to adopt a growth mindset, there needs to be an accessible vocabulary in which we can even identify the growth mindset in the first place.

So: what would you call someone with the growth mindset?

Activity perception in an online classroom

As part of the first week of KBAI this semester, we posted our Brown Bag talk for new students in the class to watch. This spawned an excellent discussion of what explains my perception of how engaged and motivated OMS students are to drive the classes and program. Is it that OMS students are more motivated entering the program due to demographic or experiential differences? Or does the design of OMS classes actually improve students’ engagement compared to in-person classes?

During this discussion, however, a third idea emerged, based in large part on suggestions from two students (whose names are temporarily withheld until I find out if FERPA would allow them to consent to having their names disclosed here). It might not be either of the above: OMS and traditional students could be equally engaged both before and during the class, but the specific elements of the construction of the online section may better channel that engagement and motivation into visible results.

From past feedback and common sense, we know that activity breeds activity: students are more likely to be engaged when they perceive that other students are engaged as well. Similarly, when students perceive that others are bored, they are more likely to become bored themselves. How do students pick up on this engagement and boredom? In-person, students can read body language, attendance percentages, and enthusiasm during interactions, and all of these can be read either positively or negatively.

Online, however, something interesting occurs. When Piazza is the classroom, it seems to become more difficult to have these negative perceptions of activity. In-person, for instance, we can very quickly evaluate attendance percentage; online, however, there is no straightforward way to gauge how many students are participating, and thus it is more difficult to perceive that a significant portion of the class is failing to participate. Online, we tend to lack the type of “null” participation that often characterizes in-person interactions: students who would not be enthusiastic about discussions or activities simply don’t participate, and what is left over is a superficial impression of high engagement and motivation. Why? Because only the engaged and motivated individuals are participating visibly.

Of course, this may be a double-edged sword. We as instructors may perceive a class as active and engaged when in reality, only 10% of the class is interested; that 10% is simply the only 10% that we see, and there are no “empty chairs” continuously reminding us of how many students are not engaged. As instructors, we need to be aware of the activity of the class as a whole, not just of the superficial perception we get when we login each day. However, if we believe the idea that the perception of enthusiasm and engagement begets more enthusiasm and engagement, then the tendency of an online class to only show the active individuals can have a tremendous positive impact.

This idea actually builds on the ownership angle I’ve described in the past. I mentioned then that there are certain things, like proposing their own discussions, sharing articles or life experiences, and maintaining conversations that target multiple levels of aptitude, that are uniquely common in an exclusively online classroom. Those same behaviors are also visible indicators of engagement and enthusiasm that have no (in my opinion) direct traditional analogue, furthering the impression that engagement and enthusiasm is higher online. Thus, it might not be the case that online students are ever more engaged and enthused than traditional students: instead, it might simply be the case that the online environment better facilitates the expression of that engagement and enthusiasm. Of course, if that is the case, it further stands to reason that engagement and enthusiasm will increase online due to the prior note that motivation breeds motivation.

Intelligent tutoring online

My background is in intelligent tutoring systems. In intelligent tutoring systems, an artificial intelligence agent typically monitors student performance and reacts accordingly, giving feedback or support where necessary.

One of the interesting things about developing an online class is that because students are already engaging with a software system, the infrastructure and context necessary for an intelligent tutoring system are already present. What’s more, not only are they present, but they’re directly integrated into the context of the lesson. Whereas oftentimes intelligent tutoring systems are separate activities that complement a previously-received lecture, putting the learning online from the get-go allows us to integrate intelligent tutoring directly into the context of the lesson.

This development is in its infancy as far as I’m concerned, but I wanted to explain one way in which we use this in our Georgia Tech OMS class. Throughout the course, we have 125 interactive exercises each equipped with an AI agent – which I’ve taken to calling a ‘nanotutor’ to reflect the tiny scope of the skills that these agents teach – that gives students feedback on their latest responses. Let’s walk through an example of an exercise.

Continue reading Intelligent tutoring online