Course Review: Developing Innovative Ideas for New Companies: The First Step in Entrepreneurship

As I mentioned a few weeks ago, I recently took the Coursera class Developing Innovative Ideas for New Companies: The First Step in Entrepreneurship (or Developing Innovative Ideas for short) from the University of Maryland‘s Dr. James V. Green. This course is part of the Entrepreneurship: Launching an Innovative Business Specialization. This review is written based on the course as it existed in April 2015.

This review serves three functions:

  • For prospective students, should I take this course?
  • For prospective employers and admissions offices, what does having completed this course say about a potential employee or student?
  • For course developers and educators, what can I learn from how this course is developed and delivered?

This review will contain two parts: an objective description of the course content, structure, and experience, and a subjective analysis of the course’s value to students, employers, and educators.

Course Description

Developing Innovative Ideas is a four-week course that explores various aspects of entrepreneurship, such as the perspectives, mindsets, and motivations of entrepreneurs and some basic understanding of the customer and the industry. The full description is available here. The course describes itself as:

Explore how to identify and develop great ideas into great companies. Learn how to identify opportunities based on real customer needs. Take the first steps to creating a successful company.

Structure

Developing Innovative Ideas is a four-week class, with each week presenting a different overall unit. Within each unit, two “experiences” are provided: the “Studio Experience” and the “Classroom Experience”. The “Studio Experience” is the traditional talking head Coursera production style. The “Classroom Experience” is a filmed lecture from the on-campus version of the class at University of Maryland. Each week concludes with a short quiz consisting of multiple choice and auto-graded free response. The course also supplies the typical discussion forums and an optional book.

Overall, Developing Innovative Ideas can be completed in around 15 to 20 hours of work, depending on how one watches the videos. The total video time is around 15 hours, but much of that is in the less efficient Classroom Experience videos which overlap and repeat the more efficient Studio Experience videos; watching only the studio experience cuts the video time down to around 5 hours. The quizzes are non-trivial and will take some time each.

Content

Developing Innovative Ideas covers four units:

  • Introduction to Innovation and Entrepreneurship. An overview of what entrepreneurship is, what innovation is, and how entrepreneurs make decisions and analyze opportunities.
  • Entrepreneurial Mindset, Motivations, and Behaviors A look at what motivators entrepreneurs, how they frame problems, how they evaluate risk, and how they make decisions.
  • Industry Understanding. A look at how to analyze an industry, how to frame one’s innovations in terms of that industry, and how companies succeed and fail based on their understanding of an industry.
  • Customer Understanding and Business Modeling. An examination of where market needs come from, how to identify them, and how to best take advantage of them.

The content of Developing Innovative Ideas is typically presented as a talking head alongside some bullet point PowerPoint slides, although other visuals are used frequently as well. For the Classroom Experience, the presentations are usually lecture-based with audience interaction. The audience for the material is clearly future entrepreneurs: this isn’t a class for researching and analyzing entrepreneurship, but rather a class for learning to actually be an entrepreneur. This class in particular covers the early life cycle of a new innovation or business: identifying the innovation in the first place, evaluating its position in the market, and setting up a preliminary business plan.

Assessments

There is one type of assessments in Developing Innovative Ideas: quizzes. Total, there are five quizzes in the course: one each week, and an additional quiz at the end. The quizzes are made of both multiple choice and free response, although the free response is auto-graded rather than peer graded. The free response quizzes aren’t just completion quizzes; I once got a question wrong according to the autograder, and then got it right after following up and revising it. However, little information was provided as to how these autograders work, so I am not sure how complex their analysis was; I suspect (although this is just conjecture) they do not operate according to much more than a keyword search. While I had one question marked wrong, I know other questions would likely have been penalized by a human grader.

Prerequisites

There are no knowledge or ability prerequisites for this course. However, interest in actually starting a business could be described as a prerequisite; without that interest in personally participating in the subject matter covered by the class, the class is not likely to be engaging.

Identity Verification

For Verified Certificates ($50 at the time that I signed up), students complete the Coursera two-phase authentication: a typing sample is submitted and a picture is taken of the student immediately after completing the assessment. Assuming that these are adequate in ascertaining the student’s identity, they are nonetheless not difficult to circumvent. It would be trivial to collaborate with someone else on the assignments or even outright submit someone else’s work. This verification only assures that the student was present when the work was submitted.

If the student is not enrolled in a Verified Certificate, there is no identity verification at all.

Course Analysis

For the most part, Developing Innovative Ideas is presented in the traditional talking head style. The Classroom Experience departs from this style a bit, but the videos are so much longer while covering the same material that it is difficult to consider them an option. Even then, Developing Innovative Ideas operates under a rather traditional lecture model.

The assessments used in Developing Innovative Ideas are interesting. While the autograded multiple choice fall prey to the typical trial-and-error approach, the autograded free responses may provide some more rigorous assessment. At the least, there are steps in place to ensure students actually answer them with the right keywords, although I am unsure if it goes far beyond a simple keyword search. Trial and error is still plausible here as feedback is given after submitting the quiz, but the retake at least involves trying to put some things in one’s own words.

Aside from the lectures and the quizzes, there are not a lot of other activities in Developing Innovative Ideas. Discussion forums are provided, but they’re more for interacting and networking amongst peers than for participating in discussions with instructors. The discussion boards are structured to give students to give students a chance to network with people in their same geographic area or industry, as well as to find collaborators for new ideas. Toward that end, Developing Innovative Ideas may present a useful networking opportunity, though I’m not sure how effective that is in practice.

Overall, Developing Innovative Ideas is a class for prospective entrepreneurs to learn the first steps to take, and toward that end, it is very good. The information it presents is non-obvious but accessible, and it goes to an adequate depth to get students ready for the next courses.

For Prospective Students

Developing Innovative Ideas is absolutely a class for prospective entrepreneurs. If you don’t have much interest in starting a business, it won’t appeal to you; but then again, if you don’t have an interest in entrepreneurship, you probably wouldn’t even bother reading a review of the course. The course’s material is accessible to novices, so few if any prerequisite skills are required. On the flip side of that coin, though, the material is so accessible that experienced entrepreneurs might find it boring or unimportant.

Take this course if:

  • You have an interest in starting a business, but aren’t sure how to identify a good idea.
  • You have an idea, but aren’t sure how to go about transforming it into a full business.
  • You work for a company involved in innovation and development (even if it isn’t a entrepreneurial start-up) and want to contribute to that side of things.

Don’t take this course if:

  • You’re an experienced entrepreneur; there likely isn’t much new here.
  • You don’t have a personal interest in starting a business or contributing to your company’s innovation.
  • You’re looking for feedback and insight into the business plan or idea you yourself have; you aren’t likely to find much aside from your peers on the forums.

For Prospective Employers and Admissions Offices

With regard to identity verification, the identity verification for the Verified Certificate is trivial to circumvent. Thus, a Verified Certificate in Developing Innovative Ideas should carry limited clout for that reason alone. The remaining analysis will infer that the student did complete the certificate themselves.

With regard to assessment, the assessments in Developing Innovative Ideas do require a bit of effort, although they can largely be accomplished via trial and error. Even the free response questions, given the apparent simplicity of the autograder and the feedback provided after wrong answers, can easily be gamed into a good score. Overall, passing the assessments in the course is not indicative of mastery of the course content.

With regard to content, the course does cover some excellent information regarding starting new businesses and innovating within current businesses. If I could trust that a person completing the course had actually mastered the material, the material itself is certainly valuable for an applicant to have.

Based on the issues with identity verification and assessment, simply seeing a Verified Certificate for Developing Innovative Ideas on an applicant’s resume or application ought not carry much weight. However, an interviewer should absolutely press on the skills covered in the class a little to conduct some assessment themselves. While simply having the certificate isn’t meaningful on its own, if the applicant is able to answer a few questions about the topics covered in the course, then it should indicate that they have mastered the course content and count as a major point in their favor. In other words: evaluate for yourself whether the applicant mastered the material, knowing that if they did, then the skills they acquired are valuable.

It is also worth noting that while assessment in this course is a bit lacking, the later courses in the specialization are a bit more rigorous, and mastery of these skills is required to understand and succeed in later classes. So, the course on its own might be of limited value on an application, but the specialization of which it is a part is certainly more valuable.

For Course Developers and Educators

There are three main things in Developing Innovative Ideas that I’d like to see more courses do:

  • Vary the viewing experience. While I didn’t take advantage of the Classroom Experience much, I did enjoy it the few times I used it. It could have been made more efficient, and the overlap with the Studio Experience could have been limited, but it was definitely interesting to see the different types of understanding that could be offered by efficient parades through the raw course material and more casual discussions with a live class.
  • Introducing a specialization. Developing Innovative Ideas is a great introductory class to a specialization, and having already taken the latter two classes in the Entrepreneurship specialization, it does a great job of setting up the overall thought processes and skills for the specialization as a whole. It would be easy to try to put together a class that covers everything, but the extent to which Developing Innovative Ideas understands its function in its specialization improves both the class itself and the rest of the course sequence.
  • Auto-grading free response. As I mentioned, I’m doubtful as to the effectiveness and accuracy the autograder that evaluated free response exercises. However, the idea is a promising one. I can imagine leveraging machine learning to train a grader on human-graded essays, then turning it loose on essays out in the rest of the course. I don’t think that’s what happened here, but it would be worth exploring.

For more information, please see the course web site. If I’ve left out anything crucial to understanding this course, let me know below!

Share on Google+Share on FacebookTweet about this on TwitterShare on LinkedInShare on RedditPin on PinterestEmail this to someone

Course Review: Emerging Trends & Technologies in the Virtual K-12 Classroom

As I mentioned a few weeks ago, I recently took the Coursera class Emerging Trends & Technologies in the Virtual K-12 Classroom, from the University of California-Irvine‘s Melissa Loble. This review is written based on the course as it existed in March 2015.

This review serves three functions:

  • For prospective students, should I take this course?
  • For prospective employers and admissions offices, what does having completed this course say about a potential employee or student?
  • For course developers and educators, what can I learn from how this course is developed and delivered?

This review will contain two parts: an objective description of the course content, structure, and experience, and a subjective analysis of the course’s value to students, employers, and educators.

Course Description

Emerging Trends & Technologies in the Virtual K-12 Classroom is a five-week class that serves as a survey of up-and-coming technologies being used in K-12 classes, such as learning management systems, social network integration, gamification, MOOCs, and open educational resources.  The full description is available here. The course is self-described with the following:

Learn about emerging trends and technologies in K-12 virtual instruction. Join us as we explore this dynamic landscape and investigate how we can more deeply engage students in the virtual classroom through the use of innovative practices and technologies.

Structure

Emerging Trends & Technologies in the Virtual K-12 Classroom was a five-week course, with each week consisting of a different “module”. The final week was light on material, but the other weeks covered around two hours of video material each. Each week concluded with a short five-question quiz, and the course concluded with a 35-question final exam. The course also featured a short peer reviewed assignment halfway through.

Overall, the course can be completed in around fifteen total hours including all activities. The lecture material averages two hours per week, while the quizzes, assignment, and peer review total likely another hour.

Content

Emerging Trends & Technologies in the Virtual K-12 Classroom is broken into five modules, although the fifth module is merely a short ‘conclusion’ module. The other four modules are: “The Role of Technology in Virtual Education”, “Social Technologies in Virtual Education”, “Game-Based Learning & Badges in Virtual Education”, and “Open Content in Virtual Education”. Each module explores a variety of different tools within that topic. The third module on game-based learning, for instance, examines gamification, games used for learning, augmented reality, and wearable technology.

By design, Emerging Trends & Technologies in the Virtual K-12 Classroom is something of a practical survey course. It does not cover any learning science or psychology in why these technologies work. It similarly does not go into great detail on how they work or how to use them. The main objective of the course is simply to introduce the tools and give some high-level guidelines for what to do and what to avoid in attempting to use them in the classroom.

In this way, the target audience of Emerging Trends & Technologies in the Virtual K-12 Classroom is very clear: teachers. This isn’t a course for developers of new educational technology, or researchers interested in how educational technology affects the classroom experience. This is a course for teachers who are interested in using educational technology in their actual classrooms tomorrow, but simply don’t know what’s out there or how to best use it.

Assessments

There are three types of assessments in Emerging Trends & Technologies in the Virtual K-12 Classroom:

  • Four five-question multiple choice quizzes, each of which can be taken up to three times and which show the student what they got wrong after each take.
  • One medium-length writing assignment (1500 word) asking the student to design and justify a learning activity based in educational technology.
  • A final quiz of 35 multiple choice questions which can be taken three times but which does not give students detailed feedback on their wrong answers after each take.

The assignment is graded through peer review, with three peers grading each assignment and each peer grading three assignments.

Prerequisites

In terms of knowledge, there are no prerequisites to this class. Experienced teachers will likely have a stronger foundation from which to build, but there is no content contingent on teaching experience. In terms of appeal, however, only those with a background in teaching are likely to be interested in the course content.

Identity Verification

For Verified Certificates ($50 at the time that I signed up), students complete the Coursera two-phase authentication: a typing sample is submitted and a picture is taken of the student immediately after completing the assessment. Assuming that these are adequate in ascertaining the student’s identity, they are nonetheless not difficult to circumvent. It would be trivial to collaborate with someone else on the assignments or even outright submit someone else’s work. This verification only assures that the student was present when the work was submitted.

If the student is not enrolled in a Verified Certificate, there is no identity verification at all.

Course Analysis

Emerging Trends & Technologies in the Virtual K-12 Classroom uses a traditional talking head approach alongside bullet-point slides. The speaker, Melissa Loble, presents the material in a friendly and engaging fashion, although the presentation of the material is always limited by the reliance on text slides rather than visuals or demonstrations. Given that the nature of the course is to inform teachers of the tools available to them, though, this isn’t overly problematic, although some demonstrations certainly could have improved the course.

As mentioned before, the content is most appealing to teachers looking for something practical to do in their classrooms immediately after finishing the course. That’s one of my hats, and so I was interested in how some of these things might be used. However, I also work in educational technology research, and I was a little disheartened by some of the course content. The course, for example, speaks in glowing terms of the appeal of gamification to create extrinsic motivation, but significant literature exists to suggest that relying on extrinsic motivation may ultimately hurt students.

As mentioned previously, there are three types of assessments. The short repeatable quizzes do not reflect much deep learning in my opinion; as I’ve referenced before, it’s often trivial to get a good score on these just through trial and error. The final exam, on the other hand, is more meaningful: 35 questions without receiving feedback on individual wrong answers makes for a more reliable assessment, even if multiple choice questions are still a poor way to evaluate deeper learning. The peer reviewed assignment was interesting and one of the more thorough ones I’ve completed as part of a Coursera course, but the evaluation was disheartening. Research indicates peer review can be as reliable as human grading, but that research relied on five peer graders. Emerging Trends & Technologies in the Virtual K-12 Classroom, like most Coursera classes, uses only three. For me, I received an average of 40 words of low-quality feedback per reviewer, and the lack of assessment and feedback pull the chair out from under the usefulness of the assignment, unfortunately.

Aside from the video lectures and the assessments, the course also has significant additional activity. In addition to the discussion forums, accounts or communities were provided through Twitter, YouTube, Facebook, Pinterest, and Google+ to demonstrate the use of those tools in a MOOC like this. The course also demonstrated the use of live Twitter chats and Google Hangouts on Air, with eight combined synchronous events.

Overall, this is a teaching class taught by teachers, not learning scientists. This means it is very practical and grounded in real technologies and opportunities, but it also means that it occasionally missteps and praises interventions that other researchers might caution against using.

For Prospective Students

Emerging Trends & Technologies in the Virtual K-12 Classroom is absolutely a class by teachers, for teachers. It won’t provide much interesting content for people working in other areas of educational technology like the business or research sides, but it provides an abundance of resources ready for teachers to use quickly in their classroom. It could use the perspective of a learning scientist to check if the tools suggested actually lead to desirable changes, but for students in its target audience, it has an abundance of interesting material/

Take this course if:

  • You’re a teacher, especially a K-12 teacher although higher-level instructors might be interested in some elements as well.
  • You’re open to using technology in your classroom, but aren’t sure how.
  • You have some experience with educational technology, but want to explore other possibilities.

Don’t take this course if:

  • You’re not a teacher.
  • You’re not able to engage yourself in optional activities; without the Twitter chats, Hangouts, and discussions, the class isn’t that impactful.
  • You come from a learning sciences, cognitive science, or psychology background.

For Prospective Employers and Admissions Offices

With regard to identity verification, the identity verification for the Verified Certificate is trivial to circumvent. Thus, a Verified Certificate in Emerging Trends & Technologies in the Virtual K-12 Classroom should carry limited clout for that reason alone. The remaining analysis will infer that the student did complete the certificate themselves.

With regard to assessment, the rigor of the course is slightly higher than many other MOOCs, and more importantly, that level of rigor is accurately assessed. The trial-and-error quizzes provide little and make up a significant portion of the grade. However, the final exam provides this accurate assessment as it’s not possible to simply trial-and-error to a good score, and at 45% of the course grade, it’s not possible to perform well in the course without performing decently on the final. The final isn’t exactly rigorous, but it’s at least reliable. The only other assignment is peer graded, and while I, personally, have some reservations about the validity and reliability of peer grading, research has shown this approach to be effective (with five graders, while this uses only three).

With regard to contentEmerging Trends & Technologies in the Virtual K-12 Classroom provides a good view of multiple strategies and technologies that can be used in a K-12 classroom. Many of these are extensible to other levels and areas as well, including higher education and corporate education. The content is exclusively relevant to teachers, so this course should not be relevant for an applicant for a non-teaching position, but for applicants to teaching positions it’s a credit to the applicant.

Personally, if I saw Emerging Trends & Technologies in the Virtual K-12 Classroom on an applicant’s resume for an application to a teacher position, I’d be most optimistic simply about the applicant’s desire to explore educational technology. While the course gives a good overview of tools, I’d be wary of whether or not someone can leverage these tools successfully based on this course alone; however, the interest in doing so is the first step toward doing so successfully, and completion of this course adequately demonstrates that interest.

For Course Developers and Educators

There are three main things in Emerging Trends & Technologies in the Virtual K-12 Classroom that I’d like to see more courses do:

  • Think outside of the video. As I mentioned above, if you’re not going to at least watch the Twitter chats and Google Hangouts, this class is not very impactful. It’s these demonstrates of educational technology in action that make the class more worthwhile than many traditional MOOCs. The videos are pleasant to watch but not overly engaging, but the additional activities and interactions set this course apart.
  • Target your audience. I’ve noted a few times that people involved in the business or research sides of educational technology are not likely to get much out of the course. That’s notable to me since I’m involved in all three sides, but for the majority of people this structure would be optimal. It’s better to have three short courses that each are perfectly targeted to one-third of the audience than to have one long course that two-thirds of the audience is going to find boring and uninteresting at any given time. MOOCs give us the flexibility to segment like this.
  • Know your audience. Narrowly targeting one audience is great, but it can fail if you are not also aware of that audience’s interests. A course targeted to teachers that focused on the business elements of educational technology companies would not go over well because it fails to understand what its audience is actually interested in seeing.

For more information, please see the course web site. If I’ve left out anything crucial to understanding this course, let me know below!

Share on Google+Share on FacebookTweet about this on TwitterShare on LinkedInShare on RedditPin on PinterestEmail this to someone

Course Review: Astronomy: Exploring Time and Space

As I mentioned a few weeks ago, I recently took the Coursera class Astronomy: Exploring Time and Space, from the University of Arizona‘s Dr. Chris Impey. This review is written based on the course as it existed in March 2015.

This review serves three functions:

  • For prospective students, should I take this course?
  • For prospective employers and admissions offices, what does having completed this course say about a potential employee or student?
  • For course developers and educators, what can I learn from how this course is developed and delivered?

This review will contain two parts: an objective description of the course content, structure, and experience, and a subjective analysis of the course’s value to students, employers, and educators.

Course Description

Astronomy: Exploring Time and Space is a six-week class covering the basics of Astronomy, including the tools of astronomy,  the planets and the solar system, extraterrestrial life, and the origins of the universe. The full description is available here. The course is self-described with the following:

This course is designed for anyone who is interested in learning more about modern astronomy. We will help you get up to date on the most recent astronomical discoveries while also providing support at an introductory level for those who have no background in science.

Structure

Astronomy: Exploring Time and Space was a six-week course. Each week consists of a couple hours of pre-prepared video material, a couple quizzes over the material, and either a peer-reviewed writing assignment or an interactive activity.

Overall, the course can be completed in around thirty total hours including all activities. The lecture material averages two to three hours per week, while the quizzes, assignments, activities, and peer review total likely another hour.

Content

Astronomy: Exploring Time and Space is broken into six units: “Our Place in the Universe”, “The Tools of Astronomy”, “Planets Near and Far”, “Star Birth and Death”, “Galaxies and the Big Bang”, “Life in the Universe”. Each unit is built around its overarching topic, and examines multiple elements of the topic. For example, the third unit — “Planets Near and Far” — covers the nearby planets, then the methods for examining planets further away, then interesting features of further-out planets.

Astronomy: Exploring Time and Space assumes nothing about the student’s knowledge; it is accessible to anyone. It goes so far as to start with the very basics of the scientific method: how science works, how observation, discovery, and reasoning lead to the scientific method, and how the scientific method is used in the context of astronomy. In line with this, Astronomy: Exploring Time and Space also keeps things high-level and accessible; little math is covered, and while the concepts covered are complex, the course focuses on giving beginner-level explanations. Rather than complex equations and slides of bullet points, Astronomy: Exploring Time and Space makes heavy use of animations, visualizations, and simulations. The course also injects a significant dose of humor and popular culture.

Assessments

There are three types of assessments in Astronomy: Exploring Time and Space:

  • Thirteen short lecture quizzes (six questions each), made exclusively of multiple choice questions and permitting three retakes.
  • Three short writing assignments (500 words each) that are graded via peer grading.
  • Two activities using external sites, graded via some multiple choice questions.

Assignments are assigned to three peer graders for peer review, and each student grades three classmates’ assignments.

Prerequisites

There are no necessary prerequisites for this class. I wouldn’t have any hesitation giving it to middle school students, and yet the course material should remain interesting to students with higher-level degrees but no experience in astronomy.

Identity Verification

For Verified Certificates ($50 at the time that I signed up), students complete the Coursera two-phase authentication: a typing sample is submitted and a picture is taken of the student immediately after completing the assessment. Assuming that these are adequate in ascertaining the student’s identity, they are nonetheless not difficult to circumvent. It would be trivial to collaborate with someone else on the assignments or even outright submit someone else’s work. This verification only assures that the student was present when the work was submitted.

If the student is not enrolled in a Verified Certificate, there is no identity verification at all.

Course Analysis

Astronomy: Exploring Time and Space uses a “talking head” approach throughout much of its presentation, but that approach is very often augmented with visuals, diagrams, animations, simulations, videos, and other forms of media. Additionally, despite the talking head approach, there are never any visuals resembling PowerPoint slides — the class is much more conversational. The ultimate effect of this is that the lectures are actually downright pleasant to watch. It feels like attending an interesting guest lecture or watching a well-prepared (albeit low-budget) documentary.

The assessments of Astronomy: Exploring Time and Space generally suffer from the structure I’ve seen commonly used in Coursera courses. The quizzes allow two immediate retakes, meaning that guess-and-check can earn a surprisingly good score on the quizzes. The writing assignments and the activities, however, do encourage significantly deeper understanding of the material for those topics covered by those activities — but as with most things, you get out of these what you put into them.

Aside from the lessons and the assessments, the course consists of the typical forums as well as a number of live sessions with Dr. Chris Impey. The forums, like many Coursera courses, are somewhat hit or miss: in this class a community didn’t seem to develop and most discussions did not go anywhere. The live sessions, however, were very interesting. Dr. Impey’s impromptu ability to answer some deep questions was particularly amazing. It would have been great to see an actual synchronous conversation on some of the questions that were brought up.

For Prospective Students

Astronomy: Exploring Time and Space is an excellent introduction to the high-level concepts of astronomy. It’s accessible to beginners in the field, so no prior knowledge is required. It nonetheless also goes deep enough into the topic to be fulfilling. I left the course wanting to know more, but I didn’t feel like the course itself should have gone in more depth on any particular topic. The title of the course is appropriate: it’s an exploration of time and space. It doesn’t cover every single thing, nor should it.

Take this course if:

  • You enjoyed Cosmos: A Spacetime Odyssey and want to learn more.
  • You have any interest in astronomy, but little prior experience.
  • You’re willing to do a little more than the traditional Coursera multiple choice quizzes (but not too much).
  • You have trouble learning from the traditional talking-head-and-slides presentation style.

Don’t take this course if:

  • You already have an intermediate background in astronomy (you likely know the course content already).
  • You’re looking for a class to replace a high school or college-level astronomy credit (the material and assessments aren’t quite deep enough to replace these classes).
  • You don’t want to write a couple essays.

For Prospective Employers and Admissions Offices

With regard to identity verification, the identity verification for the Verified Certificate is trivial to circumvent. Thus, a Verified Certificate in Astronomy: Exploring Time and Space should carry limited clout for that reason alone. The remaining analysis will infer that the student did complete the certificate themselves.

With regard to assessment, the rigor of the course is slightly higher than many other MOOCs, but not high. As referenced above, the multiple choice quizzes are easy to game without actually learning the material, meaning that the only reliable assessment is the written assignments. These are most certainly a matter of getting out of them what you put into them. In peer grading, I saw a wide variety of assignments, from comprehensive essays to three-sentence throwaway answers. While I, personally, have some reservations about the validity and reliability of peer grading, research has shown this approach to be effective.

With regard to content, Astronomy: Exploring Time and Space provides a strong overview of the basic principles of astronomy. Neither its coverage nor its assessments are sufficient to consider it as a potential replacement for high school or college credit, but it’s sufficient to demonstrate interest and qualification for taking a class that might otherwise appear to be out of the student’s ability level.

Personally, if I saw Astronomy: Exploring Time and Space on an applicant’s resume for admission to school or for a job, I would take it as a sign that the applicant has their own interests and is willing to put up more than minimal effort to pursue those interests. I wouldn’t derive any more conclusions based on this certificate, but that alone may be interesting to note.

For Course Developers and Educators

There are three main things in Astronomy: Exploring Time and Space that I’d like to see more courses do:

  • Diversify video content. Many courses are almost entirely comprised of a talking head alongside a set of PowerPoint slides. I understand the temptation of doing that; it’s easy, and Coursera courses aren’t money-makers, so the costs of production need to be kept low. However, I would not speculate that Astronomy: Exploring Time and Space cost more to produce than others; the time spent developing slides was simply instead spent gathering visuals and other resources. That’s a better use of the time in any course that can present content using something besides bulleted lists of vocabulary words and concepts. Astronomy: Exploring Time and Space is the first MOOC I’ve taken that I can imagine airing on television for a casual audience.
  • Diversify assessments. Many courses rely only on auto-graded multiple choice quizzes with several retakes. I find very little reliability in a person’s record from a course entirely made up of these assessments; anyone can complete them. The peer-graded assessments in Astronomy: Exploring Time and Space as well as the lecture activities (even though they are evaluated by similar quizzes) at least require student participation that cannot be totally faked. Better plagiarism detection measures could make this even more powerful.
  • Know your audience. The best part of Astronomy: Exploring Time and Space, to me, was in its cohesiveness and consistency. The entire course stays at the same level of presentation complexity and content depth. It knows its audience is novices to astronomy, and so it focuses on keeping things accessible, interesting, and enjoyable to watch. Not every class should focus on a beginner audience this way, but every class should know who its audience is and consistently cater specifically to that audience.

For more information, please see the course web site. If I’ve left out anything crucial to understanding this course, let me know below!

Share on Google+Share on FacebookTweet about this on TwitterShare on LinkedInShare on RedditPin on PinterestEmail this to someone

Course Review: Learning How to Learn

As I mentioned a few weeks ago, I recently took the Coursera class Learning How to Learn, from the University of California-San Diego‘s Dr. Barbara Oakley and Dr. Terrence Sejnowski. This review is written based on the course as it existed in March 2015.

This review serves three functions:

  • For prospective students, should I take this course?
  • For prospective employers and admissions offices, what does having completed this course say about a potential employee or student?
  • For course developers and educators, what can I learn from how this course is developed and delivered?

This review will contain two parts: an objective description of the course content, structure, and experience, and a subjective analysis of the course’s value to students, employers, and educators.

Course Description

Learning How to Learn is a self-paced class covering several skills for learning, memorizing, and avoiding procrastination. The full description is available here.

Structure

Learning How to Learn is entirely self-paced. Students in the class have up to six months to complete all the assessments. All assessments and course materials are always available, so students may proceed as fast or slow as they would like.

Learning How to Learn is broken up into four broad units, described below. Each unit contains approximately an hour of video with short exercises (typically short ungraded free response or single multiple choice questions) interspersed. At the conclusion of the first unit, there is a peer-reviewed short essay. At the conclusion of the class, there is a longer final exam and a peer-reviewed final project.

Overall, the course can be completed in around eight total hours, depending on how much effort one invests into the short essay and the final project.

Content

Learning How to Learn consists of four units: “What is Learning?”, “Chunking”, “Procrastination and Memory”, and “Renaissance Learning and Unlocking Your Potential”. Each unit primarily provides a number of tricks and metaphors for thinking about and improving learning. Among the topics touched on are the role and importance of sleep in learning; how and why to chunk material in one’s mind; the problems of procrastination and how to overcome them; the different modes of thinking, when to use them, and how to switch between them; and other small tricks for learning material better.

The course title is fitting: Learning How to Learn is about one’s personal learning. While it touches on some of the mechanisms of learning, its primary focus is on teaching the learner to learn, not necessarily why these skills work. It’s worth noting that, in my opinion, the course also focuses primarily on the types of learning we do in school rather than real-world learning. While real-world learning can be informed by these skills as well, the examples used throughout the course are drawn from traditional education, such as memorizing lists of facts, studying for an exam, and taking a test.

Assessments

There are 6 formal assessments in Learning How to Learn:

  • Three quizzes, one at the end of each of the first three units. The quizzes are made of multiple-choice questions and are auto-graded.
  • One final, at the end of the fourth unit. The final is made of multiple-choice questions and is auto-graded.
  • One short essay on a personal learning challenge, at the end of the first unit. The essays are peer-graded.
  • One final project to teach the concepts taught in Learning How to Learn at the end of the final unit. The final project is peer-graded.

Additionally, there are short, optional multiple-choice and free response questions throughout the course, as well as an optional peer-graded final video project.

The quizzes and exam can be taken an unlimited number of times until a satisfactory score is achieved. However, they can only be taken once every 24 hours, and the student cannot see which answers they got right and wrong.

Prerequisites

There are no necessary prerequisites for this class. It is applicable to any learner, from middle school to graduate school.

Identity Verification

For Verified Certificates ($50 at the time that I signed up), students complete the Coursera two-phase authentication: a typing sample is submitted and a picture is taken of the student immediately after completing the assessment. Assuming that these are adequate in ascertaining the student’s identity, they are nonetheless not difficult to circumvent. It would be trivial to collaborate with someone else on the assignments or even outright submit someone else’s work. This verification only assures that the student was present when the work was submitted.

If the student is not enrolled in a Verified Certificate, there is no identity verification at all.

Course Analysis

Generally, Learning How to Learn uses the “talking head” approach of a person on screen presenting material with visuals alongside. The production quality is good and the visuals are varied; they are not simply slides presented alongside the speaker. Interviews are also intermingled with the lectures to break up the style, and the occasional quizzes do a fair job of checking more than just attention.

The twist on the assessments, delaying retaking and hiding incorrect answers, is overall a useful shift. Hiding the incorrect answers does limit the feedback students receive, which is problematic, but this combined with the time delay forces the learner to actually review and attempt to correct their own error. While many quizzes in MOOCs like this can be “gamed” simply be retaking them and playing the odds on multiple choice questions, these two features introduce some additional rigor into the process.

For Prospective Students

Learning How to Learn is highly student-focused; its primary goal is to help the student personally improve their learning to excel in other learning activities. For that reason, its value to any prospective student is high. The value does not depend on one’s interests or current ability; the content is useful to everyone.

While the content of Learning How to Learn is useful, this isn’t a case of “One Special Trick to Pass All Your Exams!” The course is full of useful nuggets of information, but it isn’t a cure-all. The techniques outlined in the course require significant practice to master and benefit from, which itself is part of the point of the class. In my opinion, the most critical takeaway of the class is that learning itself is a skill you can learn: you are not endowed with a certain amount of intelligence that will never change, but rather you can learn to learn more.

Regarding the Verified Certificate, its only function is a motivation for you to complete the course. If you’re more likely to complete a course having invested money in it, by all means pay for the Verified Certificate (I did for exactly that reason). If not, just consume the material on your own.

Take this course if:

  • You don’t think you’re smart.
  • You have trouble learning new material.
  • You tend to procrastinate.
  • You’re still learning (and you should be!).

Don’t take this course if:

  • You have an intermediate background in psychology (you likely already know the material).
  • You need deadlines to succeed (as the class is entirely self-paced).

For Prospective Employers and Admissions Offices

With regard to identity verification, the identity verification for the Verified Certificate is trivial to circumvent. Thus, a Verified Certificate in Learning How to Learn should carry limited clout for that reason alone. The remaining analysis will infer that the student did complete the certificate themselves.

With regard to assessment, the rigor of the course is decently high. As referenced above, the assessments are structured such that students cannot easily “game” the quiz; in other words, they cannot simply take the quiz, see what they get wrong, and retake the quiz changing their answers to the questions that they got wrong. Passing the quizzes and exams actually represents some level of real achievement.

The essay and final exam are peer-graded. While I, personally, have some reservations about the validity and reliability of peer grading, research has shown this approach to be effective.

With regard to content, the content of the course is itself not particularly analogous to existing courses on education, but it is valuable in and of itself. One can believe that a student exiting the class has at their disposal a number of useful tricks for learning new material, and is thus at least somewhat prepared for the learning associated with a rigorous academic program or new position. Moreover, the act of completing the course can be said to demonstrate a willingness in the learner to improve their own learning ability (until and unless a Verified Certificate in Learning How to Learn begins to carry enough clout that students obtain the certificate solely for that clout).

Overall, until and unless a Verified Certificate in Learning How to Learn becomes sufficiently valuable that students pursue it specifically to put it on their resume, completing the course ought to be taken as a minor achievement and positive inclusion on an applicant’s resume. (See The MOOC Certificate Paradox.)

For Course Developers and Educators

There are three main things that this course does that I’d like to see more courses do:

  • Enforce a delay between quiz and test retakes. I love that most MOOCs let you attempt quizzes as many times as you want, but being able to try multiple times in a very short time span incentivizes narrow learning of the answers to quiz problems rather than broad learning of the material as a whole. This is compounded by the second part…
  • Structure quizzes such that retakes aren’t simply about correcting answers. I hesitate to say “limit feedback” because there are other ways to accomplish this goal. The general idea is: retaking a quiz should not be a matter of simply looking at the previous take and changing your wrong answers. Under that structure, it’s trivial to game the quiz into a good score. Instead, there needs to be something more. The questions themselves might change between retakes. The available answers might change. Or, as in the case of Learning How to Learn, the specific questions aren’t marked right and wrong.
  • If you’re going to take the “talking head” approach, display engaging or interesting visuals alongside instead of bullet point PowerPoint slides. Talking heads with PowerPoint slides in a MOOC are the easiest way to present material, but they’re also among the least compelling ways. Find ways to liven up the material with visualizations, visual metaphors, or other more visually compelling information.

For more information, please see the course web site. If I’ve left out anything crucial to understanding this course, let me know below!

Share on Google+Share on FacebookTweet about this on TwitterShare on LinkedInShare on RedditPin on PinterestEmail this to someone

Introducing MOOC Reviews

In keeping with my prior question on accreditation in MOOCs, I’m starting my own one-person MOOC accreditation service. Well, not really. But I am going to start publishing reviews of the online courses I finish to try to help develop accurate reputations for MOOCs and other online courses.

First of all, there are two types of information I hope to disseminate through these course reviews:

  • Descriptions: fact-based descriptions of the structure and content of the course or MOOC.
  • Analysis: opinion-based impressions on the value and quality of the course or MOOC.

There are three audiences for these course reviews:

  • Prospective students: What will you learn? How much time will it take? How deeply will you learn it? How strong are the assessments?
  • Employers and admissions administrators: How valuable is a certificate from this course? What content can I assume the student has mastered? How certain can I be that the student completed the MOOC themselves?
  • Educators and course developers: How did the course approach grading and assignments? How was the MOOC structured? How well did the structure work? What could be improved?

Toward these ends, each review will be structured according to several categories, each with a short description of the course’s approach:

  • Structure: How is the course structured? Is the course time-locked or open? Is the course on-demand or traditional? What are the due dates?
  • Content: What topics are covered in the course? What are the units and sub-units?
  • Identity: What steps are taken to ensure that the student receiving the credit for completing the course really is the student claiming credit for the course?
  • Assessments: What assessments are required to complete the course? To what extent do those assessments demonstrate real learning?
  • Prerequisites: What prior knowledge is required to succeed in the course? What general level of education is required to be prepared for the course?

Based on these five categories, as well as other elements, I’ll then give a brief analysis of what the course means to students, prospective employers, and other educators, touching as well on the overall course experience.

Note that although some of these analyses can get into the realm of good practices and bad practices, nothing here is ever meant to negatively reflect on the courses or MOOCs themselves. Every course has strengths, every course has room for improvement, and different courses are better-suited for different kinds of learners. Not every course is intended to give students a credential that will guarantee value to a prospective school or employer: many courses are meant to be surveys or introductions, not rigorous capstones. Thus, when I say that an employer shouldn’t put much stock in a particular course appearing on an applicant’s resume, that is not a criticism of the course. These reviews are meant to describe and inform, not criticize and judge.

You can find all my course reviews under the course reviews tag on this blog.

Share on Google+Share on FacebookTweet about this on TwitterShare on LinkedInShare on RedditPin on PinterestEmail this to someone