Fortnightly Mailing

Lessons learned from using Khan Academy content in a blended learning pilot

Final sentence added 3/1/2012.

In this December 2011 report [23 page PDF] Brian Greenberg, Leonard Medlock and Darri Stephens report on the "performance and engagement of low‐performing high school algebra students receiving a mix of traditional teacher-­led instruction and self‐guided instruction through the Khan Academy website".

The authors seek to compare the performance of two groups of "summer school" learners, one group taught traditionally and one group teacher-supported in largely individual use of Khan Academy content (thus "blended learning"). Both groups made substantial progress (as judged by the increase in percentage questions answered correctly on the MDTP Algebra II Readiness Exam).

The Khan Academy group is reported - with plenty of caveats, and with the rather disconcerting bar chart above - to have made very slightly more progress.

Along the way, the authors make interesting and useful observations about:

• classroom management under a "blended learning" model;
• the use of Google Chromebooks (cheapish quick-to-boot laptops that require a Wi-Fi connection to function fully) in a classroom context;
• how teachers in a blended learning environment can use data about learner performance to inform how, when, and with which learners to intervene;
• how having a plethora of systems in use at the same time or in the same school would prove problematic from a data-management point of view:

"One obvious challenge is that currently each online course or software uses its own data reporting system. For isolated pilots such as ours, these kinds of “walled gardens” can work fine. The teacher simply needs to learn how Khan Academy reports data. However, as teachers start using multiple products in a class or as schools blend technology into multiple courses, it becomes increasingly hard to make sense of all the data."

A striking paragraph in the report - which I can personally relate to having recently finished a course based on very short (40 second to 300 second) videos - concerns the value of Khan Academy videos:

"A final interesting perspective on Khan involves the value of the site’s videos. Most people are drawn to Khan based on its massive video library and Sal’s own charming and engaging teaching style. Like many, we assumed the videos would be the predominant learning mechanism for students tackling new material. In fact, the students rarely watched the videos. This result is consistent with some of the observations in the Los Altos pilot. The students greatly preferred working through the problem sets to watching the videos. Students turned to their peers, the hint, and the classroom teacher much more often than they did the linked Khan video. One possible reason is that the videos are aligned to the broader concept, but do not link directly to the problem students are struggling with. A second hypothesis is that the videos may be too long at eight to ten minutes. If students have 60‐90 minutes to work through multiple concepts in a class period, an investment of ten minutes for a single video feels like a lot. The badges and stars within Khan may also be a disincentive, as there is no immediate reward for watching videos as there is when completing streaks. Lastly, we wonder how many of us really enjoy watching instructional videos for extended periods of time. We are left curious about whether Khan’s videos need to be even more modular and shorter in duration and also about the value of video based instruction."

But overall, as Al Essa and Stephen Downes have indicated a somewhat frustrating and inconclusive study.

Unexpected - how an open course's conventional exam can prompt subsequent learning

Graph by automagic using code by pedrosorio.
Other posts tagged ai-course.

The AI course finished this week. Here is a link to a discussion thread involving some very mathematically able people discussing, after the exam results were in, a "disputed" mark concerning an exam question about a statistical technique called Laplace smoothing.

I'm not making a point about the substance of the discussion (a lot of which is over my head), or about the underlying investigation (which is even more so).

But the interesting thing is the way that this open course, with its underlying conventional, assessed structure, has resulted in a great deal of very high level collaborative research, investigation, and discussion, which has gone far beyond its (first year US undergraduate) boundaries.

Even if the subject matter of the post is outside your ken, it is worth taking in the kinds of reflections that contributors are making, for example:

"This is really the sort of thing and the sort of thinking and exploring that I believe professors really want to see result from efforts to teach stuff in class (whether in K-12 or undergraduate or graduate school). This one little question has generated more serious inquiry than anything else I have seen so far from our class...... But THANK YOU for all of the serious and thoughtful responses to this question which it has been a pleasure to read."

 

Sitting in a bar with a really smart friend. The ai-class: Notes from a Lab Rat

Guest Contribution by Rob Rambusch

Picture by Ressaure (CC licensed)

Me: Test Subject

Rob Rambusch is a Project Manager for software development and implementation based in New York City. This was his first exposure to any class on Artificial Intelligence. He can be contacted at robrambusch [AT] gmail.com, or through Google+.

It: Experiment

"A bold experiment in distributed education, "Introduction to Artificial Intelligence" will be offered free and online to students worldwide from October 10th to December 18th 2011. The course will include feedback on progress and a statement of accomplishment. Taught by Sebastian Thrun and Peter Norvig, the curriculum draws from that used in Stanford's introductory Artificial Intelligence course. The instructors will offer similar materials, assignments, and exams."

Them: Experimenters

"Peter Norvig is Director of Research at Google Inc.  He is also a Fellow of the American Association for Artificial Intelligence and the Association for Computing Machinery. Norvig is co-author of the popular textbook Artificial Intelligence: A Modern Approach. Prior to joining Google he was the head of the Computation Sciences Division at NASA Ames Research Center."

"Sebastian Thrun is a Research Professor of Computer Science at Stanford University, a Google Fellow, a member of the National Academy of Engineering and the German Academy of Sciences. Thrun is best known for his research in robotics and machine learning."

 One Month Before: Preparation

I started working on the prerequisites in the month before the course began. I prepared for the class by learning Probability and Linear Algebra from Salman Khan at the Khan Academy website. I followed up by watching lectures on Linear Algebra by Gilbert Strang at the MIT OCW website. So even before the class began I had exposure to two common online teaching styles, the filmed lecture and the video tutorial.

Current "line" on ICT and Computer Science in the National Curriculum Review

Here is a link to The Framework for the National Curriculum - A report by the Expert Panel for the National Curriculum review by Mary James, Tim Oates, Andrew Pollard, and Dylan Wiliam. Of particular interest to some readers of Fortnightly Mailing is this intriguing recommendation relating to ICT and Computer Science:

We recommend that: Information and communication technology is reclassified as part of the Basic Curriculum and requirements should be established so that it permeates all National Curriculum subjects. We have also noted the arguments, made by some respondents to the Call for Evidence, that there should be more widespread teaching of computer science in secondary schools. We recommend that this proposition is properly considered.

Two recent Association for Learning Technology documents relate to this:

• An ALT contribution to the current discussion about the place of Computer Science in the National Curriculum for England - December 2011;
• An ALT response to the review of the English National Curriculum - April 2011.

[Disclosure: I work half time for ALT.]

Final report from the Norvig/Thrun/Stanford/Know Labs Artificial Intelligence course

Word-cloud by Sea Otter

(Other posts tagged ai-course. Post originally published on 20/12/2011, with paragraph one updated to incorporate completion numbers kindly provided by David Stavens of Know Labs, and a new concluding sentence to the final paragraph.)

Along with just over 20,000 others (some 3,000 fewer than had taken the midterm exam) I completed the final examination for the free online Introduction to Artificial Intelligence course taught by Sebastian Thrun and Peter Norvig. Here is my final participant's  report from the course.

1. The final section of the course concerned Natural Language Processing. I've had an interest in machine translation for some years [e.g.]: and it was this interest that initially made me aware of Peter Norvig's work.) So for me this meant that the best part of the course came last, and if you want to gain an underlying appreciation of the science of natural language processing, it will take you a couple of hours to work through the courses 42 short videos about NLP, starting here. It is probably worth doing despite a certain amount of dependency on earlier sections of the AI course.

2. The course has been mercifully free from programming assignments: being capable of completion using pen and paper, a calculator (and on a couple of occasions a slide rule unused since 1973).  To conclude the NLP unit there were two optional programming problems, both of which could be tackled without programming. I did the second of the problems (recovering a message from a shredded version) using scissors and adhesive tape:

Moe and Chubb - Liberating Learning

These two short videos from the Hoover Institution will give you a sense of the line of argument advanced by Terry Moe and John Chubb, who are right-of-centre, union-hostile US academics with trenchantly positive views about the value of on-line learning, and about the need to make the same kinds of technology/labour substitution in education as has already taken place in large swathes of the rest of the economy.

Terry Moe

John Chubb

For more on/from both, go to Liberating Learning - Technology, Politics, and American Education.

Apples, screwdrivers and desks: a comparative review of three Stanford free online computer science courses

Guest Contribution by Gundega Dekena

[Update posted by Seb Schmoller on 12 July 2012. Note that Gundega now works for Udacity, the company that developed from the AI course. Read how she became part of the Udacity team on the Udacity blog.]

Gundega Dekena is a self taught Linux administrator and web programmer, based in Riga, Latvia. She has been studying all three of the October to December Stanford online computer science courses in parallel - Introduction to Artificial Intelligence (AI), Introduction to Machine Learning (ML), and Introduction to Databases (DB) - putting her in a good position to compare and contrast them. Gundega can contacted at gundega.dekena [AT] gmail.com, or through Google+.

Overview

Comparing these three courses feels a bit like comparing apples, screwdrivers and desks, yet I see a lot of students doing that, usually without much thought about the differences. So, this is my look (from the perspective of a student) atthe things that can be compared, and that can be learned from all three courses, for the benefit of the next batch of courses that is going to come from Stanford next year.

Tenth report from the Norvig/Thrun/Stanford/Know Labs Artificial Intelligence course

(Other posts tagged ai-course.)

No report this week. Instead, here is a Guest Contribution by Gundega Dekena, which provides a thorough comparison between Stanford's three current free mass online computer science courses.

Food for thought. Video recording of Q&A session between Sal Khan, Peter Norvig and Sebastian Thrun

There is plenty of food for thought and a great deal to identify with - from a very diverse set of angles - in yesterday's 45 minute "Google+ Hangout" discussion between Sal Khan, Peter Norvig and Sebastian Thrun. Examples include:

• the authenticity and hence superiority of unscripted, tentative explanations, with "low" production values;
• how lectures empower lecturers not students;
• data analysis as an "underpinning process" for real personalisation;
• why their is such a mismatch between the value that a university thinks its courses provide what students think they provide;
• the challenge of combining individual study with group interaction.

 

Ninth report from the Norvig/Thrun/Stanford/Know Labs Artificial Intelligence course

Screen shot of Filip Wasilewski's Chrome plug-in explained in paragraph 7 below.

(Other posts tagged ai-course.)

Here is my ninth participant's  report from the Stanford Introduction to Artificial Intelligence course.

1. I wrote last week that the units on Games, Game Theory, and Advanced Planning had been hard work; and that the associated homework had felt demanding and been very time-consuming. Securing an adequate mark on that homework despite too many  "stupid errors" means I probably learned more than I had realised. But I also know that I was answering some of the questions from the "stored fat" of knowledge I already had, rather than from what the course had been teaching me.