Fortnightly Mailing

The trouble with university: exclusivity, grades, and lectures

Prompted by the launch of Udacity [see also Sebastian Thrun's explanation], Matt Welsh, who used to teach at Harvard and now works for Google, writes what he describes as a "little rant" - 1500 words - that ponders "the failings of the conventional higher education model".  He focuses three main failures of HE - Exclusivity; Grades; and Lectures. But he also includes some caveats, concluding "I'm not sure there can ever be an online replacement for The College Experience writ large". Welsh's piece is the stronger for the large number of often thoughtful comments that follow it.

Taking the red pill: Sebastian Thrun's candid reflection on the AI course

After a horribly embarrassing introduction, this 25 minute talk yesterday by Sebastian Thrun gives Thrun's own candid and personal reflection on last Autumn's AI course, which had 160,000 sign-ups (nearly 100,000 of whom were on the advanced track), 46,000 submitters of the first homework, 23,000 submitters of the mid-term exam, and 20,000 who completed the final exam.

Highlights of the talk:

• the large drop-out rate from the lectures on the same course at Stanford, with students preferring to use the free video-based version;
• a volunteer army of ~2000 translators;
• individual feedback from students in terrible places in the world or under big social pressure who completed the course;
• Thrun's own epiphany concerning the wrongness of "weeder" classes;
• Thrun's decision not to teach by lecture at Stanford again and instead to concentrate his efforts on a private venture-capital funded initiative called Udacity, whose online courses will be free.

Udacity aims to enrol 500,000 students on its first two courses: CS101- Building a search engine; CS373 - Programming a robotic car.

My own and others' reports from the AI course.

A reaction to Apple's "reinvention of the text book"

Jeevan Vasagar, the Guardian's Education Editor, asked if I'd be willing to comment on today's event and announcements in New York by Apple about education.

This prompted me to follow the event using the Verge's live pictorial blog, from which you yourself will get the gist of what Apple is doing. There is more on ZDNET; and there is this piece in the Guardian.  [At the bottom of this post I am adding links to pieces I come across that seem to shed useful further light.]

My reaction?

It is difficult to see how the impressive-looking tools, content and services announced today by Apple will not be the kind of "game changer" that we've got into the habit of expecting from Apple.

Students, particularly in the US, pay a lot for prescribed textbooks even if they manage to buy them second hand. Provided they own an iPad and provided the right text-books are available (the second proviso is even more serious than the first) then Apple's text book service will provide a more interactive and probably pedagogically more effective experience than will the use of conventional text books; and at a much lower price.

Alongside this, and only from the look of it, the text book writing software that Apple will be giving away will be of intense interest to teachers and content developers in the Apple-using world. It remains to be seen whether the teacher-creators will be any match, in the quality and/or slickness of what they make, for the big publishers with whom Apple is already working, or for the smaller content developers and in-house production teams in universities who get started with the software now.

But there are lots of issues. Here are three.

1. Learning requires work (rehearsal, re-expression, discussion, making things, analysis, etc) on the part of learners, and "activity design" to ensure that the work happens. Learning is helped by formative feedback - which is notoriously difficult so far for machines to provide. Will the content that flows via Apple from publishers and authors onto iPads now or in the future make/help learners do the work of learning? Or will what we see be mainly "jazzier text books" - in effect interactive TV documentaries?
2. How if at all will creators of Open Educational Resources be able to work within the framework provided by Apple? (I want someone to comment below "it will be easy, and here is how".)
3. What is the business and licensing model for Apple and for the content providers? Will it tend to encourage a "monoculture" of online text books where what dominates is what is available rather than what is good? Will it reduce the diversity of printed text books?

Over the next 18 months we are going to find out.

Note 1. I am grateful for feedback from several members of the Association for Learing Technology - for which I work part time - helped me work out what to say to the Guardian - and influenced this piece in other ways.

Note 2 - added after original publication. Links to other interesting posts I come across on this topic are below.

• Audrey Waters in Mindshift - 19/1/2012
• Leon Cych commenting on the Guardian piece "education is what happens between your ears and not on a screen" - 19/1/2012
• Peter Kafka on why it makes sense for textbook publishers to have their textbooks sold for $14.99 or less - 19/1/2012
• James Clay - "'Reinventing Textbooks', I don't think so" - 20/1/2012
• Niall Sclater - First impressions of iBooks 2 - 20/1/2012
• Downes - wide ranging overview of reaction - 20/1/2012
• Ed Bott - a scathing attack on the highly restrictive iBooks end-user license agreement - 20/1/2012
• Jason Perlow - an oversimplifying ZDNet piece asserting that the costs of providing childred with iPads are too huge to contemplate - 30/1/2012

Pseudonymous comments

I've been having an exchange with a commenter  on this blog about psuedonymous comments. My basic position is that substantial comments are much more useful if the identity of the commenter is clear. I do not reject psuedonymous comments out of hand, but would encourage commenters who feel they must obscure their identity from readers of the blog (though not from me) to include in their comment something about themselves - role, area of work, expertise etc - which would help readers situate the comment. If I think the value of a comment is high even without that additional information then I will usually publish the comment. As in this case. I would welcome readers' views on this.

David Hu's use of AI - including Bayes networks - in the Khan Academy's assessment systems

Some weeks ago I posted a link to an interesting post by David Hu about how the Khan Academy is beginning to use machine learning and AI in its assessment systems. David Hu was an intern with the Khan Academy, and here is a video he made reflecting on the work he did during his internship. Do not be put off by the first few seconds of the video, or the later slightly breathless sections. There is real science, clearly expressed, about the "journey towards better assessment practices".

You can also read Hu's fuller post-mortem on his internship.

Interviews about "remote learning" with Dan Cardinali, Paul Mitiguy, and Peter Norvig

Here are links to three November 2011 "canteen" interviews (audio and, in two cases helpful text transcripts) with Peter Norvig (Director of Research at Google), Dan Cardinali (President of Communities in Schools) and Paul Mitiguy (Senior Consulting Professor of Mechanical Engineering at Stanford University). The interviews form part of the resources of an undergraduate project by Deniz Kahramaner, Jon Rodriguez and Ben Kallman about "the challenges and ethical implications of implementing large-scale distance education platforms". 

The interview with Peter Norvig draws on his contemporaneous experience with the Stanford/KnowLabs Introduction to Artificial Intelligence course [November 2011 article in ALT News Online; my weekly reports from the course] and contains several perceptive and diverse insights, ranging over:

• course design (and the advantages of combining the deadline driven "discipline" of face-to-face learning with the "anytime anyplace" nature of asynchronous learning);
• the advantages (from a learner's point of view) of video, and challenges (from a course design point of view) of using it;
• the need for design/production environments for teachers and for experimentation/exploration environments for learners;
• the value, from a learning point of view, of ambiguity in presentations and open-endedness in questions;
• recruitment (and a possible business model for providers of free remote education);
• the possible motivations for teachers to contribute to the production of free courses;
• the need for collegiality and collaboration between the start-ups that are getting involved in online education.

Lessons learned from using Khan Academy content in a blended learning pilot

Final sentence added 3/1/2012.

In this December 2011 report [23 page PDF] Brian Greenberg, Leonard Medlock and Darri Stephens report on the "performance and engagement of low‐performing high school algebra students receiving a mix of traditional teacher-­led instruction and self‐guided instruction through the Khan Academy website".

The authors seek to compare the performance of two groups of "summer school" learners, one group taught traditionally and one group teacher-supported in largely individual use of Khan Academy content (thus "blended learning"). Both groups made substantial progress (as judged by the increase in percentage questions answered correctly on the MDTP Algebra II Readiness Exam).

The Khan Academy group is reported - with plenty of caveats, and with the rather disconcerting bar chart above - to have made very slightly more progress.

Along the way, the authors make interesting and useful observations about:

• classroom management under a "blended learning" model;
• the use of Google Chromebooks (cheapish quick-to-boot laptops that require a Wi-Fi connection to function fully) in a classroom context;
• how teachers in a blended learning environment can use data about learner performance to inform how, when, and with which learners to intervene;
• how having a plethora of systems in use at the same time or in the same school would prove problematic from a data-management point of view:

"One obvious challenge is that currently each online course or software uses its own data reporting system. For isolated pilots such as ours, these kinds of “walled gardens” can work fine. The teacher simply needs to learn how Khan Academy reports data. However, as teachers start using multiple products in a class or as schools blend technology into multiple courses, it becomes increasingly hard to make sense of all the data."

A striking paragraph in the report - which I can personally relate to having recently finished a course based on very short (40 second to 300 second) videos - concerns the value of Khan Academy videos:

"A final interesting perspective on Khan involves the value of the site’s videos. Most people are drawn to Khan based on its massive video library and Sal’s own charming and engaging teaching style. Like many, we assumed the videos would be the predominant learning mechanism for students tackling new material. In fact, the students rarely watched the videos. This result is consistent with some of the observations in the Los Altos pilot. The students greatly preferred working through the problem sets to watching the videos. Students turned to their peers, the hint, and the classroom teacher much more often than they did the linked Khan video. One possible reason is that the videos are aligned to the broader concept, but do not link directly to the problem students are struggling with. A second hypothesis is that the videos may be too long at eight to ten minutes. If students have 60‐90 minutes to work through multiple concepts in a class period, an investment of ten minutes for a single video feels like a lot. The badges and stars within Khan may also be a disincentive, as there is no immediate reward for watching videos as there is when completing streaks. Lastly, we wonder how many of us really enjoy watching instructional videos for extended periods of time. We are left curious about whether Khan’s videos need to be even more modular and shorter in duration and also about the value of video based instruction."

But overall, as Al Essa and Stephen Downes have indicated a somewhat frustrating and inconclusive study.

Unexpected - how an open course's conventional exam can prompt subsequent learning

Graph by automagic using code by pedrosorio.
Other posts tagged ai-course.

The AI course finished this week. Here is a link to a discussion thread involving some very mathematically able people discussing, after the exam results were in, a "disputed" mark concerning an exam question about a statistical technique called Laplace smoothing.

I'm not making a point about the substance of the discussion (a lot of which is over my head), or about the underlying investigation (which is even more so).

But the interesting thing is the way that this open course, with its underlying conventional, assessed structure, has resulted in a great deal of very high level collaborative research, investigation, and discussion, which has gone far beyond its (first year US undergraduate) boundaries.

Even if the subject matter of the post is outside your ken, it is worth taking in the kinds of reflections that contributors are making, for example:

"This is really the sort of thing and the sort of thinking and exploring that I believe professors really want to see result from efforts to teach stuff in class (whether in K-12 or undergraduate or graduate school). This one little question has generated more serious inquiry than anything else I have seen so far from our class...... But THANK YOU for all of the serious and thoughtful responses to this question which it has been a pleasure to read."

 

Sitting in a bar with a really smart friend. The ai-class: Notes from a Lab Rat

Guest Contribution by Rob Rambusch

Picture by Ressaure (CC licensed)

Me: Test Subject

Rob Rambusch is a Project Manager for software development and implementation based in New York City. This was his first exposure to any class on Artificial Intelligence. He can be contacted at robrambusch [AT] gmail.com, or through Google+.

It: Experiment

"A bold experiment in distributed education, "Introduction to Artificial Intelligence" will be offered free and online to students worldwide from October 10th to December 18th 2011. The course will include feedback on progress and a statement of accomplishment. Taught by Sebastian Thrun and Peter Norvig, the curriculum draws from that used in Stanford's introductory Artificial Intelligence course. The instructors will offer similar materials, assignments, and exams."

Them: Experimenters

"Peter Norvig is Director of Research at Google Inc.  He is also a Fellow of the American Association for Artificial Intelligence and the Association for Computing Machinery. Norvig is co-author of the popular textbook Artificial Intelligence: A Modern Approach. Prior to joining Google he was the head of the Computation Sciences Division at NASA Ames Research Center."

"Sebastian Thrun is a Research Professor of Computer Science at Stanford University, a Google Fellow, a member of the National Academy of Engineering and the German Academy of Sciences. Thrun is best known for his research in robotics and machine learning."

 One Month Before: Preparation

I started working on the prerequisites in the month before the course began. I prepared for the class by learning Probability and Linear Algebra from Salman Khan at the Khan Academy website. I followed up by watching lectures on Linear Algebra by Gilbert Strang at the MIT OCW website. So even before the class began I had exposure to two common online teaching styles, the filmed lecture and the video tutorial.

Current "line" on ICT and Computer Science in the National Curriculum Review

Here is a link to The Framework for the National Curriculum - A report by the Expert Panel for the National Curriculum review by Mary James, Tim Oates, Andrew Pollard, and Dylan Wiliam. Of particular interest to some readers of Fortnightly Mailing is this intriguing recommendation relating to ICT and Computer Science:

We recommend that: Information and communication technology is reclassified as part of the Basic Curriculum and requirements should be established so that it permeates all National Curriculum subjects. We have also noted the arguments, made by some respondents to the Call for Evidence, that there should be more widespread teaching of computer science in secondary schools. We recommend that this proposition is properly considered.

Two recent Association for Learning Technology documents relate to this:

• An ALT contribution to the current discussion about the place of Computer Science in the National Curriculum for England - December 2011;
• An ALT response to the review of the English National Curriculum - April 2011.

[Disclosure: I work half time for ALT.]