BBC Jam. We need usage data.

Post updated (see italicised and struck-out text in paragraph 2), 21/3/2007

I've updated a piece I wrote on 14/3/2007 about the BBC's decision to suspend/scrap its £150m online learning service for school pupils. I started off pretty sceptical about the value of the service, and so far I'm not convinced by the assertions being made that more recently released content was of a much higher standard than the earliest Jam material, and that some amazingly good content was about to be released by Jam, which would win over the doubters.

Of particular interest is the actual use to which the content that has been released to date (a subset of what has been produced) has been put. To access the main features of Jam see Jam at all you needed to register, so not much can be read into the fact that 170,000 user accounts on Jam had been created.

What matters is data like:

• the number of users by stated age and role;
• their distribution across the country, and by language (BBC Jam had content in Gaelic and Welsh, as well as in English);
• the number of sessions that different classes of account-holder have had, over what period, and whether, for a particular content component this is rising or falling;
• the pattern and volume of use made of different parts of the materials by different classes of account-holder.

And, separately from this kind of data, which you would hope could be provided quickly and easily by the BBC from its logs (and is probably already known by people inside the Jam machine - who could post a comment here, or send me data privately), we need a feel for the extent to which teachers have been building use of Jam into course provision, if indeed that has been happening.

If learners in large numbers are dependent on a service that is being suddenly switched off, that would put the BBC Trust's decision in different light from if they are not. But how different? The average spend, for a service that has been available for over a year, has been around £500 per registered user. I'd be surprised if even 25% of the registered users are "real users", rather than users who were just taking a look. Thus an average spend to date on Jam of £2000 per real user, and possibly a great deal more, may be a more realistic figure: on a par with, say, what the state spends to teach a school pupil for one year.  Of course the intention was that the content would be in use for more than a year, and that the number of real users would grow, so the average annual spend per user would have fallen. But on the face of it I think that when and if the usage date is made available it will serve to undermine the case for Jam, rather than to support it.

Note. Other posts about BBC Jam:

• 19 March 2007 - BBC Director General Mark Thompson on Jam, in October 2006.
• 17 March 2007 - BBC Jam. We need usage data.
• 14 March 2007 - BBC suspends Jam, its flagship online learning web site.

 

Comments

[Sam]

I expect that data is available Seb, I'd be interested to see it myself. I don't think it would be particularly useful in the context of assessing Jam as a service though, mainly because it isn't a complete service yet, and of course now never will be.

I used to work at Jam. I left there for another part of the corporation about 6 months ago. I worked on the Maths 5-7 and Financial Capability 14-16 commissions, and one other on Statistics that (like so many others) has been on the shelf for 6 months and now will not go live.

For every subject you see live now, there are two more sat on the shelf that have been held up by the EC review. Jam had a soft launch last January, and a slightly more pushed launch in October. It's not been hardly sold to learners yet, mainly because it's not all there. The 170k users subscribed are the early adopters, out of a market of millions more that might have registered had the subjects that interested them been allowed to be published.

Finally on this, your implication that the extent to which teachers are using the material in classrooms will be a good measure of success, is misguided. BBC Jam was a learning resource not a teaching resource - all the material has been designed for the independent learner, wherever they maybe. It's success was not predicated on teachers using it in formal teaching, nor were they expected to be the gatekeepers to the content. Of course, teachers and consultants were involved throughout the design process (as well as kids and parents) to ensure the propriety of the content for learning, but its use in classrooms was always considered secondary.

[Thanks for this comment, Sam. I take your point about use by teachers being secondary, though I do believe that if teachers have been integrating Jam material into schemes of work, encouraging learners to use Jam etc, this is worth knowing. I also noted, and others will be interested to read, your informative response to Donald Clark's largely negative review of the existing materials. Seb - 18/3/2007]