Fortnightly Mailing

Dylan Wiliam - Formativeness as a potential property of an assessment

I heard "inside the black box" Dylan Wiliam speak at an LSIS event today for the current cohort of Research Development Fellows. Wiliam was fascinating as always. For example here is his definition of formative, which he sees as a property of (some, and only some) assessments:

"An assessment functions formatively to the extent that evidence about student achievement elicited by the assessment is interpreted and used to make decisions about the next steps in instruction that are likely to be better, or better founded, than the decisions that would have been taken in the absence of that evidence."

Statistical literacy - ARTIST - a resources site

ARTIST - "Our goal is to help teachers assess statistical literacy, statistical reasoning, and statistical thinking in first courses of statistics. This Web site provides a variety of assessment resources for teaching first courses in Statistics."

Peer Instruction in the Humanities and Arts a resource from Monash University

The diagram below is from an area of the Monash University Philosophy Department's web pages devoted to peer instruction in the humanities, mostly dated 2008.

For more on Eric Mazur's peer instruction, go to Data is not the plural of anecdote, this two-minute overview, or the Mazur Group web site. Or read the book.

Source

Cognitive neuroscientific insights into training

A novel approach to minimize error in the medical domain: Cognitive neuroscientific insights into training [link to article home page] is a newly published paper by Dr Itiel Dror, whose work I've featured previously in Fortnightly Mailing.

Here is the abstract:

Medical errors are an inevitable outcome of the human cognitive system working within the environment and demands of practising medicine. Training can play a pivotal role in minimizing error, but the prevailing training is not as effective because it directly focuses on error reduction. Based on an understanding of cognitive architecture and how the brain processes information, a new approach is suggested: focusing training on error recovery. This entails specific training in error detection and error mitigation. Such training will not only enable better responses when errors occur, but it is also a more effective way to achieve error reduction. The suggested design for error recovery training is to begin with detecting errors in others. Starting off with highly visible and even exaggerated errors, and advancing to more challenging detections and finally requiring to detect errors within oneself rather than in others. The error mitigation training starts with providing the learners with the correct remedial actions (after they have detected the error). With training, the learners are required to select the appropriate actions within multiple choice alternatives, and eventually are required to generate the appropriate remedial responses themselves. These can be used for instruction as well as for assessment purposes. Time pressure, distractions, competitions and other elements are included so as to make the training more challenging and interactive.

Two-minute video overview of Eric Mazur's approach to large group teaching

Last year I pointed to a talk by Eric Mazur about peer-based instruction. The video above provides a clear and accessible overview of Mazur's approach.

Conditions for Successful Innovation – A Baker’s Dozen Checklist

This checklist was developed by participants in a workshop – Beware Geeks Bearing Gifts – at the LSIS Embracing Technology for Success conference held in Birmingham on 2 February 2011.

Comments are welcome.

[Innovation = change for the better that helps solve a problem that need solving and which is capable of being solved.]

Innovation within an organisation tends to be successful when ……

1. there is buy-in from users and when those who have to change their practices are doing so willingly rather than by diktat (willingness and diktat are extreme ends of the spectrum, and willingness can be developed through advocacy by people in positions of influence);
2. there are clear objectives – kept in mind at all times – with all the main people involved able to answer clearly the questions “what are we doing?” and “why are we doing it?”
3.  it is capable of being done at a large enough scale for financial savings or effectiveness improvements to accrue;
4. people are given time to do it – thus resource controllers need to be sufficiently convinced of the benefits to provide time for implementation (time might be planning time, implementation time, professional development time, and last by not least, testing time);
5. those leading and/or promoting innovation need to be savvy about true costs and benefits – otherwise changes will be made that are unsustainable;
6. there is a clear business case (but this does also run counter to the other truth that some valuable inventions can arise when there is no business need whatever);
7. risk, recognition, and reward are part of the local climate (if there is fear of occasional failure then people will tend not to take risks – but do not forget that the banks failed partly because they rewarded risk-taking……..);
8.  innovation is seen as a key part of an organisation’s strategy (but this is not a sufficient condition: plenty of organisations that do not innovate have “we innovate” writ large in their strategy);
9. it is monitored and evaluated (with appropriate evaluation criteria that are not applied inflexibly), and when things that do not work are killed off – but not too quickly – rather than being left to fester;
10. resource controllers are able to spot “fantasy business plans”, and have their eyes open for those who are too personally or politically invested in a particular innovation (but remember: many innovations happen precisely because someone is personally invested!)
11. there is a continuous process of prioritising and re-prioritising – this is especially important when an innovation concerns a complex and possibly changing mix of issues – and appraisal should not stop when a product or service is launched);
12. you keep the horse (the innovation) in front of cart (the problem that it is designed to solve) at all times, and ensure that the innovation has a strong focus on clients/users/consumers;
13. every project needs a communication channel, successes and failures should be honestly and routinely communicated (negative news sticks and travels further and faster than good news).

Miscellaneous points

a. There is no point expecting technically dependent innovation to be successful if the technical infrastructure is not reliable.

b. One problem solved may create others. (This links to 11 above).

c. The checklist relates mainly to “incremental or diffusive innovation” rather than to “disruptive innovation”. It should not be assumed that the former is a more important form of innovation than the latter.

 

Snippets from the last seven weeks

When I started Fortnightly Mailing it was a genuine fortnightly newsletter, and hand coded at that. (Here is Number 8 from 8 years ago.)  Four years later I started publishing Fortnightly Mailing using Typepad, with subscribers receiving a regular email of links to individual posts.

Gradually the availability of other tools and the (nasty) "snippetisation" of discourse on the Internet has got in the way of this process. Things I might in the past have written about properly, I now comment on much more briefly using FriendFeed: the barriers to writing "properly" seem to have grown, and those like Stephen Downes, Clive Shepherd, Cath Ellis, David Jennings and Donald Clark who are managing to "keep up proper writing" seem particularly admirable.  However, there remains something to be said for snippets, and I hope that these from the last seven weeks will be of genuine interest.

1: "The Internet Problem: when an abundance of choice becomes an issue." A to-the-point piece by Cory Doctorow in the Guardian.
http://ff.im/voy82

2: Breathtaking, almost. The US Government's "Open Government Initiative" provides a process template that a lot of others could use.
http://ff.im/voz7w

3: David Campbell's Photography Multimedia Politics is a blog that "analyzes how documentary photography and photojournalism works".
http://ff.im/vcRot

4: Tuition fees, markets, and inequality. Salford's VC Martin Hall analyses the new system approved by Parliament, which, he says, should properly be described as a “minimum threshold, time limited graduate contribution system”.
http://ff.im/v8VSR

5: Many Eyes - a promising looking IBM data visualisation tool/environment.
http://ff.im/v3JHJ

6: "Doc note: I dissent. A fast never prevents a fatness. I diet on cod." Obituary of mathematician Peter Hilton, Bletchley Park code-breaker, who died on 6 November.
http://ff.im/uYANo

7: Travelling into the wind, powered by the wind, at three times the speed of the wind.
http://ff.im/uHnht

8: Submission fees in scholarly (and potentially Open Access) publishing - interesting piece by Mark Ware for the JISC-supported Knowledge Exchange.
http://ff.im/uGgpW

9: The Chinese Top Level Quality Project "a large project in Chinese higher education which uses the production of Open Educational Resources to improve the quality of undergraduate education", by Stian Håklev.
http://ff.im/uDDvx

10: Ars Technica - Adam Stevenon - The economic case for open access in academic publishing.
http://ff.im/uv0r7

11: Making "data-based instructional decisions". Rupert Murdoch's News Corp moves into education, buying 90% of Wireless Generation.
http://ff.im/urX8v

12: "In the past, the KGB resorted to torture to learn of connections between activists. Today, they simply need to get on Facebook." James Harkin's "Cyber-Con" in the London Review of Books.
http://ff.im/uq8jW

13: Why Ed Miliband is not wrong to talk about the "squeezed middle". Nick Cohen in the Spectator.
http://ff.im/upWO7

14: Interesting interview by David Jennings with Ollie Nørsterud Gardener about recognising individuality in enterprise learning.
http://ff.im/ukmCF

15: Caroline Sutton's "presidential" keynote at the Open Access Scholarly Publishers' Association in 2010.
http://ff.im/ujSzv

16: US dollar-millionaires call on the Barack Obama to let tax-cuts lapse. [He did not....]
http://ff.im/ugDkl

17: Review of DirectGov for the Cabinet Office by Martha Lane Fox.
http://ff.im/udfHG

18: "Get ahead" - a promising tie-up between the University of Salford, The Manchester College, Trafford College, and Salford City College, that helps FE students find out more about studying in university from "real" students.
http://ff.im/u9Qka

19: Senior managers in education: "That felt like being in a room with a bunch of record company executives in 1999." Two long and carefully made write-ups of interviews by David Jennings.
http://ff.im/tYQLv

20: UK Government ignores 'net neutrality'. Report in The Register by Chris Williams.
http://ff.im/tWZMG

21: "The production and management of evidence for public service reform" by surgeon Jonathan Shepherd argues strongly for stronger and more systematic connections between the research system and the public services.
http://ff.im/tPo8q

22: "In the Facebook internet, everyone knows exactly what breed of dog you are." Piece from Wired that highlights difference between Facebook's and Google's use of personal information.
http://ff.im/tIVUE

23: How will cultural data be preserved? And by whom? In 2005 I heard this talk about the data deluge by Cliff Lynch, and it still sounds fresh.
http://ff.im/tH41S

24: Kabir Chibber's "Online education disrupting traditional academic models" describes the wholly online Applied Engineering and Technology Library at the University of Texas.
http://ff.im/tC1Ri

25: The video of Sudhir Giri of Google's "Developing a Learning Ecosystem" from ALT-C 2010 is finally published on the ALT YouTube Channel.
http://ff.im/tw4Qx

26: Most users of Creative Commons use the "wrong licence". A paper by Derek Keats which summarises some principle-based guidelines for choosing between Creative Commons licenses.
http://ff.im/tnFKE

27: Self-Selected or Mandated, Open Access Increases Citation Impact for Higher Quality Research", by Yassine Gargouri, Chawki Hajjem, Vincent Larivière, Yves Gingras, Les Carr, Tim Brody, Stevan Harnad.
http://ff.im/tbffi

28: Browne's Gamble - Stefan Collini's trenchant and sensibly selective critique of the Browne Report in the LRB.
http://ff.im/sWMun

Anticipating the Web - information available "in amplitude in proportion to its degree of relevance"

Source

During a discussion about Open Access journal publishing Brian Whalley posed this "christmas quiz" question:

Who wrote the following and when?

"The kind of organization we wish to aim at is one where all relevant information should be available to each research worker and in amplitude in proportion to its degree of relevance. Further, that not only should the information be available, but that it should be to a large extent put at the disposal of the research worker without his having to take any special steps to get hold of it."

Google and Sheila Webber's interesting¹ 2003 Journal of Information Science article Information Science in 2003: A Critique led me to discover that this anticipation of the Web was by the UK-based Irish physicist JD Bernal, pictured above. In 1939.

¹ Webber's categorisation of disciplines, building on Becher, as "Hard pure", " Soft pure", "Hard applied" and " Soft applied" caught my eye.

Judging experimental evidence: a couple of informative essays

Proponents of ideas sometimes find, stake their careers on, and then cling to bogus evidence in support of their ideas, ignoring other, stronger, negating evidence. (think free schools, learning styles, neuroplasticity, and The Shallows.....).

Two essays by Peter Norvig: Evaluating Extraordinary Claims: Mind Over Matter? Or Mind Over Mind? and Warning Signs in Experimental Design and Interpretation are dryly written, entertaining, but at the same time very informative essays about research methods. The first essay illustrates the problem using an extended case study. The second provides a clear framework for judging experimental evidence more generally.

Clayton Wright's worldwide listing of educational technology conferences

Clayton Wright continues his very valuable contribution to the field of technology in learning with another comprehensive listing: "Educational Technology and Related Education Conferences for January to June 2011".