Prof Rose Luckin on why measuring the impact of edtech matters (and how to do it right)

Prof Rose Luckin on why measuring the impact of edtech matters (and how to do it right)

Professor Rose Luckin of the UCL Institute of Education has been working on learning science, education technology and artificial intelligence in education for over 20 years. She was named one of the most important 20 figures in British education’ in The Sunday Times’ Seldon List 2017 and - perhaps most prestigiously - she will be speaking at FutureBook 2017, on a panel in the dedicated edtech stream entitled 'Efficacy: how can publishers truly demonstrate the impact of digital learning materials?'    

It's a question Luckin is perfectly positioned to interrogate. The UCL is one of the partners in the EDUCATE project launched earlier this year - a £4.5m collaboration with founder platform F6S, The Education Foundation and Nesta, that aims to build a ‘golden triangle’ between teachers, learners and edtech companies to ensure the UK produces products and services that have a genuine impact on education.

I caught up with Luckin to discuss the state of edtech efficacy now.

Tell us more about the EDUCATE project running at UCL

The whole mission is to help connect people who develop technology to support learning and teaching with the people who use them, and the people who research into how those things can be used effectively. Say, for example, we had a start-up who wanted to develop a toy for three-to-four-year-olds, to develop language skills. We wouldn’t just look at existing evidence about toys and language; we would also look at the kind of work coming out in developmental psychology - about how children learn languages, what the differences are across ages, cultural differences - and we'd put the startup in touch with somebody within UCL who has that kind of early years learning expertise. It’s trying to raise the level and quality of the conversation about what it means to say: ‘This educational way of working, or educational technology, is effective'.

Where should startups begin with efficacy research?

At least 90% of the people we work with want to do an experimental piece of work that demonstrates that their particular thing is better than something else, or better than nothing. We take a deep breath and go: ’OK, it’s probably not the way you want to go about it'. Because actually, that is a very narrow way of looking at things – if the evidence doesn’t demonstrate that, what have you learned? Not much. A much more sensible way is to say, ‘What has actually happened? How are the learners using it?' The number of times I have looked at a piece of technology, and you realise when you look at what learners are doing with it, they’re not actually using a key feature that the designer believes is the best thing since sliced bread. So you do need to try and look at the process as well as the product.

What’s the ideal timescale for a research project? 

Often the evidence actually takes years [to accumulate]. When I was first lecturing in university, I taught a group of first-year computer science students software design, and they hated the course, they absolutely hated it, because it made them work in groups and they didn’t want to work in groups, and they’d complain like billy-oh. Two years later they would come [to me] and say, ‘That was the best course I ever did'. Sometimes you can’t immediately see the effect of something. 

What’s the potential in this field right now?

The more people that interact with, or are observed by, technologies, the more information there is about how they are learning, and the more things we can understand about how we can support them to learn better. There’s a lot of information we can potentially get now that we have never been able to get before – so whether you are a researcher, teacher, publisher or edtech company, the existence of that information is in and of itself really interesting. I think it’s a fascinating time to be doing research where you’re trying to understand what is effective.

Book your tickets to FutureBook 2017 (including lower-rate EdTech for Publishers-only passes) here.