Speaker 1: As the others said, I'm going to focus more on some of the innovative approaches to collecting data using mobile phones, which kind of go beyond the stock survey. And I'm just going to give you a few examples and think about whether or not these are transformative or not. And in what sense do mobile devices give us the opportunity to do transformative research and change how we're collecting data? And I'm going to raise three interlinked challenges about ethics, about burden, and about non-response, and then take a small step back to think, well, can we just check that they really are giving us better quality data? You know, are we assuming too much? Are we carried away by the hype of what mobile phones can do for us? And then just a few comments about what we might do to make progress in this much newer area. So before I go on, I just want to make two acknowledgements. The first is that what I'm saying today is based on a paper that I've been writing with Patrick, so he can butt in if I'm being unclear or I'm misstating things. And secondly, that although there are a great number of attempts at innovation using mobile phones out there, a lot of the examples I'm going to use today are projects which I've been involved with or been carried out by TNS BMRB, because I'm also an associate there, so I just wanted to acknowledge that. So what kinds of opportunities are people using, and how transformative are they? This is just a small selection, and I'm really interested to hear if other people have examples themselves, but one of the commonest kind of themes, really, in the use of mobile surveys as kind of alternative ways of capturing data is to try and get closer to the moment of experience by asking people to do micro-surveys. Now Tim mentioned the great number of people do their surveys on a mobile at home, so we know people are carrying their phones around, they're with them. We shouldn't assume that they're always going to be using their phones in a mobile way. Nevertheless, we have an opportunity to try and capture people when things are happening. So there are a number of surveys from commercial research where people are asked to do a micro-survey while they're doing their shopping, or soon afterwards, or when they're out drinking. Very interesting study by TNS, where they asked people to report on the choices about drinks during the course of the evening. You can imagine that response doesn't exactly improve later at night. But nevertheless, different ways of measuring behaviour close to the moment. And these are also being done in social research cases, so for example, a study where children are being asked to wear an accelerometer while the parent is also being asked to use a short mobile survey to report on the child's activities. Just one example of something slightly different, but also about behaviour, is that Public Health England commissioned TNS to do a survey about whether or not people took up and used a programme which was encouraging them to cook more healthy food. So they were sent recipes, and the evaluation was whether or not people were actually making these foods, and whether they were making a difference to their lifestyle. One of the questions in the short survey, short mobile survey, that was part of this evaluation was, did you make this recipe, and please will you take a photograph of it. And I don't think when people first asked for that, they were sure how they might use that data. But when the photos started coming in, and they looked at the photos, they had like this middle one here, was a picture of the vegetable curry that somebody had made, very healthy, and they were serving it up with chips and garlic bread. So in the end, the photos were really used as a validation measure, which was really interesting, and helped evaluate the effectiveness of the programme. Antoworld Panel, which is kind of sister company, have an app where they ask people to report on their food snacking during the course of the day, because they want to understand food on the go to complement a wider project that they run. And people are asked to scan barcodes. So in the last example, you saw people using some of the intrinsic facilities in a phone, photographs. So barcode scanning allows you to get more information, and the key thing here is it's not that the mobile phone is just getting a picture of the barcode scan, it's the linking to very complex, interesting data about the nutritional value of that item, salt content and so on, and also back to consumer information about costs and the like. At the moment, Understanding Society, which is run at the University of Essex, they won an ESRC transformative research project, which is to take some of this commercial research technology and to implement it in the context of a large-scale random probability sample. So Understanding Society are doing a great project where people from the innovation panel are being asked if they will scan every single receipt that they receive during a five-week period, and the reason that's being done is because we're trying to find out whether or not we can get better information about people's expenditure. It's very hard to answer the question, how much did you spend in the last five weeks? So can we get better information by changing the nature of the question, asking people to simply photograph their till receipts during the period? Well, we will see how that goes. Coming to the end of the examples, but obviously there are the capabilities within a phone, for example, of GPS, and it really creates additional opportunities. One project you may be familiar with, because it was really quite popular in Britain a while ago, is Mappiness, which is a study that a guy called George McCarran at LSE set up as part of his PhD project. Not only does he ask a short survey about your momentary well-being, your happiness, how relaxed you feel, how awake you are, who you're with and what you're doing, he asks you to take a photograph and he records where you are, and he has used that information, that combination of data, to look at the environmental impact. Are people happier when they're outside in more natural environments? That's in fact what he found. And GPS is obviously also being used, the GPS built into mobile devices and mobile phones is being used to try and reduce the burden in travel surveys, so Natsen did a project some years ago, TNS-Nepo are working with people at Utrecht and other places to understand how you can use GPS in people's mobile phones or on devices you ask them to carry in order to improve travel surveys, their accuracy and easing the burden. But I just wanted to also draw attention to a different thing that's happening with geo-information at the moment, which is that they are also triggering surveys when people arrive in particular geographical zones, so you can basically geo-fence an area or you can put a beacon in an area and in the Netherlands this has been used to trigger surveys when people go through particular traffic hotspots and they're asked what their experience is. So in the same way that George McCarran, one thing I didn't mention about George McCarran's mapiness, he triggers surveys to you twice a day at random moments, so he is experience sampling across time and this is a different kind of use of control, as it were, of when surveys happen. So there are a whole host of other examples I won't go into here, perhaps worth saying that one of the things I haven't mentioned is given how important the mobile phone is to people's lives and how much is going on, there's another strand of work which is really about drawing the data out of people's mobile phones, asking permission to track what's going on, which websites they're visiting, when they're texting and so on. And that's almost a different theme of research but nevertheless very relevant. So I've talked about a whole host of different projects and you can see that there are a number of strengths of mobile phones as a tool. As several people have said, they're always on, they're connected, I'm mainly talking here about smartphones and I do appreciate that there are people who, you know, keep the phone in the handbag or don't have one at all, but nevertheless they're always on, always connected, it's very powerful. They have multimedia and sensor capabilities and there are these different options. You can either ask people to do tasks when they're out drinking, when they're out shopping or you can randomly ask them to do these things at different times or you can trigger them perhaps when they've had an interaction or in a particular zone. So I think, I'm interested to know what your views are, but I think there are real genuine opportunities there to collect different kinds of data that do change the nature of survey research and do allow us to imagine at least looking into behaviours in a different way to the standard survey, which is more fixed and occasional. But of course there are challenges, I'm probably doing terribly for time, but the first one of these is ethical considerations. Now at least we are engaging with people through their phones in a sort of survey environment so we can ask for consent, but we do have to remember that some people will tick almost anything, so we have to take some responsibility for what we ask people. In addition, one of the aspects of these kinds of surveying is that they're more intrusive into other people's lives. So I do mappiness for example and I'm busy clicking photographs of my friends and family and they aren't participants and they weren't asked consent, so there are issues about the effect on other people. In addition, there are higher risks of disclosure, not necessarily because this is a mobile phone survey but because any survey that begins to bring together different data types is bound to create more risk of disclosure. So again we take the instance of my mappiness participation, it knows I was in Berlin last week, it knows I'm here now, it knows I live in London, and there are various photographs of all around my environment, so the combination is important and risky. A final thing I want to mention is there's a recent trend towards, I think some time ago people were very experimental about giving people kit during surveys, what extra things can we measure. The trend now given constraints on budgets seem to be how far can we use other people's own kit, how can we use their own equipment, their own devices. Quite a lot of discussions for example about whether you can use people's exercise trackers and draw data from those and obviously using people's own mobile phones for these kinds of examples. But what you're relying on then is you're relying on the sort of data security of people's devices. A recent report for example which looked at exercise trackers showed that a lot of them have got terrible data security, sorry to say, and very varied. And so a researcher who's kind of relying on data drawn from other people's devices is sort of, needs to think hard about whether they're leaving all of that risk to the individual and what that means. And probably the answer is that we're going to do these things but we need to think hard, we need to give people good information and we need to be very careful with the data we collect but that in turn raises challenges about reducing opportunities for replication if we stick the data in a bank and won't share it. Two quick points about burden. The first is that as researchers we talk about, oh it's very burdensome to try and remember how much you spent in the last period, it's cognitively difficult. We probably have to remind ourselves from the perspective of the participant that although it may be cognitively less difficult to photograph your receipts over a course of time, you've still got to download an app, you've got to remember to do a task and it's probably going to take you an awful lot more time and effort. So we probably need to reflect on the whole idea of burden and keep reminding ourselves what the participant's experience is because it's the participant who's going to decide whether or not they're going to take part. The second thing I want to say about burden is that maybe we need to think about it a little bit differently and more from a different perspective. Tim was talking about, in fact both Tim and Michael were talking about people's expectations that their mobile phone should be part of their life and their expectation that they should be multitasking, doing different things with their phone. And maybe there's a positive, maybe rather than thinking about burden, it's a good thing that I'm doing Duolingo on my phone at one minute and the next minute I'm photographing my TIL receipt or telling George how happy I'm feeling just now. And the final of the three challenges, very much interconnected of course to ethics and burden is non-response. A lot of the examples that I've used come from the commercial environment or have been tried on quite small samples and we don't yet know, I think, how much non-response we're going to experience. In a sense, a pretty fundamental question because even if these measures are brilliant, if we've already excluded a group of people who don't have smartphones and then we lose others of particular kinds of people, introducing bias, that's a real consideration. And projects like the Understanding Society project are really exciting because they take these technologies and they put them into random probability samples. We're going to have much more evidence about this in the years to come. So more work to do on this I think is the best thing to say. I'm going to skip past that. Here's my step backwards about quality. All of these things I think are sort of super exciting and some of them I'm sure will bed down. For example, the use of GPS and location in travel surveys and things of that kind. It feels as if those are no-brainers and that even if there is a learning curve about how to do it well, some of these are going to take hold. Others, it's less certain. The key thing for us I think is to remind ourselves to explore quality issues and to remember. For example, we talk about in-the-moment measurement being a key feature of mobile surveys, but when we try to replicate George McCarran's mappiness study using an online panel, what we found is a great number of people respond very, very late and it's not really in the moment. It's not really at the moment you ask them. We need to check whether we're getting what we believe we're getting. Similarly, to come back again, I'm slightly obsessed with till receipts at the moment, to that particular project, is it really measuring people's expenditure? To what extent are we opportunistically taking hold of technology and an opportunity to measure something visible like till receipts? How far are we going to succeed if we map people's genuine spending? How far are we going to actually crack the problem of measuring expenditure more accurately? I definitely think it's worth exploring. I definitely think that we only learn by doing these things, but it's worth further investigation and consideration. That said, measuring quality may be more difficult with these kinds of studies than it might be in a traditional survey. For example, if I miss an occasion to tell you about a drink or a trip to the shops or whatever, you won't necessarily see it because you don't have a rectangular data set. What do we need to do to make progress? Well, obviously, more research. We're still learning how to design for mobile, as others have said. The sorts of work that Michael's doing, the thoroughness of that research in the area of using voice and using text, it would be fantastic for there to be more in-depth research about these newer technologies. We need more of that kind and more experimentation. I would argue, though, that we really should test and learn rather than sit and wait, because I think it's important to be evolving and trying these methodologies rather than waiting, say, for market research, commercial research, to find the answers and then find things have moved on again or you lack the expertise. Obviously, these kinds of projects involve a lot of collaboration. It's one of the things that makes them very exciting but quite difficult. There's also a kind of requirement to get them right that there should be a lot of early and open sharing, which doesn't always fit so well with publication timetables and people's aspirations and competitiveness as well between companies who are trying to race to the latest innovation. But, nevertheless, I hope you agree that there are transformative opportunities there if we can grab them.