The Data Science for Local Government project was about understanding how the growth of ‘data science’ is changing the way that local government works in the UK. We define data science as a dual shift which involves both bringing in new decision making and analytical techniques to local government work (e.g. machine learning and predictive analytics, artificial intelligence and A/B testing) and also expanding the types of data local government makes use of (for example, by repurposing administrative data, harvesting social media data, or working with mobile phone companies). The emergence of data science is facilitated by the growing availability of free, open-source tools for both collecting data and performing analysis.
Based on extensive documentary review, a nationwide survey of local authorities, and in-depth interviews with over 30 practitioners, we have sought to produce a comprehensive guide to the different types of data science being undertaken in the UK, the types of opportunities and benefits they create, and also some of the challenges and difficulties being encountered. Our aim was to provide a basis for people working in local government to start on their own data science projects, both by providing a library of dozens of ideas which have been tried elsewhere and also by providing hints and tips for overcoming key problems and challenges.
Our key conclusions are as follows:
- Data Science is still in a nascent stage in UK local government work. For example, few authorities are exploiting the potential of machine learning to enhance service delivery, or exploring the use of artificial intelligence to enable different forms of interaction with customers and citizens. Hence there is enormous potential for the use of these techniques to be expanded, and thus to deliver better services to citizens.
- The key reason for this is that doing ‘data science’ in local government faces a number of crucial barriers. People we spoke to consistently highlighted the difficulty of finding time (and support from senior management) to produce innovative data science projects. Whilst in theory the context of austerity provides stimulus for innovation, in practice the dramatic reductions in budgets have meant that back-office analysts who have retained their positions are almost exclusively focussed on statutory reporting, with hardly any possibility of engaging in new work (especially with any risk of failure).
- Despite all these barriers, local government is also a site of considerable innovation, with a huge amount of pilot projects in progress in areas such as machine learning, artificial intelligence, data merging and A/B testing. There is often talk of a skills gap in local government, with people unable to hire the staff they need. But we found lots of examples of skilled analysts and business intelligence specialists working on remarkable projects with shoestring budgets. Hence, we would encourage local governments to invest more in the people they currently have by providing them with training and space to innovate, whilst looking less to third party contractors and consultants.
- It is also important to be clear about the potential outcomes of data science projects. The current case for many such projects is often built around the idea that they will save money. In the current climate of intense financial difficulty this is understandable. But we also believe this is fundamentally the wrong way to conceive data science in a government context: many useful projects will not, in the short term at least, save money. For example, data science projects which identify areas for early interventions still need to be supported by funds to actually carry out those interventions; whilst data science projects that identify needs more efficiently may also identify needs which were previously unknown. In short, data science should be conceived of as something that improves services for citizens, and allows people working in local government to optimise their time, rather than something which will save money.
- Data science projects are inevitably people focussed: they might be about supporting a frontline social worker in their day to day activity, providing insight and intelligence to senior management, or making decisions about intervention pathways for particular citizens. So, it’s critical that these people are involved in the projects! The best examples we found in our work involved close collaboration with agencies and citizens, with data science conceived of as a service rather than something that tells people what to do. Interestingly, when people who are generating the data can see how it is being used, then the quality of the data (and acceptance of systems) gets a whole lot better.
- There are strong concerns about privacy, ethics and accountability in the introduction of new data science technologies. These are important and can’t be marginalised, and the practitioners we spoke to were acutely conscious of issues such as potential bias when (for example) deploying new decision making technology. Clear and open standards and guidance about how to use data science techniques in a way compliant with existing legal and ethical frameworks would be a really important enabler for the sector.
- Finally, though many people have highlighted concerns about both the quality and quantity of data in local government, we found that while ‘big data’ might be desirable small data is often enough. It is true that many advanced analytical techniques are being developed in an industry context where having hundreds of millions of data points would be the norm. But we found encouraging examples of machine learning projects leveraging datasets of a much smaller scale. Hence, even though pooling data (and getting access to more) is tricky, people working in the area should be encouraged to start small and work with what they have, to develop quick proofs of concept, and to not be put off by potentially limited access to data.