Projects, organisations and businesses are all likely to have access to an array of data that could be utilised to understand and tackle social issues. However, the potential for this wealth of data is often unrealised because it is not shared in a way that is accessible and useful. While the need for and value of open data is now widely recognised, the practice of effectively delivering open data is variable. Skills gaps, leadership undersight and lack of strategic prioritisation tend to limit the effectiveness of open data for public good, but the right kinds of tools and resources can address this.
Engaging potential audiences
Over the last three years, Microsoft and the ODI have been working together as part of Microsoft’s Open Data Campaign, to support data collaborations to publish and use data to tackle complex social problems systematically and impactfully. One aspect of this work is to create tools and resources that support organisations to work towards more effective publishing of open data that reaches a diverse community of users. An initial review revealed several toolkits that focus on supporting people to utilise data for creating social impact, but far fewer that address the foundational needs of collating data appropriately, open publishing and effective distribution. Hence, a focus on publishing and user engagement related to open data seemed justified. Similarly, while provision for government stakeholders appeared generous, there appeared to be less aimed at researchers and professionals in the private and third sectors or for research-focused institutions, so much of our early outreach was to involve this audience. To better understand this audience and their needs, we engaged with potential users that would benefit most from the tools and resources. Through targeted outreach, we approached a range of projects, consultants, organisations and academics for structured interviews. This enabled us to explore views and experiences and validate early ideas about the type of content that might be useful to understanding, collecting, analysing and publishing open data. We connected with those who collect or publish data through our social media platforms, strategic communications to the ODI’s membership network and email communications.. For consistency in our conversations, we aligned our definition of ‘social good’ with Microsoft’s Open Data for Social Impact Framework which describes it as: ‘a positive change towards addressing a societal problem, such as reducing carbon emissions, closing the broadband gap, building skills for jobs, and advancing accessibility and inclusion’. This definition enables us to focus on broader social impact, rather than sector-specific support, especially as the toolkit and resources aim to support researchers and others who may have data where the value and impact might be further realised by being able to access and compare it across several areas.
Emerging needs
Several themes have emerged from engaging with audiences in the data field. The following needs tended to cut across sectors and professional disciplines but varied in intensity depending on the context or role in delivering open data.
The need for basic practical publication support.
For non-data professionals or projects with little resource for analytics, there was a need expressed for general and accessible guidance on how to collate, clean, present and publish data openly. Especially for smaller, one-off projects or small organisations, where processing data requests were beyond their capacity or remit, there was a need to make open publishing as easy and resource-light as possible. Building in sustainability was mentioned often, referring to the need for data maintenance to be built into data infrastructure, as well as the need to continue measuring impact (and sharing it) beyond the life of a project or programme.
The need for more specialised data skills.
People recognised the value in interrogating and reusing data as much as possible and the potential in packaging data so it could be shared across platforms and made comparable with other datasets. However, there appeared to be a lack of resources available to work with skilled professionals who were capable of doing so, in a timely fashion. Also communicated to us was the increasing demand to respond to bespoke data requests, the need to publish data in a wider range of accessible formats and the need to drive demand with different ways of analysing existing data, which were all impacted by availability of data skills.
The need to be more user centric.
There was little evidence or experience of data publishers and users engaging in proactive outreach or strategic follow-up to establish shared value or strategies about the impact of open data. While the desire to collaborate was palpable, relationships were temporary and one dimensional. Hence, user needs remained unmet and partnership potential was diminished. One regular example that cut across different sectors was the increasing need for automation and standardised ways to send, request and access routine data, such as longitudinal datasets or national statistics. This would help reduce the need for bespoke and time-consuming ad hoc data requests.
The need for ambitious leadership to set the data culture trend.
Some anxieties still remained regarding open data and perceptions of risks to reputation or threats to data security. Those we spoke to expressed a desire to see more ambitious leadership within organisations that managed (or where necessary challenged) the sensitivities around data sharing and encouraged a culture of investing in data skills, improving data quality, driving data strategy for wider good and using data with impact in mind. Some felt that corporate accountability should be strengthened by ambition, not by mandated or legal requirements, for example where open data benchmarking helped localities to compare and monitor progress. Others agreed that collaborative data collection across a region enabled a stronger collective voice when developing funding bids or collaborating on public initiatives.
Defining a response
In early 2023, we expect to design and test some ideas that support data publishers to address the key emerging needs. Data teams from international research projects, health organisations and academic programmes have been enlisted to help articulate and confirm how we address needs practically and create resources that can be widely applied within and across sectors, countries and professional disciplines. We will initially focus on developing guidance and tooling which work at scale, particularly around basic practical publishing support for more user-centric open data, but will look to address all of the outlined needs in greater detail in the long term. As we begin developing content for the resources that aim to address these needs, there is an overarching theme that will shape what we deliver. This is the recognition of the need for a more dynamic relationship between data publishers and data users, despite some actors filling both of these roles. Simply ‘getting data out there’ as a response to legal requirements or reputational expectation, although a key driver for open data, is no longer sufficient in the context of social responsibility. The reality is that effective and regular engagement with data users facilitates better recognition of how different actors value data, how they apply it in their work, and how it can be transformed into innovation or insight to help tackle long-standing social challenges. If you would like to test early versions of the tools and resources we are creating or would be interested in applying a more finished version to your work with data, please get in touch.