Data can help unlock huge value for the economy, society, and the environment, by improving public services and increasing efficiency and effectiveness in business. But data needs to be trusted and trustworthy to support growth and innovation.

Why do we need data assurance?

Some of the biggest challenges we face as a society, such as managing the impacts of climate change, securing access to healthcare and reducing poverty, require data to be shared between organisations and across sectors, societies and governments.

Organisations and individuals collecting, using and sharing data need to provide – or receive – assurance that data is fit to share and trustworthy – to inform their decisions and support their products and services. At the ODI, we believe that the need for a trustworthy data ecosystem has become even more urgent in the context of recent AI innovations and advancements in machine learning and natural language processing. Data is essential for building capable AI and digital systems and a trustworthy data ecosystem enables secure and seamless data flows.

The way that data is shared is changing. Many organisations share data person-to-person, with a small amount of automated machine-to-machine sharing. This model is expected to alter, with data being more commonly shared between machines and little – or no – human intervention. For example, in digital twins – virtual representations of real-world physical activities and infrastructure, such as traffic models – there is a requirement to receive real-time information from many sources as well as the core baseline data. With less human interaction, trust is an important consideration that requires upfront work to enable this model to work.

Trust in how organisations collect and use data is becoming increasingly important across society and the economy. This is evident in lower levels of trust in institutions, like banks, utility providers, social media organisations, and government organisations, that fail to follow ethical practices in collecting personal data. Equally, concerns over fraud or data theft can undermine trust in legitimate data-sharing practices.

What is data assurance?

In this context, data assurance is becoming increasingly important. At the ODI, we define it as “the process, or set of processes, that increase confidence that data will meet a specific need, and that organisations collecting, accessing, using and sharing data are doing so in trustworthy ways”.

We have identified eight common factors likely to be important when assessing whether data supply and reuse (data flows) are trustworthy. These are:

• Organisational governance and internal oversight – the extent to which there is a clear strategy for data, policies, and processes are in place to identify and manage risks, and there is leadership with responsibility, accountability, and capacity to deliver.
• Privacy – the level of protection given to the data of employees, customers, and partners.
• Security – the extent to which systems, policies, and processes keep data secure and protected. This might include adherence to common cybersecurity standards.
• Quality and accuracy – the extent the quality and consistency in how data is collected, used, and shared meets required standards or specifications set out.
• Ethics and transparency – the extent to which an organisation states and adheres to ethical principles, sustainability standards and is open to public examination.
• Data access processes – the extent to which policies and processes surrounding data sharing, provision, and access are embedded and resourced.
• Skills and knowledge – the extent to which the knowledge, training, and learning required to collect, use and share data in trustworthy ways is understood and in place.
• Engagement and accountability – the extent to which the interests and concerns of data stakeholders are actively sought and addressed, and the organisation can be held to account if necessary.

Data assurance at the ODI

Through the ODI’s data assurance work, we are exploring this important and rapidly developing area in managing global data infrastructure.

We believe the adoption of data assurance practices, products and services will reassure organisations and individuals who want to share or reuse data, and support better data governance practices, fostering trust and sustainable behaviour change.

Our work so far

The ODI’s interest in this area is not new. Over the years, we have worked with companies and governments to both help them assure data, and be assured by the quality of the data provided by others – we just haven’t always used the label ‘data assurance’.

Our work to date has included:

What’s next?

Our future work will include:

• Continuing with research to assess the need for data assurance, and working on public policy advice and initiatives to enable as many people as possible to benefit from the value of data sharing
• Creating tools, and a code of conduct, to support those that are working to assure both data and the organisations sharing and using it
• Developing guidance for data assurance. In time, the ODI will enable the development of certification and accreditation services to support data assurance codes of conduct and standards
• Convening a steering board across government, regulators, not for profit organisations and industry to support the strategic direction of this work, specifically to determine which sectors and use cases will deliver the greatest benefit from the development and adoption of standards to support the trusted flow of data across the ecosystem. Our initial research suggests these would include healthcare, energy and finance (in addition to environmental performance)

To get in touch with the team, email dataassurance@theodi.org