– Updated March 2023 –
Case study: Training – leaders in data literacy
Inadequate skills and knowledge can impede an organisation’s ability to use data effectively. This can impact business outcomes, hold back digital transformation, and, reduce the ability of an organisation to compete in an increasingly data-enabled, digital-first environment.
The ODI has created new training methodologies and tools that develop new cohorts of data leaders and individuals every month online and in person. Partnering to co-develop precision courses and tools with government, industry, and the third sector empowers their teams to act on opportunities offered by data best practices and data sharing.
From 2018, the BBC in partnership with the ODI developed courses on machine learning, AI and ethics. These have been delivered to hundreds of employees and have led to changes in the way the BBC takes responsibility for its use of these technologies.
Since it opened its doors in 2012, the ODI has trained more than 50,000 people worldwide.
BBC Datalab is a team of data scientists and engineers that works closely with editorial colleagues to help audiences find relevant content. Datalab’s innovations use machine learning to ensure that audiences of all ages and from all backgrounds are able to engage with the BBC’s content. Among other things, Datalab has launched in-house recommendation engines for BBC Sounds, Sport, News, and the World Service to deliver a seamless experience to audiences.
Background
The BBC is the world’s largest public service broadcaster. It has weathered many changes since its first radio broadcast almost one century ago in 1922 – not least the impact of rapidly changing technology and the challenges and opportunities this brings. The BBC’s remit is to serve all audiences through the provision of impartial, high-quality, and distinctive output and services to ‘inform, educate and entertain’ – and to reflect the diversity of its audience with respect to age, culture, and socio-economic background. Every week, the BBC reaches 90% of the UK population with a diverse range of content. But, to stay relevant for all its audiences and to keep in step with other organisations, delivering a personalised experience to the consumer is a key requirement. Unlimited content across multiple platforms and increasingly personalised news and entertainment options are the norms for today’s media consumers, but the development and implementation of personalised services via recommender systems can be challenging. Recent issues that have surfaced in the news include gender bias in music recommendation algorithms and feedback loops amplifying biases in movie streaming services. The CMA launched a publication this year examining how algorithms can harm consumers, and the CDEI (Centre for Data Ethics and Innovation) recently published a roadmap to an effective AI assurance ecosystem as part of the National AI strategy.
Datalab has been at the forefront of the BBC's effort to provide its audiences with engaging and personalised content since its launch in 2017. As a team, it has developed and deployed recommendation engines across the BBC, including for BBC Sounds, News, and Sport. A key part of Datalab's work has been incorporating public service values into its recommenders. To that end, the team has devised several approaches and tools, which enable data scientists and engineers to closely collaborate with their product and editorial colleagues to develop recommendations that are relevant, trustworthy, and fair.
Working with the ODI
The ODI has been working with the BBC since 2018, when we first delivered a workshop to examine the future of data-powered services and new technologies, such as machine learning. This initial workshop examined familiar topics such as how to value data, who should ‘control’ different types of data, and how ethics should be taken into consideration. Feedback from the original pilot session was positive, and the ODI went on to develop a bespoke training programme for the BBC Academy, based on the ODI’s existing course Applying Machine Learning and AI Techniques to Data. This course is intended to encourage an open and ethical approach to developing machine learning innovation at the BBC, ensuring that the large variety of data necessary for innovations such as recommender systems is collected and used ethically and transparently, and that the BBC is able to explain to audiences how and why data about them is used in developing services. The course also explores how incorrect application of machine-learning techniques can very quickly lead to negative outcomes that are not always obvious at the outset. It is not just aimed at developers, but can be taken by anybody needing to understand the ethics, opportunities and limitations of applying machine learning techniques to data.
The outcome
The course has now been running for three years. It is one of the most popular in the BBC Academy and has been delivered to 48 different departments. All participants on the course can join the BBC’s Data Ethics working group which, in collaboration with BBC Technology Strategy and BBC Research & Development, has supported the creation of the BBC Machine Learning Engine Principles. The BBC is committed to certifying a number of staff as ODI Data Ethics Professionals and Facilitators. The BBC’s commitment to take full responsibility for the functioning of its machine learning engines (both in-house and third party) and to explain how they work is now supported by a high level of data literacy within the organisation. At the ODI, we firmly believe that this is essential in enabling people to think critically about data in different contexts and examine the impact of different approaches to ensure equal and fair outcomes. Going beyond principles, operationalising data ethics in an organisation requires clear guides and processes in place. The BBC has made its checklist public as part of the Machine Learning Engineering Principles. Additionally, the ODI’s Consequence Scanning and Data Ethics Canvas are two industry-leading tools to help support organisations, from project inception to ongoing monitoring and review. These tools are not only applicable to applications of machine learning but to any project involving the collection, use or sharing of data. By employing a thorough approach to data and machine-learning ethics, and building thoughtfully and inclusively, the BBC is able to stay relevant for audiences in a new environment, delivering a personalised experience that is relevant to all.
Get involved
At the ODI, we want to support the public sector in better understanding and using data. We have developed a free Data and Public Services Toolkit to help service managers overcome the challenges with data sharing and to improve services for citizens. We also offer bespoke courses and training programmes for organisations, such as the ‘Introduction to Machine Learning’ course for the BBC. If you would like to get in touch to see how we can help, please email us at [email protected] to find out more. You can learn more about Datalab by listening to the ODI Fridays lunchtime lecture from Alessandro Piscopo (Principle Data Scientist, BBC) on How the BBC builds public service recommenders with data, or by reading the BBC’s paper describing its approach.