Effective Government Data Management in Public Sector

Data management is a complex issue. The Information Commissioner’s Office (ICO) registered multiple data breaches and losses in recent years – technology and process needs to support better mitigation for these risks of error and oversight releasing private citizen data to the outside world.

The ICO is currently working on the UK Government’s digital identity and attributes trust framework, with a focus on Government Data Management. This will address data protection, reduce security risks and establish accountability for the way that personal data is processed.

Security is often the aspect that brings the most public scrutiny. But there’s so much to think about in the data management space and this complexity—across functions and departments—exacerbates the challenges. The three most common challenges are:

• 1. Data storage (how you store the data)
• 2. Data availability (how you get the data)
• 3. Security (how you expose the data)

It is important to note that—despite the challenges—the public sector is ahead of other industries when it comes to data efficiency and usability, according to a study by Veritas Technologies. The Veritas report argues that the “Public sector stores double the amount of clean data (30%) vs general industry (15%).” It also states that “While 61% of respondents said reducing costs was their biggest challenge, 34% prioritised improving data efficiency and data management practices.”

1. Data Storage

The Veritas report found that the average public sector respondent was spending as much as £696,460 a year on data storage, half of which is dark.” Data storage can be costly for Central and Local Government. But the pace of technological change means that the shift to the cloud is holding back legacy transformation. According to a survey, 57% of the Public Sector leaders polled said their department would sacrifice technology performance in favour of enhanced security.

The key steps in getting data storage right are to:

• i) Define the value of your data: This means assessing how soon you need it back if lost, how long the data needs to be retained, the security standards it needs to meet and any regulatory requirements that should be considered.
• ii) Remove redundant data: It’s easy to accumulate redundant data and be hit by rising costs. Identify what data you need and use technologies that remove duplication and enable clones and snapshots. So that your data can store backups offsite and back up several times a day.
• iii) Make sure you can find the data you need: If you can’t access the data, there’s no point in having it. You need to find a middle ground between enterprise access and the needs of your data scientists. Providing a cleansed data set allows users to interact with the data in ways you haven’t thought of yet and drives innovation in cross-departmental analysis.

2. Data Availability

The Public Sector has set out its plans to use data more effectively to deliver better public services for citizens through data management. It is no secret that accessing and sharing data can improve the design, efficiency and outcomes of services. In Somerset, they have found that sharing data digitally between GPs and hospitals can enable early identification of high-risk patients, which has reduced hospital admissions by up to 30 per cent. The challenge now is to create a data infrastructure to share data across multiple departments and services. But this is not easy against a landscape where each service can have hundreds of incompatible legacy systems.

Reform identifies key ways that we can move this forward whilst a broader data model is developed:

• i) Public Sector Departments can restrict their software purchases to those that are compatible with Application Programming Interfaces, to support interoperability.
• ii) Adopt Open Standards at every opportunity to avoid vendor lock-in.
• iii) Departments can participate in the Data Advisory Board to help to address the issues faced by multi-agency data sharing.
• iv) Local Government can start to test data-sharing approaches to demonstrate where initiatives can be scaled nationally.

Moving forward towards a stronger, more inclusive data structure need not be a mammoth task. You can push for change at the micro level which can then be taken up to Data leadership for cross-departmental collaboration.

3. Data Security

The biggest barrier to data sharing is public trust. In 2020, the CDEI published its first report on public sector data sharing. The report’s survey results suggested that a significant proportion of the population (between 40-60% of people) believes that the government’s use of data is not serving their interests. Making sure that all data sharing is held to agreed standards of security and ethics is key to winning public trust. As long as trust remains weak, progress towards sharing across departments will be limited.

Departments can start to lay the groundwork for stronger public trust by embedding security at every stage of data use. Security's a huge topic that is better addressed at initiation than trying to retrofit safety. You need to look at whether you have a legal basis to see that data, how actions and events are logged and audited and visibility of data breaches and compromised datasets.

GDPR has transformed how we manage data but the focus is on large organisations storing and managing data. Legal challenges are gaining momentum—for example, the UK Government faces legal action over Test and Trace data retention. The public will not take breaches of trust lightly and there are likely to be tighter data regulations in the future. It is sensible to think about these issues now; get the basics of data management governance right from the start and these shocks will be easier to absorb.

The Future of Public Sector Data Management

The issues surrounding data storage, data availability and data management illustrate the unavoidable barriers to progressing towards data sharing. If we want to reach data democratisation, where local and central government can share data sets to foresee consequences, we need to first push for incremental change in our current data governance and management. The hidden rewards, or costs of moving towards this will only reveal themselves with time. But data innovation starts here.