Russell Macdonald, chief technologist at Hewlett Packard Enterprise, explains an approach for the public sector to use its data as a catalyst for modernisation
It is possible to think of the UK public sector as a House of Data. It contains immense stores of data from which there is a huge potential to create social value, providing better services, improving the lives of people and making the country a better place.
This builds on a wide awareness that more can be achieved by sharing data between organisations to provide insights on societal problems and provide better targeted and more coherent services.
But it comes with a number of complex challenges, not least that even within the public sector the data is dispersed among a wide range of organisational and technology siloes. While more and more government digital services are deployed in public cloud, much of the core data that underpins those services remains in legacy applications and databases. This increases the complexity of the challenge, demanding a flexible and technologically sophisticated approach to collect, sort, interrogate and share the data for a myriad of purposes.
Constant change
Part of the challenge is in the basic characteristics of the data. It is constantly changing to reflect new types of transactions, a need for new attributes in understanding people and places, and the requirements of the organisations that hold it. There are also variations: two or more organisations can hold data that is essentially the same but labelled and stored differently to reflect how they need to use it.
In addition, it is often widely dispersed in organisational siloes. These have existed for a credible reason – to facilitate specific processes – but issues with legacy systems can make them difficult for other teams from inside and outside the organisation to access, even when they have a legitimate reason.
All this requires a sustained effort to keep up with the changes and understand the data and where to find it, and a number of important factors have to be taken into account.
A big one is around data retention. The public sector holds an ever growing amount of data, which brings an environmental cost in the energy used in storing it, and raises questions about how long it is legitimate to retain information on individuals.
Transaction factor
Much of the data is based on transactions, some of which could have a long term relevance to future activities, but by no means all. Organisations need to work out and clearly define how long they should keep specific types of information after a transaction has taken place.
This is not easy, especially as it impossible to foresee all of the future use cases for a dataset; but it needs a realistic assessment of what data will be needed in the future, and how long after a transaction it should be retained.
Issues around privacy and ethics have to be taken into account. There has been a high profile debate on what data on individuals it is legitimate to collect, where is the line between a valid purpose and intrusion, and where and how it should be shared. The UK General Data Protection Regulation provides a legal framework, but there is scope for interpretation of its details, and this will be tested further as new types of data emerge.
Another long running challenge is the need for data to be structured to support the interoperability of digital systems for service teams and organisations to work together – often a problem when different sources have developed their data with different priorities. Achieving interoperability requires a granular understanding of the data and a wider use of common features – such as a shared taxonomy – within the public sector.
Complexity in collaboration
A move towards collaboration between public and private sectors is adding to the complexity. Public sector bodies see the value in opening up some of their data for app developers, infrastructure companies and other service providers to use; and they can provide data that feeds into operations in areas such as traffic management, public health and environmental management. But this needs interoperability and has to stay within the privacy and ethical frameworks.
The emergence of AI is also significant. It comes with a huge potential for the delivery of new services, but to do so it also needs data with a reliable provenance and the right characteristics. Organisations will need to invest time and effort to ensure their data is fit for the purpose of AI; but the technology will also help them to sift and categorise the huge volumes of data that they hold.
All of these factors are already at work within the public sector House of Data, which incorporates huge repositories in on-premise data centres, public and private cloud systems, and applications hosted in the cloud and in users’ systems and devices. These can be thought of as different rooms in the house, and it needs a clear view of their contents, easy access and the ability to move data between them to create social value.
Part of the solution is in adopting a hybrid data strategy that identifies and draws from all these sources, finding the full value from the data already available while looking to develop new opportunities and go further afield. The crucial element is to join up these sources wherever possible to develop better services and provide new streams of social value.
Multi-platform advantage
Hewlett Packard Enterprise (HPE) provides the tools to make this possible. It provides software and technology products, all of which operate on a multi-platform basis. This makes them effective on-premise and in public and private cloud environments, which reflected the hybrid data reality in which most organisations find themselves.
It also challenges the idea that the only way to modernise the use of data is to move it to the public cloud, believing that end-to-end public services are now delivered most effectively through an ICT estate that is balanced across public cloud, intelligent edge computing and on-premises systems.
This is not practical in every instance, and many services draw on data held both on premise and in the cloud but delivered on local systems.
HPE's solutions make it possible to modernise systems and services using data wherever it is held from edge to cloud. They facilitate a widespread joining up of data and strengthen the capacity for an organisation to identify a desired outcome then develop a solution to deliver this outcome from the data.
This capability provides a great catalyst for the modernisation of public services; and more importantly, it can help to create social value in providing real improvements in people's lives.
HPE has investigated these issues in more depth through a series of interviews with experts on the public sector’s use data. You can learn more through a series of three documentaries on the subject, covering: the value of data and citizen trust; the challenges in data hoarding, regulation and interoperability; and the future of AI and supercomputing. The documentaries are available here.
