Immediate need to improve the management of accumulated digital records, says Cabinet Office report
Public bodies face “significant risks” from the legacy digital information they have accumulated but are often unable to obtain the benefits, according to a Cabinet Office report on the painfully drawn-out transition to a digital Civil Service.
In its final report on Better Information for Better Government, the Cabinet Office says there is “an immediate need to improve the organisation and management of departments’ accumulated digital records”.
The most significant risk is breaching data protection law with old data. Keeping it indefinitely would contravene the principles set out in the Data Protection Act, as the body of information held on government servers would, almost certainly, include personal information, the report warns.
The danger will become greater under the EU General Data Protection Regulation which is scheduled to replace the Data Protection Act in May of next year. The new law is based on the principles of data minimisation and storage limitation.
“Wholesale retention would place the UK at risk of significant fines and risks a regulator ordering processing to cease,” the paper warns.
e-Discovery
One solution would be to adopt artificial intelligence technology in the form of “e-discovery” systems currently used by City law firms to sift through vast unstructured archives for data of interest, learning as they go. However, automating knowledge management may not be possible for the forseeable future.
“eMails pose a particular challenge to departments that is not easily solved by any single tool,” the report notes. They provide a good example of how technology does not provide a catch-all solution: digital material still needs to be saved, filed or deleted as part of daily business, otherwise government will continue to accumulate digital material in an unstructured way.
“For that reason, human information management activity will still be required for the time being.”
The transition from paper based working to email and electronic documents has “undermined the rigour of information management across much of government.” While little information has been lost altogether, much of what has accumulated over the past 15-20 years is “poorly organised, scattered across different systems and almost impossible to search effectively”.
“This not only undermines government’s ability to structure and preserve long term records, it also creates real and immediate risks for accounting officers, who may be unable to provide evidence for past decisions and actions or to meet their statutory obligations for public records and freedom of information.”
A properly managed archive of searchable digital legacy information would have many potential uses. One would be giving individuals embarking on new policy projects the ability to search for similar work before.
“Wasted effort recreating old work might cost government nearly £500 million per year,” the report says.
Image from GOV.UK, Open Government Licence v3.0
