The Cabinet Office is planning to use an algorithm each year to review the relevance of digital files for possible disposal.
Named the Automated Digital Document Review, it has been developed by the department’s digital knowledge and information management team and is based on a tool from software solutions provider Automated Intelligence.
As reported by UKAuthority in May of last year, it has been used to identify data that has no perceived value and could be deleted. This is in response to the growing ‘digital heap’ of unorganised, unstructured data in government, and provides an alternative to the time consuming human review of files.
Its progress has now been reported on as one of the Government’s algorithmic transparency pilots. The report, published last week, says the algorithm has now been used to review 5.1 million documents with further 300,000 in line for the current financial year.
Under the first round of deletions more than 1.6 million files were removed in July 2022 after being assessed as redundant, outdated or trivial.
Tens of thousands
The team plans to carry on using the tool, and while the number of files for the future unknown, the report says it is likely to be between 30,000-80,000 per year.
“The algorithm is part of a disposal methodology for digital information that allows the rapid review of high volumes of digital information that would otherwise be an extremely onerous and expensive task using traditional human review methods,” the report states.
“The algorithm sorts information using a mixture of classifications and language patterns/key words. The outcome of a review can then be visualised to support human officers to make decisions about what information to retain.”
