Companies developing AI technology for the NHS have been urged by the Government to be transparent about the use of data and algorithms and make use of open standards.
These are the core elements of 10 principles with a new Code of Conduct for Data-driven Health and Care Technology published by the Department for Health & Social Care (DHSC).
The document also includes a number of commitments on the Government’s part, among them the development of a trusted approval scheme for data based products in healthcare.
The DHSC has published the code as a basis for suppliers to work with government and provide protections for patient data in new technologies, with an emphasis on AI.
Stand-outs of the 10 principles for tech companies include showing the type of algorithm being developed or deployed, the evidence base it uses, and how performance will be monitored and validated.
It includes elements of a standard operating procedure for the implementation of algorithms, which in turn includes the need to show their strengths and limitations, the learning methodology and the bar for acceptable use.
Need for transparency
The principle reflects one of the points frequently made in discussions about the use of algorithms in public services – that there has to be transparency around the process and the relevant data.
This is reinforced in three of the other principles: be fair, transparent and accountable about what data is being used; use data that is proportionate to the identified user need; and be transparent about the limitations of the data used.
As part of the latter, the document suggests a two-stage approach in applying analytics to any data: algorithms should be trained to understand the levels of data quality; then use the variables given to achieve their objective. This approach should be built in to handle variations in data quality.
The other principles are:
- define the user;
- define the value proposition;
- make use of open standards;
- make security integral to the design;
- define the commercial strategy;
- and show evidence of effectiveness for the intended use.
The prospect of an approval scheme for products emerges from one of five commitments for the DHSC within the code. It says the scheme will help to create the environment to support innovation by enabling suppliers to demonstrate that their products comply with the code.
It will involve ensuring that technology used in care is proportionate to the level of risk in different classes of product and robust enough to safeguard patients. In addition, it should be as straightforward as possible for compliant products to obtain approval.
Simplification and experimentation
Of the other four commitments, one is to simplify the regulatory and funding landscape to make it easier to innovators to scale up the development of their products within the health service. Another is to create an environment that encourages experimentation, making patient data available when it is within the legal frameworks.
Third is that there should be an increase in interoperability and openness, with standards for health and care data, the development of APIs and ‘rules of engagement’ for access to de-identified clinical datasets.
Finally, the DHSC says it will listen to users of the technology, including clinicians, commissioners and patients.
Health Minister Lord O’Shaughnessy said: “Artificial intelligence and machine learning is a field that is moving at lightning speed and has tremendous potential across the healthcare sector.
“That is why I am pleased to announce that we have today launched our initial technology partnerships Code of Conduct – 10 principles which set out the rules of engagement between industry and the health and care system. These principles provide a basis to deepen the trust between patients, clinicians, researchers and innovators.
“This is an important first step towards creating a safe and trusted environment in which innovation can flourish to the benefit of all our health.”
DHSC said it is launching the code in an initial form and encouraging feedback with the aim of strengthening its contents; but it is encouraging its immediate use. It also said it does not replace or change any existing regulatory requirements.
Image from GOV.UK, Open Government Licence v3.0
