The AI Security Institute (AISI) is to prioritise research on how artificial intelligence could be used in cyber attacks, how to ensure it remains in human control and prevents societal disruption, and the risks in the technology influencing human opinions.
The UK Government body has highlighted the issues in publishing its research agenda, saying it is also placing an increasing emphasis on developing technical solutions that could be brought forward to face down AI threats by improving safeguards and advancing the fields of AI alignment and control.
This follows its change of name from the AI Safety Institute earlier this year, reflecting its focus on serious AI risks with security implications.
AISI said it is opening questions that must be addressed as the technology becomes more capable.
The agenda outlines work on various areas of risk research, including cyber misuse, criminal misuse, autonomous systems, societal resilience, human influence, science of evaluations and capabilities post-training. These are accompanied by solutions research on safeguard analysis, control and alignment.
It also sets out AISI’s work on the methodological approaches that underpin the understanding of risks from frontier AI, by developing more rigorous ways to evaluate models and conducting thorough risk assessments.
Instilling confidence
Secretary of State for Science, Innovation, and Technology Peter Kyle said: “If we are going to instill confidence in the British public about the incredible potential of this technology, then we need to mitigate any potential risks.
“This research agenda sets out for the first time not only how the Institute’s work is safeguarding our national security, but how we are laying a secure foundation to use AI to deliver positive change.
“Through the institute’s work, we are building the leading technical authority on AI security in the world – creating a scientific base of evidence which will support policy makers across the globe to make the right decisions.”
