The UK Government is beginning the next stage of testing the GOV.UK AI chatbot through providing advice to small businesses.
The Department of Science, Innovation and Technology (DSIT) said that up to 15,000 people will be able to ask GOV.UK Chat for advice on business rules and support, with the chatbot linked to 30 of GOV.UK’s business pages, such as ‘Set up a business’ and ‘Search for a trade mark’.
People with access to the trial can ask questions about tax and the support available to them.
A team of in-house data scientists, developers and designers are building the experimental tool using OpenAI’s GPT-4o technology, which aims to help people more quickly navigate complex advice to understand what matters to them.
It works through the Retrieval-Augmented Generation (RAG) method, under which a user asks a question, the system searches for relevant information on GOV.UK pages then generates an answer, ensuring that it is appropriate and safe.
DSIT said that users will receive straightforward, personalised answers that collate information that may otherwise be spread across dozens of pages.
Trial details
Under the trial - which is set to run through this month - selected pages on GOV.UK directed at small businesses will include links with an invitation to try the chatbot.
The results from the trial will determine the next steps, which could include larger scale testing and could ultimately lead to the chatbot being rolled out across the full government website, made up of 700,000 pages.
The GOV.UK website attracts over 11 million users per week and is the best known digital service in the UK according to research company YouGov.
Science Secretary Peter Kyle said: “Outdated and bulky government processes waste people’s time too often, with the average adult in the UK spending the equivalent of a working week and a half dealing with public sector bureaucracy every year.
“We are going to change this by experimenting with emerging technology to find new ways to save people time and make their lives easier, as we are doing with GOV.UK Chat. With all new technology, it takes time to get it right so we’re taking it through extensive trials with thousands of real users before it is used more widely.
“This is an essential part of our ambition to use AI to improve public services in a safe and reliable way, making sure the UK Government leads by example in driving innovation forward.”
Results from the trial, which will be published when they have been reviewed, will help to shape further development of GOV.UK Chat.
User satisfaction
DSIT said that in the first trial, which was conducted late last year, nearly 70% of users agreed that the responses provided were helpful - where under 15% disagreed. But the trial also showed that more testing and development was required to meet the high accuracy standards for advice and information on GOV.UK.
Because of the way generative AI technology works, government cannot predict and fully control every response. However, the technology allows the tool to provide tailored responses to individual questions and specific challenges users might have.
The department added that stringent safety measures and guardrails have been put in place, such as determining which questions the chatbot should not answer. These include preventing it from responding to queries that may prompt an illegal answer, share sensitive financial information or take a political position.
Experts at the AI Safety Institute have also been consulted on this work.
Onboarding and accessibility
User experience improvements have also been made, including the introduction of an onboarding process and accessibility improvements - as well as changes to increase the accuracy and completeness of the answers generated.
DSIT warned that, give the emerging nature of the technology, it is likely that the chatbot will still provide some inaccurate and erroneous results; but the new trial will provide the insights necessary to make further improvements and develop the tool for larger scale testing.
The GOV.UK team in the Government Digital Service, which is developing the chatbot, said it is looking to gather a strong dataset about how GOV.UK Chat is used in real world scenarios, which it estimates will take four weeks.
It also emphasised that no personal data is used in the service, and that the system can detect any common forms - such as phone numbers - to prevent them being included in questions.
Further details of the technology and trial added on 5.11.24 pm on availability of information from GDS
