AI — we want your input (emerging patterns, use cases, research findings)

[kubabartwicki]

Hello

We are starting a discussion about emerging design patterns for utilising AI in public services. This probably mostly covers generative AI, but other design patterns for making use of machine learning, data science, and similar technologies are also useful.

We’re doing this because there is a lot here! Many teams across government are experimenting with AI, and we are working out how to design for this. We need to work out how to surface AI-generated content to users, how to talk to them about accuracy, and how to design more dynamic interactions. A space to share learnings in the open seems like a good place to start 🥁

I work on the GOV.UK AI team, where we are experimenting how AI can help people navigate the complexity of government services and information. We have a ton of insights from testing so far, and we know that other teams do too. Most of our learnings come from experimenting with GOV.UK Chat. You can learn more about that in this blog post, or drop us a message if you want to chat more about it.

We’ll start by sharing some of our insights, but please join the discussion if you work in government and have things to share. Keep in mind that it’s all work in progress, and evolving fast!

[LucyMcArthur-GDS]

Describing accuracy limitations in an AI chat

Generative AI can sometimes show inaccurate information with a high degree of authority. We’ve been learning lots about how to communicate this and make sure users don’t follow inaccurate advice.

In the GOV.UK Chat experiment, we’ve had some success explaining accuracy limitations in a chat conversation, as part of the onboarding. Showing onboarding as messages received from the chat was a lot more effective than displaying the content on a static page. It also allows us to break the information down into shorter messages without overwhelming users.

We are also experimenting with integrating actions into the chat interface. During the onboarding, the bottom area of the chat shows two buttons. Once you’ve onboarded, this changes to include a text input field. Users responded really well to having rich UI elements in the chat, and it’s a pattern we want to build on. We have some really interesting content insights on this which we’ll share soon too.

[LucyMcArthur-GDS]

Checking answers

We also need to give users a really clear next step to do what they’re trying to do, and to make it easy for them to verify information generated by AI.

In our chat experiments, we are integrating a “check this answer” step in the user journey. This includes links to relevant GOV.UK guidance pages directly after chat’s responses. In research, we found that links were viewed as a way to find further information, but not always as a checking step, so we made this more prominent. Checking content isn’t a typical user behaviour within digital tools, but using a prominent “check this answer” component with a warning icon helped encourage it.

To make this more intuitive, we are also exploring things like linking people to specific parts of pages, or highlighting relevant content snippets.

[munhitsu]

How about referencing specific paragraph, sentence? I know it is harder to implement but it also makes reference checking faster.

[LucyMcArthur-GDS]

Thanks for your comment, it would be cool to explore referencing within the answer which I think is what you are suggesting (e.g. how Perplexity have numbers after specific sentences in the answers). We also explored highlighting specific sentences on a linked page, which is useful for users if their question is less complex.

[uxcharlotteryall]

Thanks for starting this thread :)
I'm working on patterns and components for gen AI as part of the work we're doing in the Incubator for AI (i.AI) , here in Cabinet Office.
I've started some conversations with Tim Paul and Charlotte Downs, in GDS, about how we might collaborate across gov to experiment and iterate on emerging AI components. But it'd be great to be more connected with others about what is happening across gov departments to all be helping each other with insights and learnings.

I've also been looking at other gen AI components outside of gov that we might be able to learn and experiment, here in the Incubator.
Lots of exciting times ahead. 👍

[kubabartwicki]

Nice, Tim mentioned this, would be good to chat!