From d8f2cf731b0f01144bf09333cedb6bceba01d8b4 Mon Sep 17 00:00:00 2001 From: Julia Lindpaintner <42982184+juliaklindpaintner@users.noreply.github.com> Date: Mon, 4 Mar 2024 12:46:31 -0500 Subject: [PATCH 1/9] Create 2024-03-05-customer-experience-beyond-surveys.md Creates a new blog post --- ...3-05-customer-experience-beyond-surveys.md | 80 +++++++++++++++++++ 1 file changed, 80 insertions(+) create mode 100644 _posts/2024-03-05-customer-experience-beyond-surveys.md diff --git a/_posts/2024-03-05-customer-experience-beyond-surveys.md b/_posts/2024-03-05-customer-experience-beyond-surveys.md new file mode 100644 index 000000000..fbdcf7848 --- /dev/null +++ b/_posts/2024-03-05-customer-experience-beyond-surveys.md @@ -0,0 +1,80 @@ +--- +date: 2024-03-05 +title: > + Customer experience: beyond surveys + authors: + - amanda-kennedy +tags: +- 18f +- how we work +- +excerpt: > + Want to measure customer experience? Surveys aren’t the only way! Consider these guiding questions to help your team select an approach based on what you want to learn. +image: +--- + +# Customer experience: beyond surveys +Good customer experience (CX) in government is critical to meeting the needs of the public and agencies alike. Not only is CX a guiding principle within the [21st Century Integrated Digital Experience Act (IDEA)](https://digital.gov/resources/delivering-digital-first-public-experience/), but it can help foster trust in government services. The Office of Management and Budget’s [Section 280, Circular A-11](https://www.whitehouse.gov/wp-content/uploads/2018/06/s280.pdf) defines CX as: + +_The public's perceptions of and overall satisfaction with interactions with an agency, product, or service…[Customer experience] refers to a combination of factors that result from touchpoints between an individual, business, or organization and the Federal government over the duration of an interaction, service journey, and relationship._ + +Surveys are a typical starting point for measuring customer experience. I can see why — you can measure things like customer satisfaction with a service over time. However, surveys can be complemented by other efforts. + +I encourage government practitioners to start with what you want to learn from users, and why, before deciding on a particular research method. This post includes a series of guiding questions to help you and your team brainstorm and select an approach for measuring customer experience. + +## What are your overarching goals? +Before thinking about what to learn and why, you should start with your overarching program or service goals. Learning itself is often not the end goal, but something that can help you understand if you are on track to reach a larger goal. Ideally, this goal or set of goals should already be established and aligned on by your program. + +## Why do you want to learn? +This is one of the most important questions to ask yourself and your team. Measuring and learning give you the opportunity to make decisions in service of a larger goal. Sometimes, information gathering is about fulfilling a reporting requirement, and that’s okay. Regardless, what decisions can this initiative help you make? What can this initiative help you do? How can you improve your program or service with these inputs? Is it to benchmark progress? Measure the success of a launch? Learn about user needs? Increase enrollment in a government program? Whatever the reason, being intentional about why you want to start learning something can clarify your objective and inform what you want to measure. + +## Who are your users? +Next, identify which users you want to learn from. Try to add at least some level of specificity. “Members of the public” or “agency employees” may be too broad. “Members of the public who applied for benefits in 2024”, “taxpayers who are eligible but not enrolled in this program”, or “agency employees who processed permits in the last quarter” can help you make more informed decisions and more easily target these users during recruitment. + +This Mad Libs-style thought starter can help you hone in on your own audience: [user type] who [have completed an action or fit certain criteria] within [timeframe] + +A key aspect of customer experience is considering not just end users but agency employees or other stakeholders who administer or support your government program. +Also, keep accessibility and underserved communities in mind throughout this process so you can provide an equitable customer experience. Later in this exercise, as you plan out what method(s) you’ll use to learn from users, you should consider ways to reach people with disabilities or whose first language is not English, like including multilingual content, involving interpreters in interviews, or ensuring a screen reader can access your survey. + +## What do you want to learn? +Next, move onto what you want to measure. Other ways to ask this include, what do you want to learn? What can you measure that will help you understand if you are on track to reaching your goal? + +Consider framing what you want to learn in the form of a question. “Why” and “how” questions are especially useful for identifying root causes and user context. + +For each question you brainstorm, also ask yourself why you want to learn this, which can help you understand if they are in service of supporting a certain goal. + +## What method(s) could you use? +Then brainstorm what sources or methods make sense for collecting this information. Outline a few examples, like [user interviews](https://guides.18f.gov/methods/discover/stakeholder-and-user-interviews/) or [contextual inquiry](https://guides.18f.gov/methods/discover/contextual-inquiry/) (best for "why" and "how"-based questions), analytics (best for "what" and "where in the journey"), and surveys (best for attitudinal-based questions, and to some degree "what" and "where in the journey"). + +Find inspiration for different methods by reviewing [18F’s Methods cards](https://guides.18f.gov/methods/). + +## Choose your method +As you choose a method or set of methods with your team, consider risks, limitations, and constraints, like timeline, budget, or administrative burden on the public. + +Also be mindful of any tradeoffs that come with each method you’ve brainstormed. For example, it is easy to over-index on survey scores like [Customer Satisfaction Score (CSAT)](https://digital.gov/2016/08/05/csat-nps-ces-3-easy-ways-to-measure-customer-experience-cx/). They have neat and tidy numbers and are easy to show on a slide deck, but they don’t provide context. + +Surveys should not be created and analyzed in isolation. Instead, they should be part of a larger strategic effort. Consider pairing surveys with qualitative user research methods like [user interviews](https://guides.18f.gov/methods/discover/stakeholder-and-user-interviews/) or [contextual inquiry](https://guides.18f.gov/methods/discover/contextual-inquiry/) that provide context into user behaviors, needs, motivations, direct observation of them interacting with your service. + +Regardless of which method(s) you choose, you should see if there is any need for Paperwork Reduction Act (PRA) clearance. Your agency’s PRA Office will be a great resource for navigating this question. If you are conducting a round of user interviews or usability testing with 9 or fewer participants, you will not need clearance, according to [PRA.digital.gov](https://pra.digital.gov/do-i-need-clearance/). + +## When do you want to learn? +There are three ways you should answer this question for each item you’ve aligned on that you want to measure. The first is, when do you want to start gathering this information? And are there any limitations to how soon you can start gathering this information? If your method requires users to fill something out, you may need to wait until you have a significant number of responses, compared to a more passive data collection method, like web analytics, which you could already have at the ready. + +The next question is, which points of a user’s service journey would it be most useful to collect this information? At the end of an experience? If they are eligible but not enrolled? Or somewhere in the middle, like if they have applied but haven’t received a decision? Would any of these points better serve your end goals or help you make more informed decisions? + +The last question is related to frequency: How often would it be helpful to measure this information? For example, if you are benchmarking enrollment data, you may need to collect this information on a monthly or quarterly basis, compared to understanding user needs and pain points while designing a service, which could be done once, and possibly validated later after launch to understand if the new service meets their needs. + +## Implement +Finally, after aligning with your team and going through any other steps to prepare based on your method(s) of choice, you are ready to get started! How you implement your initiative will depend on the method(s) you’ve chosen. + +Here are other resources on learning about your users based on different methods: +### User interviews + - [18F’s UX Guide](https://guides.18f.gov/ux-guide/research/plan/) + - [Recruiting customers with feedback panels](https://18f.gsa.gov/2024/02/01/gathering-feedback-with-customer-panels/) +### Usability testing + - [18F Methods card on usability testing](https://guides.18f.gov/methods/validate/usability-testing/) +### Surveys + - [Touchpoints](https://touchpoints.digital.gov/) +### Analytics + - [Web analytics playbook](https://digital.gov/guides/web-analytics-playbook/)https://digital.gov/guides/web-analytics-playbook/ + From 2ffa90f7de2c8909aa2e02a90fcb549594569b76 Mon Sep 17 00:00:00 2001 From: Julia Lindpaintner <42982184+juliaklindpaintner@users.noreply.github.com> Date: Mon, 4 Mar 2024 12:47:56 -0500 Subject: [PATCH 2/9] Create amanda-kennedy.md Add author Amanda Kennedy --- _authors/amanda-kennedy.md | 7 +++++++ 1 file changed, 7 insertions(+) create mode 100644 _authors/amanda-kennedy.md diff --git a/_authors/amanda-kennedy.md b/_authors/amanda-kennedy.md new file mode 100644 index 000000000..821fd3a91 --- /dev/null +++ b/_authors/amanda-kennedy.md @@ -0,0 +1,7 @@ +--- +name: amanda-kennedy +first_name: Amanda +last_name: Kennedy +full_name: Amanda Kennedy +published: true +--- From ea0653c8aec8c22c577997b4c9f28d3bb35d3022 Mon Sep 17 00:00:00 2001 From: Julia Lindpaintner <42982184+juliaklindpaintner@users.noreply.github.com> Date: Mon, 4 Mar 2024 16:27:22 -0500 Subject: [PATCH 3/9] Update 2024-03-05-customer-experience-beyond-surveys.md Add missing tag --- _posts/2024-03-05-customer-experience-beyond-surveys.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/_posts/2024-03-05-customer-experience-beyond-surveys.md b/_posts/2024-03-05-customer-experience-beyond-surveys.md index fbdcf7848..37f455abf 100644 --- a/_posts/2024-03-05-customer-experience-beyond-surveys.md +++ b/_posts/2024-03-05-customer-experience-beyond-surveys.md @@ -7,7 +7,7 @@ title: > tags: - 18f - how we work -- +- user-centered design excerpt: > Want to measure customer experience? Surveys aren’t the only way! Consider these guiding questions to help your team select an approach based on what you want to learn. image: From b726a50e14231231b61fe477c08b67bed91bf0f3 Mon Sep 17 00:00:00 2001 From: Julia Lindpaintner <42982184+juliaklindpaintner@users.noreply.github.com> Date: Mon, 4 Mar 2024 16:58:42 -0500 Subject: [PATCH 4/9] Update 2024-03-05-customer-experience-beyond-surveys.md --- _posts/2024-03-05-customer-experience-beyond-surveys.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/_posts/2024-03-05-customer-experience-beyond-surveys.md b/_posts/2024-03-05-customer-experience-beyond-surveys.md index 37f455abf..69de97f3f 100644 --- a/_posts/2024-03-05-customer-experience-beyond-surveys.md +++ b/_posts/2024-03-05-customer-experience-beyond-surveys.md @@ -2,7 +2,7 @@ date: 2024-03-05 title: > Customer experience: beyond surveys - authors: +authors: - amanda-kennedy tags: - 18f From c0391fe4498505636ff1ed90b99b84d63ebde0fc Mon Sep 17 00:00:00 2001 From: Julia Lindpaintner <42982184+juliaklindpaintner@users.noreply.github.com> Date: Mon, 4 Mar 2024 17:07:53 -0500 Subject: [PATCH 5/9] Update 2024-03-05-customer-experience-beyond-surveys.md Trying to get rid of indentation in h3 headers at the bottom of the post... --- _posts/2024-03-05-customer-experience-beyond-surveys.md | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/_posts/2024-03-05-customer-experience-beyond-surveys.md b/_posts/2024-03-05-customer-experience-beyond-surveys.md index 69de97f3f..6cf84f2d0 100644 --- a/_posts/2024-03-05-customer-experience-beyond-surveys.md +++ b/_posts/2024-03-05-customer-experience-beyond-surveys.md @@ -68,13 +68,17 @@ The last question is related to frequency: How often would it be helpful to meas Finally, after aligning with your team and going through any other steps to prepare based on your method(s) of choice, you are ready to get started! How you implement your initiative will depend on the method(s) you’ve chosen. Here are other resources on learning about your users based on different methods: + ### User interviews - [18F’s UX Guide](https://guides.18f.gov/ux-guide/research/plan/) - [Recruiting customers with feedback panels](https://18f.gsa.gov/2024/02/01/gathering-feedback-with-customer-panels/) + ### Usability testing - [18F Methods card on usability testing](https://guides.18f.gov/methods/validate/usability-testing/) + ### Surveys - [Touchpoints](https://touchpoints.digital.gov/) + ### Analytics - [Web analytics playbook](https://digital.gov/guides/web-analytics-playbook/)https://digital.gov/guides/web-analytics-playbook/ From e1e3a2e26b18afed281b6e6c14f344a7f8262080 Mon Sep 17 00:00:00 2001 From: Julia Lindpaintner <42982184+juliaklindpaintner@users.noreply.github.com> Date: Mon, 4 Mar 2024 17:08:23 -0500 Subject: [PATCH 6/9] Update 2024-03-05-customer-experience-beyond-surveys.md Deletes stray URL --- _posts/2024-03-05-customer-experience-beyond-surveys.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/_posts/2024-03-05-customer-experience-beyond-surveys.md b/_posts/2024-03-05-customer-experience-beyond-surveys.md index 6cf84f2d0..c5d0664df 100644 --- a/_posts/2024-03-05-customer-experience-beyond-surveys.md +++ b/_posts/2024-03-05-customer-experience-beyond-surveys.md @@ -80,5 +80,5 @@ Here are other resources on learning about your users based on different methods - [Touchpoints](https://touchpoints.digital.gov/) ### Analytics - - [Web analytics playbook](https://digital.gov/guides/web-analytics-playbook/)https://digital.gov/guides/web-analytics-playbook/ + - [Web analytics playbook](https://digital.gov/guides/web-analytics-playbook/) From de59d1a1dcd061f40fd51b4c27ce3281d4a73278 Mon Sep 17 00:00:00 2001 From: Julia Lindpaintner <42982184+juliaklindpaintner@users.noreply.github.com> Date: Mon, 4 Mar 2024 17:09:29 -0500 Subject: [PATCH 7/9] Update 2024-03-05-customer-experience-beyond-surveys.md Make Mad Libs style suggestion code snippet --- _posts/2024-03-05-customer-experience-beyond-surveys.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/_posts/2024-03-05-customer-experience-beyond-surveys.md b/_posts/2024-03-05-customer-experience-beyond-surveys.md index c5d0664df..a9df8dba6 100644 --- a/_posts/2024-03-05-customer-experience-beyond-surveys.md +++ b/_posts/2024-03-05-customer-experience-beyond-surveys.md @@ -31,7 +31,7 @@ This is one of the most important questions to ask yourself and your team. Measu ## Who are your users? Next, identify which users you want to learn from. Try to add at least some level of specificity. “Members of the public” or “agency employees” may be too broad. “Members of the public who applied for benefits in 2024”, “taxpayers who are eligible but not enrolled in this program”, or “agency employees who processed permits in the last quarter” can help you make more informed decisions and more easily target these users during recruitment. -This Mad Libs-style thought starter can help you hone in on your own audience: [user type] who [have completed an action or fit certain criteria] within [timeframe] +This Mad Libs-style thought starter can help you hone in on your own audience: `[user type] who [have completed an action or fit certain criteria] within [timeframe]` A key aspect of customer experience is considering not just end users but agency employees or other stakeholders who administer or support your government program. Also, keep accessibility and underserved communities in mind throughout this process so you can provide an equitable customer experience. Later in this exercise, as you plan out what method(s) you’ll use to learn from users, you should consider ways to reach people with disabilities or whose first language is not English, like including multilingual content, involving interpreters in interviews, or ensuring a screen reader can access your survey. From 9c486334c7a3b0caf8dba71437233ea2c4b3462f Mon Sep 17 00:00:00 2001 From: Julia Lindpaintner <42982184+juliaklindpaintner@users.noreply.github.com> Date: Tue, 5 Mar 2024 09:32:09 -0500 Subject: [PATCH 8/9] Update 2024-03-05-customer-experience-beyond-surveys.md Change h3s to bolded text --- _posts/2024-03-05-customer-experience-beyond-surveys.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/_posts/2024-03-05-customer-experience-beyond-surveys.md b/_posts/2024-03-05-customer-experience-beyond-surveys.md index a9df8dba6..88135885a 100644 --- a/_posts/2024-03-05-customer-experience-beyond-surveys.md +++ b/_posts/2024-03-05-customer-experience-beyond-surveys.md @@ -69,16 +69,16 @@ Finally, after aligning with your team and going through any other steps to prep Here are other resources on learning about your users based on different methods: -### User interviews +**User interviews** - [18F’s UX Guide](https://guides.18f.gov/ux-guide/research/plan/) - [Recruiting customers with feedback panels](https://18f.gsa.gov/2024/02/01/gathering-feedback-with-customer-panels/) -### Usability testing +**Usability testing** - [18F Methods card on usability testing](https://guides.18f.gov/methods/validate/usability-testing/) -### Surveys +**Surveys** - [Touchpoints](https://touchpoints.digital.gov/) -### Analytics +**Analytics** - [Web analytics playbook](https://digital.gov/guides/web-analytics-playbook/) From c0b1dee1ff50471bd590d38d542b2143abc7f4f6 Mon Sep 17 00:00:00 2001 From: Julia Lindpaintner <42982184+juliaklindpaintner@users.noreply.github.com> Date: Tue, 5 Mar 2024 09:50:59 -0500 Subject: [PATCH 9/9] Update 2024-03-05-customer-experience-beyond-surveys.md Update heading --- _posts/2024-03-05-customer-experience-beyond-surveys.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/_posts/2024-03-05-customer-experience-beyond-surveys.md b/_posts/2024-03-05-customer-experience-beyond-surveys.md index 88135885a..0d35a0687 100644 --- a/_posts/2024-03-05-customer-experience-beyond-surveys.md +++ b/_posts/2024-03-05-customer-experience-beyond-surveys.md @@ -48,7 +48,7 @@ Then brainstorm what sources or methods make sense for collecting this informati Find inspiration for different methods by reviewing [18F’s Methods cards](https://guides.18f.gov/methods/). -## Choose your method +## What method(s) should you choose? As you choose a method or set of methods with your team, consider risks, limitations, and constraints, like timeline, budget, or administrative burden on the public. Also be mindful of any tradeoffs that come with each method you’ve brainstormed. For example, it is easy to over-index on survey scores like [Customer Satisfaction Score (CSAT)](https://digital.gov/2016/08/05/csat-nps-ces-3-easy-ways-to-measure-customer-experience-cx/). They have neat and tidy numbers and are easy to show on a slide deck, but they don’t provide context.