To comply with the non-disclosure agreement, I have excluded and obfuscated any confidential information in this case study.
Background Story
Our client, an international gym, aimed to use an AI Chatbot to enhance training sessions with their customers.
Upon integrating the Chatbot into the client's mobile app, it became apparent that the majority of users were interacting with the chatbot in their native language.
The challenge arose when we realised our Chatbot was programmed to communicate solely in English.
Discovery
Following discussions with the client, we decided that conducting user interviews would provide deeper insights into the issue. Consequently, we initiated the recruitment of participants, guided by a specific set of criteria.
Some of the questions posed during the interviews included:
Have you experienced any problems while using the app? How did you resolve them?
Were there any challenges in communicating with the chatbot?
Did you encounter difficulties in conversing in English with the chatbot?
How would it benefit you if the chatbot could communicate in your native language?
Discoveries from the user interviews revealed:
A significant number of chatbot users are uncomfortable communicating in English.
Even among those who understand English, a preference for native language was evident.
Technical terminology proved too challenging for some users.
The client's existing mobile app, which hosts the chatbot, is available in the native language.
Considerations for moving forward included:
The importance of not requiring the client to overhaul their content.
Ensuring the solution is adaptable to multiple languages.
Assessing whether our information architecture can accommodate this change.
Working within a tight timeline.
Competitor Analysis
I carried out a competitor analysis to gain insights from existing solutions in the market.
Competitor Analysis
There are several notable approaches observed:
Developing separate bots for each language.
Implementing machine translation services.
Recognising my technical limitations, I understood the importance of leveraging the team's skills and expertise. I persuaded the product owner of the value in dedicating time to a design sprint, emphasising how it could help us validate our ideas and minimize the risk of developing an unnecessary feature.
Pre-Design Sprint
I convened the stakeholders, including 6 developers, 2 QA analysts, 1 Product Owner, 1 Sales Manager, and 1 Business Analyst. I explained the concept of a design sprint and how it can be utilised to address crucial business queries through design, prototyping, and user testing of ideas.
Design Sprint Day 1
Empathising
I created the Persona and User Need Statement to help the team get a clearer picture of the situation, introduce the affected user profile, and the difficulties that they were facing.
The business analyst and sales manager were invited to articulate the problem space from the business angle and to fill up any gaps we had.
Persona
User Need Statement
Define
The product owner and I began this section by detailing the customer journey, an exercise that was not new to the team, as we had conducted it several times before together.
The team took time to assess all the knowledge they had accumulated to date and were given a moment to identify both the challenges and opportunities present.
Each member was then requested to propose at least two How Might We (HMW) questions. This was followed by a dot voting session, which allowed us to prioritize the areas of focus for the Sprint.
How might we & dot voting
The Problem Statement How might we design a chatbot configuration process that minimises duplication of effort and enables the chatbot to communicate in the user's preferred language?
In retrospect, I acknowledge that our problem statement was overly broad. It would have been more effective to craft a problem statement that was more specific and targeted. Nevertheless, considering this was our initial attempt, we are still in the learning process!
Design Sprint Day 2
Ideation
Crazy 8s is a rapid sketching exercise designed to encourage participants to generate eight unique ideas in eight minutes. Contrary to the traditional one-minute-per-sketch rule, which I found too pressure-inducing, I adjusted the pace.
I also encountered some resistance from the developers, many of whom felt they were not skilled at sketching. I reassured them that this was perfectly fine! Instead of sketching, they were allowed to write their ideas. My primary aim was to foster a comfortable environment for team members to share their thoughts freely, without the pressure of needing to exhibit artistic talent.
Crazy 8s
Solution Sketching & Lightning Demo
Following the initial idea generation phase, we proceeded with solution sketching. In this stage, each team member received three frames to elaborate on their favourite idea, providing a more detailed visualisation.
After the sketching was complete, we conducted lightning demos. In these quick presentations, each team member showcased their idea to the group. Here’s an example of how that process unfolded:
Solution sketching & dot voting
After the lightning demos, we conducted a dot voting session. This allowed the team to democratically select the direction or concept that would proceed to the prototyping phase. By giving everyone a chance to vote, we ensured that the most compelling ideas were chosen for further development.
Design Sprint Day 3
Storyboarding
On the third day, the product owner and I dedicated our efforts to translating the selected idea into a storyboard format. This involved discussing and sketching out various scenarios to effectively demonstrate the solution, including potential failure cases.
Key scenarios we focused on included:
Setting Up a New Language for the Bot: Illustrating how users can add a new language to the chatbot, showcasing the steps involved in this setup process.
Viewing Content in a Specific Language: Depicting how users can view the chatbot's content in their chosen language, emphasizing ease of access and user interface design.
Editing Content of a Specific Language: Outlining the process for users to edit the chatbot's content in a specific language.
Selecting Language on the Chatbot UI: Demonstrating how users can select their preferred language directly from the chatbot's user interface, ensuring a seamless and intuitive experience.
Testing Content of a Specific Language: Showing how users can test the chatbot's content in different languages to ensure accuracy and appropriateness.
Through these storyboard scenarios, we aimed to cover the full spectrum of user interactions with the chatbot, from initial setup to everyday use, including how to handle errors or language switches effectively.
Design Sprint Day 4
Prototyping
On the fourth day, I began the process of prototyping the crucial pages required for user testing. Leveraging our Taiger Design System, I was able to expedite the creation of the prototype significantly. This design system provided a cohesive set of design standards, components, and best practices, enabling me to develop a functional and visually consistent prototype swiftly.
-- Content Hidden --
On the Following Week
User Testing
After developing the prototypes, I conducted tests with both our internal users and the clients to assess the functionality and user experience.
The tasks we set for participants during the user testing included:
Adding a New Language to the Chatbot: We evaluated how intuitively and efficiently users could add a new language to the chatbot's settings, testing the interface's clarity and the process's simplicity.
Creating an Intent: This task aimed to assess the ease with which users could create a new intent within the chatbot, focusing on the user interface's navigability and the process's intuitiveness.
Creating Intent Content for Both English and [Language]: Participants were asked to create content for an intent in both English and another specified language, testing the system's multilingual capabilities and content management features.
Testing the Intent on Chatbot UI, for Both Languages: This involved participants testing the created intent in the chatbot interface in both languages, evaluating the chatbot's response accuracy and language handling.
Asking a Question in a Language Not Set Up in the Chatbot: This task tested the chatbot's ability to handle queries in languages that had not been configured, assessing its response strategy and user guidance in such scenarios.
These tasks were designed to cover a broad spectrum of the chatbot's functionality, focusing on multilingual support, content creation, and the overall user experience across different languages.
User Testing Feedback
Learnings from User Testing
The user testing phase provided valuable insights into the prototype's functionality and user experience. Here's what was learned:
Ease of Adding New Languages: The process to add a new language to the chatbot was found to be straightforward and user-friendly, indicating that the design successfully facilitated this aspect of chatbot configuration.
Content Creation for Multiple Languages: It became apparent that when creating a new intent for a multilingual chatbot, it was not intuitively clear that content needed to be created for each language. This highlighted a need for a more obvious prompt or guidance within the interface.
Challenges in Multilingual Content Creation: The task of creating content for multiple languages was perceived as cumbersome by users. This feedback pointed towards the need for a more efficient way to manage multilingual content creation.
Concerns Over Machine Translation Accuracy: Users expressed concerns about the reliability of machine translations, underscoring the importance of ensuring high-quality translations for a seamless user experience.
The feedback collected from this round of user testing was instrumental in guiding further iterations of the chatbot, with a focus on improving user guidance, streamlining content creation for multiple languages, and addressing concerns about translation accuracy.
Outcome of the Design Sprint
The design sprint concluded with a validated concept that presented clear avenues for enhancement. This represents significant progress, demonstrating the effectiveness of the design sprint methodology in identifying and refining solutions to address user needs and challenges efficiently.