Pavel Samsonov

Self-service chatbot design tool

A major Canadian financial services company wanted to optimize their internal operations. I designed a repeatable and regulations-compliant AI tool generation workflow for “pioneers” driving the digital transformation of their teams.

Product impacts

Helped double assets under management by increasing the productivity of customer-facing agents and their support teams, who were now empowered to solve problems visible only to front-line staff, as well as iterate on them rapidly.
Reduced dev time to minutes compared to months of effort in the past, as customers with an identified GenAI use case no longer had to chart their own course through the complexities of IT tickets and governance approvals.
40% decrease in I.T. requests by allowing users to define their own bots with existing components that already had governance approval; this capability would go on to be extended to all AWS customers as Amazon Q for Business.
75% of support queries automated by generating a self-help response from the existing - though disorganized - knowledge base, allowing support staff to focus on higher-value tasks.

Empowering financial services teams to innovate independently

The customer’s executive leadership set aggressive annual targets, and teams looked to Generative AI as a way to increase their productivity. IT and compliance orgs were inundated with requests for implementing AI apps. My team was asked to define the value case for a self-service platform, and design the intranet experience around it.

Identifying enduring needs
  • Business users
  • IT leadership
  • Governance teams
Building the Value Case
  • Executive alignment
  • Amazon PRFAQ writing
  • Journey mapping
  • Solution hypothesis

Problem framing interviews showed demand for three main use cases: data retrieval with deep knowledge requirements, manual data transformation processes, and filling out forms. With plugins supporting those use cases, thought leaders could create their own original apps or discover and remix successful apps created by others.

I designed an experience powered by Amazon Q Apps that allowed users to self-service simple AI app generation. These apps could be deployed and shared in the internal environment without needing IT resources or governance approval for each individual instance.

Users can describe what they want their app to do, and Amazon Q generates a UI based on the prompt.
The generated app serves as a way to save & share useful prompts with predefined inputs that avoid the articulation barrier and prevent jailbreaking.

Building to learn

We brought engineers and customers together for a one-day build party, creating a vertical slice experience of the discovery and app generation flow. Users could generate apps and publish them into an internal catalog that acted as the landing page. Apps are discoverable through domain-based (i.e. "insurance") or task-based (i.e. "generate a spreadsheet") searches.

Iterating through artifacts
  • Service blueprint
  • Wireframes
  • Functional PoC
Validating with customers
  • Line-of-Business leaders
  • Super-users
  • Governance teams

Throughout the event, the team observed customers deploy 15 data retrieval and transformation apps in under 5 minutes. I defined the research strategy to gather data from this exercise, interviewing participants before and after their attempts, and observing them as they tested the platform.

One important delighter was the presence of the "Like" button. Users treated app rankings as a measure of quality, and app owners felt encouraged to improve their apps.

From one user to many

Following the build party, I led the synthesis activit to identify opportunities for improvement. This research informed the design of a follow-on iteration for limited beta release that emphasized the quality, governance, and usage of the generated tools.

The model can process the prompt to evaluate its quality, suggest data sources, and catch policy violations.
The metadata for the app is also LLM-generated, but app owners can update it - as well as control who can access their app.

Get in touch if you want to learn more about this case study.