Pavel Samsonov

Self-service chatbot design tool

Improving productivity through AI tool creation platform, supporting a repeatable and regulations-compliant workflow for “pioneers” throughout the company.

    My roles

  • drove C-suite alignment on problem definition, product strategy, and business outcomes to acquire funding approval
  • led user research to identify opportunities and select data retrieval as highest-value use case for the MVP stage
  • iteratively designed 0-to-1 product based on feedback as we scaled the product from a PoC through beta to general availability

    Product impacts

  • 2×
    2× assets under management by increasing productivity of agents and support teams
  • 75%
    75% of support queries automated with LLM agent synthesizing org-wide data sources
  • 99%
    99% reduction in lead times for new apps to be developed, deployed, and iterated
  • 40%
    40% fewer I.T. requests by allowing development and governance to be done once and used everywhere

Summary: Empowering teams to innovate independently

The customer’s executive leadership set aggressive annual targets, and teams looked to Generative AI as a way to increase their productivity. IT and compliance orgs were inundated with requests for implementing AI apps. My team was asked to define the value case for a self-service platform, and design the intranet experience around it.

Identifying enduring needs
  • Business users
  • IT leadership
  • Governance teams
Building the Value Case
  • Executive alignment
  • Amazon PRFAQ writing
  • Journey mapping
  • Solution hypothesis

Problem framing interviews showed demand for three main use cases: data retrieval with deep knowledge requirements, manual data transformation processes, and filling out forms. With plugins supporting those use cases, thought leaders could create their own original apps or discover and remix successful apps created by others.

I designed an experience powered by Amazon Q Apps that allowed users to self-service simple AI app generation. These apps could be deployed and shared in the internal environment without needing IT resources or governance approval for each individual instance.

Users can describe what they want their app to do, and Amazon Q generates a UI based on the prompt.
The generated app serves as a way to save & share useful prompts with predefined inputs that avoid the articulation barrier and prevent jailbreaking.

Building to learn

We brought engineers and customers together for a one-day build party, creating a vertical slice experience of the discovery and app generation flow. Users could generate apps and publish them into an internal catalog that acted as the landing page. Apps are discoverable through domain-based (i.e. "insurance") or task-based (i.e. "generate a spreadsheet") searches.

Iterating through artifacts
  • Service blueprint
  • Wireframes
  • Functional PoC
Validating with customers
  • Line-of-Business leaders
  • Super-users
  • Governance teams

Throughout the event, the team observed customers deploy 15 data retrieval and transformation apps in under 5 minutes. I defined the research strategy to gather data from this exercise, interviewing participants before and after their attempts, and observing them as they tested the platform.

One important delighter was the presence of the "Like" button. Users treated app rankings as a measure of quality, and app owners felt encouraged to improve their apps.

From one user to many

Following the build party, I led the synthesis activity to identify opportunities for improvement. This research informed the design of a follow-on iteration for limited beta release that emphasized the quality, governance, and usage of the generated tools.

The model can process the prompt to evaluate its quality, suggest data sources, and catch policy violations.
The metadata for the app is also LLM-generated, but app owners can update it - as well as control who can access their app.

Get in touch if you want to learn more about this case study.