AI Studio
House all-things-AI
Vision
Centralize and streamline AI modeling workflows for all user personas.
Problem:
No unified platform or process for building and managing Cresta’s AI models.
Business need:
Accelerate AI modeling to shorten sales cycles.
My role:
End-to-end design leadership from concept to launch.
Outcome:
Reduced model delivery time by 40%.
Learning:
Even complex, outdated processes can be successfully untangled with cohesive design.
What is an AI model?
Cresta’s AI layer recognizes visitor and agent actions.
The AI Delivery team identifies expert behaviors and builds a sequence of actions that agents must follow during a call.
But building this flow took time…
Model delivery took 35+ business days
It took nearly two months from data ingestion to get the model running—a lengthy delay that impacted our sales cycles.
Why was it taking so long?
Who's involved?
This is a simple representation of the processes the AI Delivery team would go through:
Conversation Designers discover expert agent behaviors from historical conversation data we received from our customers.
Conversation Analysts label the examples and QA the deployed models.
The Machine Learning Engineers train, deploy, and evaluate the models.
When I learned about the AI Delivery process, two areas stood out that I wanted to explore further.
One was that labeling took a long time. Why was that?
Problem 1: Labeling took too long
Fragmented tools
Conversational Analysts used several different tools to label data.
The process started on the old Studio app, QA on google sheets, and then manually update tasks in a separate doc... Working on the wrong sheet was a pretty common issue 😩
No task management
Communication relied on Slack, creating poor visibility, delayed updates, and ad-hoc tracking solutions that required manual maintenance. We weren't able to distribute tasks efficiently.
Where did the workflow break?
The second part I wanted to understand further was the dependency on the ML Engineers.
Often, the AI delivery process was paused and blocked by the ML Engineers because only they knew how to deploy models.
Can we break away from this dependency?
Problem 2: Dependency on ML Engineers to deploy models
Cryptic model deployment process
Deploying new models was a cryptic and complex process that only a few ML engineers knew how to do.
And until deployed, nothing could move forward, significantly delaying model delivery.
Research insights
1. Cresta needs a centralized platform
The AI work was only going to grow as the company grew. Now was the time to spend the effort to build an internal platform.
2. Workflows shouldn't require re-learning
Studio should detangle and iron out complex problems without adding complexity.
Solution 1: Unified tools
Everything under one roof
✅ Complete the entire AI modeling process in one tool. All the tools and workflows were brought under the same roof. No more looking for the right page, switching between gazillion tools.
Solution 2: Streamlined labeling
Just click Yes or No
Labeling in the new Studio looks like this:
Studio provides sample conversations with expected intent, and all that’s required is clicking Yes or No (keyboard shortcuts included). No more looking for the right sheet and manually updating the tasks excel cells separately.
Solution 3: Spread out the work, efficiently
YES task management
Simple task management view provided the team with a shared source of truth. All labeling tasks are managed and distributed in Studio, instead of in spreadsheets and Slack. Conversation Analysts can immediately start by clicking on their task.
Solution 4: Everything written in human
Simple deployment process
Deployment process was simplified into 4 simple steps. No more cryptic forms. Now anyone, a CD or a CA, can deploy models without getting blocked by MLEs.
Impact
Increased Output from the AI Delivery Team
After launching Studio, we increased model delivery speed by 40%.
From over 35 days, we reduced it to just 21 days since conversation data ingestion to model delivery.
