AI Studio
House all-things-AI
Vision
Streamline AI modeling processes across all user personas in a centralized platform.
Problem to be solved:
Lack of unified process or platform for building and managing Cresta's AI models.
Business need:
Accelerate AI modeling to shorten sales cycles.
My role:
End-to-end design leadership from concept to launch.
Outcome:
40% acceleration in model delivery time.
Learning:
Intuitive design reduces AI anxiety and enables seamless workflow integration.
What is an AI model?
Cresta’s AI layer recognizes visitor and agent actions.
AI Delivery team identifies expert behaviors and builds a sequence of actions agents must follow in a call.
But building this flow took a while…
It took 35+ business days for model delivery
It took us almost two months to get the model running from data ingestion. This was a long time and it affected our sales cycles.
Why was it taking so long?
Who's involved?
This is a simple representation of the processes AI Delivery team would go through.
Conversaion Designers discover the expert agent behaviors from the historical conversation data we would receive from our customers. Conversation Analysts label the examples and would QA the deployed models. The Machine Learning Engineers train, deploy, and evaluate the models.
When I learned about the AI Delivery process, there were two pieces that I wanted to understand further.
One was that labeling took a long time. Why was that?
Problem 1: Labeling taking too long
Fragmented tools
Conversational Analysts used multiple different tools to label. It would start on the old Studio app, QA on google sheets, and then manually update tasks in a separate doc... Working on the wrong sheet was a pretty common issue 😩
No task management
Conversational Analysts used multiple different tools to label. It would start on the old Studio app, QA on google sheets, and then manually update tasks in a separate doc... Working on the wrong sheet was a pretty common issue 😩
Where did the workflow break?
The second part I wanted to understand further was the dependency on the ML Engineers.
Often times, the AI delivery process was paused and blocked by the ML Engineers. MLEs were the only ones able to deploy the model.
Can we break away from this dependency?
Problem 2: Dependency on ML Engineers to deploy models
Cryptic model deployment process
Deploying new models was a cryptic and complex process that only a few ML engineers knew how to do. And until deployed, nothing could move forward, significantly delaying model delivery.
Research insights
1. Cresta needs a centralized platform
The AI work was only going to grow as the company grew. Now was the time to spend the effort to build an internal platform.
2. Workflows shouldn't be re-learned
Studio should detangle and iron out complex problems without adding complexity.
Solution 1: Unified tools
Everything under one roof
✅ Complete the entire AI modeling process in one tool. All the tools and workflows were brought under the same roof. No more looking for the right page, switching between gazillion tools.
Solution 2: Streamlined labeling
Just click Yes or No
What labeling looks like in the new Studio.
Studio provides sample conversations with expected intent, and all that’s required is clicking Yes or No (keyboard shortcuts included). No more looking for the right sheet and manually updating the tasks excel cells separately.
Solution 3: Spread out the work, efficiently
YES task management
Simple task management view provided the team with a shared source of truth. All labeling tasks are managed and distributed in Studio, instead of in spreadsheets and Slack. Conversation Analysts can immediately start by clicking on their task.
Solution 4: Everything written in human
Simple deployment process
Deployment process was simplified into 4 simple steps. No more cryptic forms. Now anyone, a CD or a CA, can deploy models without getting blocked by MLEs.
Bonus solution
Unified and simple
Before, the workflows were unique to each persona. But with the new Studio, the team was able to work more fluidly.
Unified tools and simplified flows allowed anyone to perform these workflows without much learning because of similarly designed workflows and paradigms.
Impact
Increased Output from the AI Delivery Team
The impact was pretty big. After launching Studio, we were able to increase the model delivery speed by 40%. From over 35 days, we reduced it to just 21 days since conversation data ingestion to model delivery.
