AI Perception Engine

Extract relevant customer information from your communications

Project Status


Our team has completed research on this topic. Twilio may or may not release products related to this research in the future.

Research overview

Every time a company engages with their customer, there's an opportunity to learn about them to help inform and personalize future engagements. But extracting and organizing all those customer insights can be challenging, which means that many companies leave rich troves of data from their customer interactions behind.

Wouldn't it be amazing if you had an employee tasked with writing down any valuable customer details that came up in a conversation?

With the AI Perception Engine project, we set out to better understand how LLMs can help our customers capture key insights from their customer conversations across different channels like Messaging and Voice, and translate unstructured data into structured data within thecustomer profile. Our customers could then reference these key traits later to unlock personalized experiences like custom routing or tailored marketing campaigns.

By leveraging LLMs, we were able to analyze conversation data to infer key traits about a customer without any custom training or the creation of templates on the user's side. By taking historic content into consideration, we could extract customer traits from more complex conversations that spanned multiple messages. Something that was previously near infeasible at scale.

Illustration showing a conversation between a support agent and a customer turned into trais for a profile

With this research, we also considered some risks with automatically inferring traits, including having the process feel too opaque to end-users and over-collecting traits. A couple of mechanisms we explored include:

  • Scoping traits: Not everything is relevant for every company, so our customers should have the ability to scope the inferred traits to their industry or use case.
  • Classification: labeling inferred traits so a customer can choose to treat them differently than those that they specifically collected.
  • Transparency: best practices to share with customers around how they can ensure that customers have transparency, and the ability to correct incorrectly-inferred traits.
  • Internal Transparency (audit log): providing tools to allow our customers to understand where in a conversation traits were inferred and why, similar to how we provide tooling to help understand how predicted traits made a decision. We also explored tools to help reject/revert wrongly inferred traits and fine-tune the model.

Learn more

We continue to explore these concepts within the AI Assistants project, and would love your feedback on our approach!

Join the AI Assistants waitlist(link takes you to an external page)
Learn more about AI Assistants