How Hostinger evaluates AI applications
With Liucija Veale, AI Engineer
Hostinger is a leading provider of web hosting solutions, serving over a million creators from 150+ countries. At Braintrust, we've had the opportunity to work closely with Hostinger as they work on exciting AI applications, like an AI Customer Support chatbot that now handles over 40% of support chat conversations.
In the video below, Liucija explains how she approaches AI evaluations.
Hostinger's approach to evaluations (0:10)
As Liucija explains in the video, the Hostinger team has 3 main goals when they run LLM evaluations:
- Evaluate how offline changes impact application behavior during development, including running evaluations as part of the CI/CD process
- Continually assess the performance of live AI applications
- Identify and prioritize emerging issues to further improve the application
What evaluations does Hostinger run? (1:47)
The Hostinger team runs a variety of evaluations. Some examples include:
- Component checks (URLs, correct doc)
- Automated comparison vs. expected answers (factuality, truthfulness)
- Automated evaluations without expected answers (sentiment analysis, safety)
- Internal human review (thumbs up/down, comments)
- Live user feedback (thumbs up/down, comments)
How Braintrust supports Hostinger (2:40)
Before Braintrust
After a quick setup process (Liucija got started by herself!), the Hostinger team now leverages Braintrust to run evaluations, track progress using a visual dashboard, drill down into specific improvements / regressions, manage / version datasets, and log production data. Braintrust saves the Hostinger team hours of manual evaluation a day, allowing Hostinger to work on 3x more AI features with high quality assurance.
Save hours daily on AI evaluation
Learn how Hostinger uses Braintrust to run automated checks, track live performance, and prioritize emerging issues so their teams can build more features.
Read more customer stories
How Fintool generates millions of financial insights
“Loop was our way of getting data or synthesizing log data more efficiently at an aggregate level. We use it to find common error patterns every single week.”
“With Braintrust, our science fiction writer can sit down, see something he doesn't like, test against it very quickly, and deploy his change to production. That's pretty remarkable.”