Let's Talk
Do you have a project that would benefit from a world-class team of data analysts, pop culture writers, and marketing strategists? We’d love to hear from you.
Get in Touch
Generative AI Editorial Evaluation
THE PROJECT
An SVOD client wanted to explore the possibility of using generative AI to provide descriptions of its content catalog. They wanted to understand both the strengths and weaknesses of their machine learning algorithms so that they could iterate on and improve it with the goal of determining its viability as an integrated part of their global synopsis-creation program.
THE SOLUTION
TBC assembled an international multilingual team to train the client's AI model by evaluating content descriptions created in English, German, Japanese, Korean, and Spanish. This included multiple variants for each piece of content, reflecting different orientations users have to the media that they seek out. Our evaluators were meant to grade individual blurbs and supply qualitative summaries of the AI’s potential.
THE RESULTS
Our team evaluated over 2,500 AI-generated descriptions, scoring each one across several criteria including accuracy, grammar, clarity, tone, and sensitivity, as well as providing comments. That feedback was captured as annotated metadata strings—accompanied by written post mortem reports—that the client was able to use to further develop their tooling and prompts.