Chloé Monnin came to artificial intelligence through language, not code. Trained as a linguist, she specialized in multilingual engineering before finding herself, almost naturally, testing the limits of large language models. Today at SBS, she works on the quality of generative AI solutions: a role that didn’t exist ten years ago, in a field that reinvents itself almost every week. She talks about her unique career path, what it really means to “trust” an AI system, and what a wool sweater can have in common with an algorithm.
How does your background in linguistics help you in your role as a QA engineer at SBS?
At SBS, I help ensure the quality of AI. My role is to test the models we use, along with all the banking-specific processing layers built around them. The goal is simple: to make sure the answers generated are reliable, consistent, and fit for our customers’ needs. What makes large language models (LLMs), the technology behind tools like ChatGPT, Gemini, or Copilot, different is that they’re non-deterministic. You can ask the same question a hundred times and get a hundred different answers. So instead of judging a response based on its wording, we focus on its meaning. We need to make sure the information remains accurate, regardless of how it’s expressed. That’s where my linguistic background comes in handy. Because we’re working with free-form text rather than structured data, language plays a crucial role in ensuring quality. It helps me analyze AI-generated responses, identify the key information, and verify that it’s being conveyed correctly.
You’re currently working on the SBS AI Foundation for banking clients. What does that involve?
The idea is to allow bank advisors to ask questions about their customer portfolio in natural language, just as they would with a chatbot. Behind the scenes, the platform automatically translates those questions into queries that databases can understand, retrieves the relevant information while respecting each user’s access rights, and then presents the results in clear, readable language.
My role is to test that translation layer between human language and technical language. We need to make sure the system correctly understands what users are asking and retrieves the right information. In the end, the user doesn’t need to know anything about the underlying technology. They simply ask a question in plain language, and the AI handles all the complexity behind the scenes.
What’s the biggest challenge you’re still trying to solve as a QA engineer ?
The main challenge is the non-deterministic nature of LLMs. In traditional software, the same input always produces the same output. With language models, that’s no longer true. As a result, we can’t just test the wording of a response, we need to evaluate whether the meaning remains correct and whether the information provided is consistent. It requires a completely different way of thinking. To do that, we rely on techniques such as semantic similarity analysis to check whether two different responses are actually communicating the same idea. This is still a very young field. Many of the methods we use come directly from research and are still evolving. We’re constantly testing, measuring, and refining our approach. Accepting a degree of variability while still guaranteeing quality is probably the most challenging part of my job today.
How do you make sure the AI isn’t saying something wrong?
My job is to ensure the substance is correct, even when the wording changes. I check that the AI retrieves the right information, doesn’t overlook anything important, and most importantly gets it right. One important point is that we’re not asking the AI to invent answers. We’re asking it to rephrase information that already exists. Users can also access the source data through tables or charts and verify that the answer matches what’s in the database.
We also test explainability. We don’t just ask the AI for an answer, we ask it to explain how it reached that answer. To do that, I work closely with subject-matter experts, who help validate that the information being presented is relevant and trustworthy. Ultimately, our goal is to guarantee three things: that the information is accurate, complete, and appropriate for the question being asked.
When you encounter hallucinations or unexpected behavior, what do you do?
You can never eliminate hallucinations completely, but you can reduce them and, most importantly, measure them. Depending on the model, we can adjust certain parameters, such as temperature, which influences the model’s level of creativity. The lower the temperature, the more stable the responses tend to be.
We also work on everything surrounding the LLM itself: controls, validation mechanisms, and business rules built into the processing pipeline. If the same question produces a correct answer 95% of the time but fails in the remaining 5%, we need to understand why and reduce that risk as much as possible. At the same time, we have to accept a reality: no LLM is perfect. The real challenge isn’t eliminating every error: it’s identifying them, controlling them, and making them as rare as possible.
What excites you most as a QA engineer about working with generative AI?
What fascinates me most is how quickly the technology is evolving. Every week brings new models, new capabilities, and sometimes new limitations as well. It can be overwhelming, but it’s also what makes the field so exciting. I also enjoy looking for weaknesses in LLMs: understanding when they fail, how they can be bypassed, and why certain safeguards work or don’t work. It’s a bit like a game of cat and mouse.
What impresses me most, though, is the creativity of users. People constantly find new ways to interact with AI and push its limits, often in unexpected ways. Watching this ongoing dialogue between human ingenuity and machine capabilities is fascinating.
Outside of work, do you have any hobbies that influence the way you think?
I’ve had one passion for a long time: knitting. It might sound surprising, but I see a real connection with computer science. In knitting, everything is built from just two basic stitches. With only those two elements, you can create an infinite variety of patterns. Computing works in much the same way. Everything starts with zeros and ones, and from those simple building blocks we create incredibly complex systems.
There’s even a fascinating historical story related to this. During World War II, some knitters are said to have encoded information about troop movements and patrol schedules into their knitting patterns. Using just two types of stitches, they created sequences that represented specific data. It’s the same principle: behind systems that look incredibly complex, you often find surprisingly simple foundations.
Knitting has also taught me the value of planning ahead. When you start making a sweater, you need to think about the structure before you begin. The same applies to any project.
If you could give one piece of advice to someone starting out in this field today, what would it be?
What has struck me most throughout my career is that the jobs I’ve done didn’t exist when I started my studies. In fact, the role I have today didn’t even exist when I began my career. That taught me that nothing is set in stone. Technologies evolve, jobs evolve, and you need to be ready to adapt continuously.
We’re seeing that happen right now in software development. AI is changing how people enter the profession. Many of the tasks that used to help junior developers build experience are now being handled by AI tools. The challenge is that you learn to develop by doing. If a machine writes the code for you, how will the next generation gain the experience needed to become experienced developers themselves? That’s one of the big questions we’ll have to answer in the years ahead.
If I had one piece of advice, it would be this: develop your resourcefulness. In a world that changes so quickly, the ability to learn, adapt, and seize new opportunities has become essential.
