196: AI Model Strategy in Translation — Alter Everything

Welcome to Alter Everything, a podcast about data science and analytics culture. I'm Megan Bowers, and today I'm talking with Olga Beregovaya, Vice President of AI at Smartling. In this episode, we chat about translation technology, her approach to implementing AI models, and interesting takeaways from the translation industry in the age of AI. Let's get started. Hi, Olga. It's great to have you on our show today. Could you give a quick introduction to yourself for our listeners?

Yeah. Hi, Megan. Thanks so much for having me. I run AI practice and all of our AI development and initiatives at Smartling. Smartling being an AI-first translation platform. Me personally, I've been in language technology industry for good over 25 years now, actually. If you count college, it's got to be more than that. Pretty much haven't done anything but that, and obviously being the witness and the participant of the growth trajectory and progression in the language technology.

Yeah, that's awesome. I'm excited to hear from you about just all of your experience. I think a good place to start would be hearing about some of the biggest transformative shifts you've seen in language technology over these past 20 years you've been in it.

Language technology, I would imagine that language technology is in general evolving in a very similar fashion to the way overall machine learning and overall intelligence technology has been evolving. So it's pretty much very similar. If you think about it, we had rule-based tech, right? For years, we had rule-based technology such as rule-based machine translation and different rule-based libraries, and we had ontologies, but everything was pretty much hand-coded and hand-created. I mean, if you were to create an ontology or if you were to create a dictionary for machine translation 30 years ago, 25 years ago, it basically involves manual coding, manually creating the parsers, or if you were running sentiment analysis, that would be manually training the engines. So I think the first transformative shift was probably going from more rule-based approaches to, at the time, statistical-based approaches. Where language technology alongside, again, other areas of machine learning actually learned to how to obtain the data and how to learn from the corpora or from any kind of massive data. So I would say going from rule-based to statistical full language technology was a big shift, but nothing compared to what happened when transformer models were introduced. Obviously, when transformer models were introduced and when did that paper come out, Attention Is All You Need was 2016, I would imagine that there was a massive, the biggest shift and the main breakthrough when it comes to language technology, when suddenly things that were not possible before became possible. And even if we look at earlier generation of transformer models, I was actually extremely impressed when I looked at, maybe it was not as much GPT-1, but when GPT-2 came out and you could actually see that you have give or take reasoning, give or take rationale. Behind language technology. So I would say rule-based to statistical was a leap, but a major— the biggest leap was actually going into transformer models and going into deep learning.

Totally. That makes sense. It has been a huge shift. And I know when we talked earlier, you talked about having a vertical approach to AI models for translations. I was wondering if you could share a little bit more about what that means and how that might differ from just like general purpose approaches?

First of all, you totally can get translation from a generalized foundational model. There is no problem with them. And I think that's one of the bigger misconceptions of the modern time and day, that you can actually just grab generalized model and get the translation that would totally meet your— I'd probably talk more about the enterprise goals, would meet your enterprise goals. Now, the reality is, if you think about a model that's built as a jack of all trades, right? And the model that can equally, I don't know, design your dinner menu, write code, again, with very degree of success, and translate for you. Obviously, what are you dealing with? You're dealing with a lot of data, trillions or terabytes and terabytes of data. And this data comes from all sorts of sources. So when it comes to one specific task, I think just common sense would suggest that if the model is trained on clean corpus and for a specific task, a purpose-built model, it just has very higher chance to provide much better output, whether it's for translation industry or whether it's for any other industry for that matter. And something is built for purpose or something that can do everything under the sun. So I think that would be my first immediate reaction, although you can get fairly decent translation for your personal needs when it comes to general purpose, generalized models. So that would be my first response. Second, if it comes specifically to translation, if it's a purpose-built model, the way you design the model and the data that— the way you curate the data and the way you pick your data for translation would be dramatically different from the way you would actually approach it when just the model happens to be multilingual and it just happens to be able to perform translation alongside other tasks.

Interesting. So then, how many different models do you use at Smartling for translation? What does that look like?

I think first things first, we still do not discard neural machine translation as one of the engines and one of the mechanisms to get reasonable translation. And again, if you think about it, you train something scraping all of the internet and grabbing everything under the sun, and usually that would be monolingual. Whereas neural machine translation, although same goes for many purpose-built models, is trained on bitext, right? You have your source, you have your target, you have alignment. So that's why neural machine translation, as much as it has its constraints, you still get much higher predictability when it comes to encoder-decoder-based neural machine translation models. First, we do still use— it's a still because we do understand that the shift is inevitable, and at some point neural machine translation as we know it is probably probably going to be a thing of the past, but as of right now, we do use neural machine translation models that we customize and train with our customer, with our client's data or domain-specific data, and this is where we still get the most predictable result. So neural machine translation models, i.e., like your usual suspects, right? Amazon Translate, Google Translate, DeepL, they still have a place in our universe. Now, having said that, we do use various large language models for our translation tasks, and we make sure that we remain agnostic, model agnostic, because different models perform different for different languages and different tasks. So at this point in time, our model portfolio includes all of the models that would be hosted in Google Vertex AI, all of the models that are hosted in Watson X, models that are hosted in Bedrock, models that are hosted in— what else would it be? Azure, OpenAI. So Basically, all the models out there, we have a very robust mechanism of model benchmarking and what a model can do. And we make sure that we're really not only pick best in class, but build enough of our proprietary IP, such as fine-tuning, such as RAG, to make sure that we extract the best from those models. So if I were to count them, our principal engineer who runs our R&D day to day, he actually has this project that he calls 40 Prompts Project. Which basically means that at all, at any point in time, there are roughly 40 different prompts for different models that are used in different stages of our translation pipeline or global content generation pipeline. So that obviously means that, I don't know if you're familiar with the term language ops, lang ops, that now they live, lang ops live alongside MLOps, just have to do with multilingual content management production, and it's not an easy task. To run an ecosystem that can actually invoke in-house models, fine-tuned models, fine-tuned generalized models, RAG, and manage prompts for all of those. So it is actually— it is pretty sophisticated pipeline that we're very proud of alongside, again, very robust benchmarking mechanism.

Wow. Yeah. Very interesting.

And also, like, when, say, Anthropic comes with the next version of Sonnet, because of the way our system is designed, it's fairly easy to swap in the latest version of a specific model when this version of a specific model actually makes more sense for a specific task.

Gotcha. Yeah, super interesting. I had not heard of the language ops role, but it makes sense if you have 40 prompts going throughout the pipeline, you need someone with a fine-tooth comb going through that and managing all of that.

I think actually, honestly, again, if we go back to this misconception or conception of just grab a an LLM API and plug it into your ecosystem. Think about scaling, operationalizing it, how are you going to mitigate latency, and all those things that often people do not think about when they think that just plugging in a model is a trivial task. Let alone, again, when it comes to fine-tuning, when it comes to designing the application layer and all of that. So yeah, but we remain model agnostic and I think we can— why not capitalize on R&D of a huge enterprise when we can, and then we can just build and fine-tune and train around it.

Yeah, definitely. Yeah, I'm thinking about our audience. On our show, we've talked a fair amount about generative AI and about the importance of the inputs and the quality input data. And so I'd love to hear more about how this might be unique for translation and what your data curation looks like for training these models.

I think translation industry, we're absolutely blessed because all we've got at our disposal is human-curated, human-quality, quite often labeled data, right? If you think about the genesis of translation industry and you ask me about translation technology, I should have taken a step back and spoken about things like Levenshtein distance, where translation memory came about, which is basically a way of storing and then calculating fuzzy and exact matches from human-curated corpus. And then we go further into machine translation as a foundational step for translation and then obviously human post-editing on top of machine translation. So we sit on a goldmine of curated, clean, domain-specific, vertical-specific, and quite often enterprise-specific data. So our datasets, and again, like when everybody, we go out there, we go to academic conferences, we speak to people from other industries, everybody's data hungry, right? That's for instance why we see right now, we see, what do we see? We thought that unsupervised learning is going to be, it's going to be the silver bullet. And now you see data labeling companies again being in huge demand. If you talk about companies like Scale AI or TELUS, and I don't know, many other companies in that business, that data labeling and annotation has becoming the thing again. Now, we in translation industry inherently own it. We have it because the output of our process is publishable, human-quality curated content. We didn't do anything to design those datasets. They just designed themselves because of the way translated data is stored in translation platforms like Smartling. It is always parallel. It always has some metadata to it. If nothing else, it would have domain, it would have date of creation. There was not as much now, but in the past there was an editorial layer to it. So one person translates, the other person corrects. What does it give you? It gives you the correction delta. So that alone, you already have translation at a distance there. And then furthermore, LQA, Language Quality Assurance, is a huge thing for us. So in— for many companies to make sure that their deliverables meet their standards, they run LQA programs using different quality estimation metrics. More often than not, it would be multidimensional quality metric or MQM. Now, what does it give you again? It gives you a dataset labeled for quality. We're at that blessed space where our datasets are pre-curated for us. Now, obviously, human error is human error, but for that, there are curation techniques such as deduplication, removing things that look semantically look suspicious, do not pass your semantic checks. Like when the target is suspiciously far away from the meaning. Terminology is probably equally important for our industry as for any other industry. So we have techniques to make sure that terminology is consistent with the domain and consistent with the brand tone and voice. So I guess what I want to say, our datasets inherently are 70% there and 30% is curated through different error mitigation techniques.

Gotcha. That's a nice place to be. I'm sure many listeners will be jealous when it comes to having just like really high-quality labeled data. From the start, do you run into any issues with model bias or anything like that?

Your datasets are good, but your datasets are not perfect. So model can be culturally biased, right? Model can be gender biased. Also, if we look, for instance, what did not quite matter 10 years ago matters now. Things that are getting much more attention, such as gender disambiguation. There is a term, another term that I love, which is model toxicity or translation toxicity when the model produces content that's potentially toxic. So there are a lot of things in the light of a lot of DEI initiatives and in the light of what matters and stop lists and things that you do not call these days what you used to call it before. Obviously, we do run into that. Like, we have one of our favorite— one of our favorite cases was that one of the government officials is a female. But no matter what you would do to the model, just because the model didn't know any better, it would continuously translate her as a male into morphological language, intermorphological language where gender is very explicit. So we obviously had to run some tweaks and make sure, and you cannot just say, no, she's always a female because what if it's a sentence where one person is a male, the other one is a female. So there are things, obviously we do come across model biases. Quite a bit. But again, when we go back to purpose-built or fine-tuned models, quite often because our starting place is parallel, we don't have the issue that the foundational generalized models would have where a lot of training data is disparate and monolingual. So the way— if it does make sense, I hope it does, right?

Yeah, I was gonna ask if you can break down that concept. Of the models being parallel just a little bit more.

Oh, yeah. What is translation? What is— and the concept of translation memory, which is basically previously created translation that can be reused. Inherently, it is a bi-text. Inherently, it is a source and a target. So what it means, if you compare it to the way the generalized foundational models are trained, you have your source and you have your target aligned. Usually in aligned sentences or on a substantival level, or it could be aligned on a segment level, but regardless, you have them conceptually aligned. So the odds of it producing inaccurate, factually irrelevant, or biased output are reduced just by the mere fact that you train on translation pairs. And we do know that most foundational models are— I want to be careful with percentages, but I think it's 60% English predominantly. The average number is 60%, like when you look at the GPT or Gemini on Tropic, just because of the internet being predominantly English, right? And the source is being predominantly English. So that would be the main difference between fetching data from the translation universe and fetching data from all over the world. Just statistically, you get less data. Subsequently, you introduce factual errors, and subsequently, you introduce potential biases. So I guess that's the main difference.

Makes sense. That's super interesting. How do you think that the kind of purpose-built model strategy could transform other industries or could apply outside of translation? Where do you see the purpose-built model strategy winning?

So first, let's say the places, multilingual areas where foundational models would shine, that would be general domain. And general domain, like for instance, opinion portals or retail, user-generated content, which is inherently noisy. It can speak about everything and anything under the sun. If you look at Reddit, for instance, I would imagine that there is nothing better for Reddit than a generalized model because God knows what people will be talking about tomorrow. You cannot fine-tune for that. There is just no way you can do it. Or if you look at retail, for instance, especially when it comes to marketplaces, people-driven marketplaces, individuals-driven marketplaces, you cannot quite train for something that you don't know. Right? You don't know what's coming tomorrow. You don't know. Or patents, for instance. Yeah, you can do things like structurally breaking down a patent into pieces and take different translation, different processing approaches for those. But at the end of the day, patents can deal with everything from Crocs shoes to, I don't know, pick something, pharma. So there are areas that are so broad where foundational models will do just great, but I believe that there isn't an industry outside of more general content and outside of more general processes that would not benefit from purpose-built models. If you think about it, like you take construction, you immediately, and actually have a friend who runs a startup, uh, that has to do with process engineering, process engineers. So you need to be able to parse out diagrams, CAD drawings. Flowcharts, workflows, and everything under the sun. So if you have that data, and there is another parameter to it, if you have that data and actually you were able to train, train your models on that data, it's going to perform much better in your process engineering space. Also, the way you would design, actually, you would design your model itself, you can make it smaller, you can make it faster, you do not deal with latency. So I'm just using process engineering as an example, but if you think about it, Every industry would benefit from it, like life sciences. The risk— I would actually, let me put it this way. The more risk-adverse an industry or a vertical is, the more it will benefit from purpose-built models that are designed specifically for that. The model design itself actually caters specifically to that domain, and also the training data is very focused, curated, and narrow. For that domain. So more general, less space for purpose-built models, more risk-adverse and more focused, more domain-specific, more space for transformative space for purpose-built models.

That makes a lot of sense. If you're an industry with, all industries have like their own unique formats, like the drawings or the language. And if you're risk-adverse, you want to make sure all of that is is handled super specifically, like up to your standards.

Actually, there is another dimension to it. One, like I said, CAD drawings, and then I thought about legal space, for instance, right? And legal space, we still see, especially in the eDiscovery space, we see a lot of need for OCR and reconstruction of something that could be a photograph, could have been handwritten. And just for entertainment's sake, I don't want to name any specific models, publicly available models out there, but I would say grab any of them, run a simple test, take a PDF, not parsable, a flat PDF, feed it in, try to get it understood, and even less so try to get it reconstructed. And you will see that the foundational model that does not have it as a specific task is extremely unlikely, not just to shine, but to deliver. So there is one other thing. It's not just the context. It is not just the model design. It's not just the training data, but it's also the ability to handle formats that are specific for that particular domain. And they're dramatically different. Life sciences, formulas, right? Clinical studies, reports, fintech, shareholder reports. They all come with their own can of worms that generalized models just cannot solve, and fine-tuning or other techniques would not solve either.

Yeah, 100%. I mean, that does make me think even to an episode we did recently where we were talking about clinical trials and the way that some of them were planned and prepared in Word documents and just the unique challenges of copying and pasting these Word documents and building these trials. And they had some unique formats that they had input and just some unique challenges around that. So that makes a lot of sense. It's really interesting to think about. Yeah, my next question is like shifting gears a little bit, but I'm curious, a lot of times there can be pushback on introducing AI into like a traditionally human-centered industry. So, you know, you mentioned you have all of the, all of the data that was human translated and that's great data to serve, but how do you build trust with your clients and manage that moving from the human-centered into using AI for those processes?

I think actually the challenge is complete opposite. How do you convince— Oh, okay. No, I think again, I mean, there are pockets and we just spoke about life sciences, for instance, and regulated industries. There, indeed, building trust is, that's a major, I wouldn't say challenge, but it's a process, right? You need to run certain number of pilots. You need to deliver outcomes, satisfactory outcomes. You need to pick content types where you feel safe. And where you don't introduce as much of a risk. So there will always be industries where you cannot just plug in AI without building up trust through testing, through benchmarking, through piloting. But the problem that we're dealing with right now, and I don't think it's translation, I think it's the world, is this belief that, hey, just plug in AI. There— I'm yet to meet a customer where there isn't an executive mandate to use AI for their processes and potentially it's equally innovation, cost cuts, ROI. There are multiple reasons why companies are mandated to use, to implement AI. So I think it's actually the other way around. I think it's really going in showing that yes, AI is fantastic. It's gonna take you this far, but you need human in the loop at least for for a certain portion of scenarios. So I don't think we're selling AI versus human. It's more, hey, let's remember that human in the loop still has a role. So I think it's actually the other way around, but otherwise, maybe we can talk about the workforce and translators for a little bit. I mentioned translation memories, right, which was basically a translator suddenly does not get paid for the whole new word. The translator gets paid only for a portion of it because some of the content comes from translation memory. There was massive pushback because translators believed that like, hey, you know, we're cutting into our wages. Then the second wave of pushback from translators was when machine translation was introduced into the workflows, and there was a conversation about, hey, it's cutting into our creativity. Again, we are making less money. We feel that we're just an appendix to a machine. So there was a pushback there, and then there was a third. I think now with AI, I think it's much less pushback. But I think then translators started discovering that, hey, I'm actually making as much money, if not more, because my productivity goes up. So suddenly into my 8 hours, I can fit much more content and much more work. So there were two processes of selling innovation. Let's go with AI-driven innovation. First, you sell it to the workforce, and then once you are convinced that you can deliver at the same quality level, then you actually go and then you build the trust with your enterprise customers. Trust both ways, right? AI only or AI with human in the loop.

[Speaker:OLGA Beregovaya] So, I think a good place to end would be to talk about how you stay up to date when there are new AI models coming out constantly. And I'm sure like our audience can relate to that too. Like the second that you've got one model in another, the next version comes out or the new one comes out. So, how do you stay ahead in integrating new models?

I think the real question is how do you still stay sane?

First, you stay sane.

And I'm not sure, for all I know, I could have crossed that line long ago. No, but it is true, right? You wake to a new model every day. And I think, first of all, it is very important to follow what's happening out there. It is very important to be subscribed to newsletters and follow your LinkedIn feeds and follow the podcasts. How can you be in the know without being in the know? And again, that's where AI comes to your rescue. You can read summaries. You can read digests for what came out that day. So I think, first of all, it's very important to see what's going on. Second, I think it's important to understand the patterns and understand, for instance, like how much of a breakthrough was previous version with the next version, for the next version to the next version. So you don't fall for the frenzy and you don't freak out because just a new model or a new approach. Or a new, I don't know, whatever open source initiative came about. I think at some point you just build the understanding of what kind of a breakthrough you can possibly expect and not run with the next shiny object, but say, hey, we're at a good place right now. Let us see, let us like, let's read the model card. Let's run a couple of experiments and see if it's even worth considering updating to or implementing, or is it not contributing much? Again, I don't want to throw any names out there, but recently there was a huge splash in the news about the latest model release, and when we tested it for translation purposes, like, yeah, you know, there is no massive breakthrough. This model was built for other reasons. So be in the know. I will go back to it. Model testing frameworks is an industry of its own. If I go back to SmartLink, I think fast and robust way of testing, be it a model, be it a new corpus, be it whatever the tweak to an existing model, or be it a new way of fine-tuning, as long as you have a good, healthy, automated framework in place, it's going to take you a day to know how much you can or cannot get out of this particular approach or this particular model. So method, it's all not fall for the frenzy, not panic, not freak out, be appraised and have very healthy testing practices in place.

Yeah, yeah, and testing for your use cases, your industry. Maybe it's a breakthrough for someone else, but it's not a breakthrough for you. So you don't get too caught up in the hype.

Yeah, yeah, not getting— and I think that's also where we help our customers, our enterprise partners, because you do need— I mean, you can have somebody, an executive who says like, you know, I read a good thing about Google Gemini or I don't know, Anthropic Claude 4.0. And this is, I think, where companies like us, companies like Smartling would come in and say like, hey, we'll take on the burden of the testing. For you. We will actually tell you what's best or whether it could possibly make sense for you to build your own or some other approach.

Makes sense. Well, yeah, thank you so much for coming on the show today and for sharing your experience. It's been really interesting for me to learn more about the field of translation and just model best practices. So thank you. Okay.

Thanks so much for having me.

Thanks for listening. To learn more about topics mentioned in today's episode, head over to our show notes on alteryx.com/podcast. If you like this episode, leave us a review. See you next time.