The Human Judgment Behind AI

July 15, 2026 00:22:52
The Human Judgment Behind AI
LangTalent Podcast
The Human Judgment Behind AI

Jul 15 2026 | 00:22:52

/

Hosted By

Eddie Arrieta

Show Notes

There is a common misconception that as AI advances, multilingual professionals will inevitably become less relevant. In this episode of LangTalent, MK Blake, VP of Delivery Services at Welo Data, flips that narrative on its head. She argues that linguists are not just supporting the "last mile" of localization—they are absolutely foundational to the "first mile" of AI training.

MK breaks down why human judgment is critical for teaching models how to interpret meaning across cultures. We explore why unique human skills like "pragmatic competence" and navigating ambiguity are more valuable than ever.

View Full Transcript

Episode Transcript

[00:00:03] Speaker A: Lang Talent is supported by Global Search RC Translating recruitment into results. [00:00:09] Speaker B: Welcome to Lang Talent, the podcast by Multilingual Media, exploring the human side of the language industry and the future of work. I'm Eddie Arrieta, CEO at Multilingual Media. Today's conversation looks at one of the biggest misconceptions surrounding AI and language work, the idea that multilingual professionals are becoming less relevant as automation advances. Our guest is M.K. blake, VP of Delivery Services at Willow Data, who works at the center of large scale AI training and evaluation programs. Her perspective challenges the common narrative that AI is replacing linguists. Instead, she argues that multilingual professionals are becoming increasingly essential to how AI systems interpret, interpret meaning, evaluate quality and operate across cultures and languages. Mk, welcome to the show. [00:01:06] Speaker A: Thanks for having me. [00:01:08] Speaker B: And of course, mk, this is, as I said at the backstage, one of the most anticipated but also popular conversations within the multilingual ecosystem. I should tell you as well that many has avoided, many professionals have avoided having these conversations and we understand it's not an easy conversation to have. So we really happy to have you with us today. [00:01:32] Speaker A: Great, great. Excited for the conversation. [00:01:34] Speaker B: And like we said in the introduction, you've argued that multilingual contributors are foundational to how AI systems function, not just support roles around the ages, like you'd say. But what does this look like in practice? [00:01:48] Speaker A: Yeah, so a lot of people talk about sort of the last mile of AI. How can we localize, how can we translate, how can we make the model work for that language? And I think that's the wrong, that's sort of the wrong mental model. What we really want to think about is sort of the first mile of AI. So when a model is learning a good response, whether that's in French or Spanish or Urdu, they really need to come with that from the user so that when a user is actually interacting with a model, they have the correct response for them if they're in Osaka or if they're in Toronto. That judgment and what the model sounds like comes from a human. And that's embedded right from the beginning and embedded in the training. And so that's not really a support role. That's something that we need from multilingual professionals right at the beginning of our model training. [00:02:36] Speaker B: And of course this is going to relate to the abilities, the skills and what people have been trained for. In multilingual AI work. There is often a huge amount of nuance and contextualization making involved. Like you are suggesting, what are some of the human skills that still matter most in These workflows? [00:02:56] Speaker A: Yeah, absolutely. So there's two main skills that we see here for our multilingual professionals, which is pragmatic competence and then navigating ambiguity. So what do I mean by pragmatic confidence? That's kind of a silly phrase. But, you know, when we do, when people do localization or translation, they're thinking about how does this sentence makes sense in terms of meaning, in terms of tone. And that becomes even more important when you're doing AI data because you're trying to understand what is this user saying, sometimes across text, sometimes across multimodal, across actions. And you need to be able to make those judgment calls and say, oh, this person's being sarcastic or this person's being funny, or this is something that they said, you know, five steps earlier in the conversation. You need to be able to keep that in your head while you're working through sort of the training data. And then the other skill that's a little bit almost the opposite, which is navigating ambiguity. So a lot of the times these raters or multilingual professionals will get huge guidelines that tell them some of the information that they need to know in order to create all of the judgments that they are putting into the system. But they're doing thousands of judgments a day. So they have to take these guidelines, create their own mental model with sort of their own circle of the universe, saying, when I look at this, this is within the circle or this is without the circle. And as they go through thousands of judgments, they need to keep that in their head and understand there's going to be edge cases. This is black, is this white, is this gray? They need to be able to understand that when they're doing that. And so they have to have that pragmatic confidence, which is just knowing what, knowing what people are saying, knowing if they're being funny. And then they have to have that ability to navigate that ambiguity and understand, here's what I'm going to say. I'm going to do it consistently. I'm going to keep it within the sphere of understanding that I've created, and [00:04:48] Speaker B: thank you for sharing it that way. This is going to be great for our team to take it and share it on social media as a bite size. Insight from NK Blake. So we thank once again our listeners to the Lang Talent Podcast and members of the multilingual community for listening today. What kinds of skills and capabilities, of course, are now increasing in value that perhaps were not as valuable before AI systems were scaling globally. Today, it's almost like, out of the question. I think two, three years ago, I would say three years ago, we're still thinking, is this gonna be as revolutionary as people think? Should we be as afraid as everyone was? But what we're realizing is that we, of course, are still needed. That's very obvious. But what's becoming more valuable in this new era of AI? [00:05:40] Speaker A: Yeah, absolutely. So one of the really interesting things about AI is that at the early days we were sort of gobbling up information, right, Trying to get a base foundation layer for these models. And now it's becoming a lot more specialized and really specific and trying to get to both consumer and enterprise use cases. So if you're a multilingual professional, my advice is to lean into your domain expertise. And not everyone's going to be a biologist, not everyone's going to be an oncologist and be able to speak Portuguese and tell me all about cancer treatment and diagnosis. That's not always the most important thing. So, for example, right now the World cup is going on. Let's say you are the preeminent expert, expert in the Senegal team for the World cup. And you know a ton about that. And you can speak French. That's going to be a hugely valuable skill because there's going to be users who want to know about Senegal, they're going to want to know about the World cup and they need to be able to help our models be trained in both how to talk in French, but also what's the domain expertise of football? How are we going to talk about the team? What's the most timely topics? And so an evaluator who's really able to sell themselves both in their language skills, but also that domain expertise, whatever that is, is going to be someone who's going to be really successful in this field. [00:06:58] Speaker B: MK and of course, I'll dig a little bit into more of these questions, but as VP of Delivery Services, you lean on your team to make sure that these services are in fact delivered. In your internal conversations about artificial intelligence, what's been the reception on this evolution on artificial intelligence and technologies? Is the team bullish on it? Is the team excited? What do you see in the company? [00:07:30] Speaker A: Yeah, the team's really excited. I mean, one of the things that they're really interested in is we're getting into areas where people need to get their chain of thought. So AI systems are moving into the agentic world where they're taking complex multi step processes. They are imitating one of us. Right, like how does Eddie do a podcast? What does he do when he's doing the podcast, what's he doing behind the scenes? And so those agents are trying to figure out what are the little subjective judgments that you are making when you're actually performing work or when you're performing tasks or when you're bringing your expertise to the table. And so the team gets really excited to see when we get. When we work with professionals who can really articulate that when they come in and they can bring their writing skills, their reasoning skills, and tell us exactly why. Why did you make the, you know, the judgment to cut the podcast right there? Why did you make the judgment to ask this specific question? And being able to find someone who can get that subjectivity and get it out on paper, that's been really exciting to see. And we get to work with professionals in a really different way than we were before. [00:08:38] Speaker B: And I'm really glad you put it that way, because we still also believe that companies are underestimating the complexity of multilingual AI systems and the expertise required to support them. From what you are seeing, where do you think companies are missing the mark, to understand that they really need those excited team members, linguists that are, you know, working, working with you, MK in there. What do you think companies are missing the mark? [00:09:05] Speaker A: Yeah, we still get a lot of requests that are like, we need Arabic, we need Spanish, we need Mandarin, and our first question is, what dialect do you need? You know, where. Where are you actually bringing this product? Like, who are you talking to? Because you're not going to necessarily want Arabic from Morocco if you're bring your product to market in Saudi Arabia. Similarly, when you work with a Spanish speaker, what they say in Mexico City is not the same as in Buenos Aires. It's just not the same. And if you train your model on the wrong dialect and the wrong culture, then that product's not going to be able to speak to those users. And so it's a really exciting opportunity for linguists from those areas, from really specific places to bring their expertise into the models. And one thing we get wrong as companies, and I think a lot of language companies have been trying to enter the AI data space, is just to assume that someone who's a really brilliant multilingual professional is going to be good at this work, because there are things that you can do as a multilingual professional to make yourself better at this work. And we've talked about a few of them, which are, you know, understanding that it's going to be a little bit more ambiguous than maybe the type of work that you're used to, you're going to have to probably do a little bit more explanation of your chain of thought, of your reasoning of why you got to the answer. Because you have to remember that if you're imparting your cultural expertise, a lot of the engineers, a lot of the product managers who are looking at this data, they don't know anything about your culture, they don't know anything necessarily about your domain. And so they need to see why did you make the choices that you made? So they can make the model better at actually doing those things. So we see that mistake happen a lot of the time with some of the folks that come and work in this space. But we're really trying to overcome that with a lot of training and kind of educating our multilingual users on what are the skills they need. [00:11:01] Speaker B: And Emeke, I'm really grateful that you are taking it to this level of nuance because given the rise of artificial intelligence, it seemed, at least from what we are observing and the interviews and conversations that we are having, that we deviated a little bit from that localization conversation, that idea that we do need to care about the specific locales. And it seems like internally at Willow, this is not the case, that you are really thinking about this and that this is part of the top priorities as you deliver your services. Is this the case? [00:11:36] Speaker A: Yeah, exactly. We really go through a strong assessment process to truly understand what a multilingual professional is good at. We don't just look at one type of dialect. We, we go through what is the exact sort of region you're looking for, what's the exact specialization you're looking for? How can we help your product be successful? And so when we talk to our clients, we ask them, you know, where is this product going to market, where, where are you thinking about implementing this workflow? Is this an agent for a customer service, you know, workflow in Germany, or is this a consumer product that's going to be widely available in the US Those are going to be very different tones and very different ways that you train a model. [00:12:18] Speaker B: This is wonderful because this is directly related to the services that you provide. So if I may ask, Willow Data works, of course, across large scale AI training and evaluation programs. And you are also involved with WillowWorks. Can you talk about the types of services and opportunities Willow provides for multilingual professionals entering AI related work? [00:12:40] Speaker A: Absolutely. So Willow Data works with most of the major labs. We do training data, we do text, we do multimodal, we do robotics, we do increasingly Agentic work. So we work across a wide spectrum of places where multilingual is coming strongly into play, including, you know, that physical AI, that next step of robotics and getting, getting those multilingual professionals working in those spaces. One of the things I think I touched on earlier was it's not just about the work that Wheelo data is getting from their clients, it's about our platform. So we have Wheelo Works, which I'm really excited about. That's our platform for multilingual professionals to come in, find opportunities. And one of the really exciting things is that it offers that training that I was talking about a little bit earlier, which is it gives you a sense of what are AI models doing now. So there's basic training courses on how to interact with AI models, how to prompt them, how do they typically respond, how do they function, how so that you can come and upskill yourself and be ready to enter a project where that knowledge of not just your domain and your language, but also how does AI function really comes into play and can make you more successful on a program. [00:13:55] Speaker B: Mk, if you can indulge me today, someone might have asked internally, how does someone like MK ends up as VP of Delivery Services? How did that happen? You didn't wake up one day in college and be like, you know what? I'm going to be VP of Delivery Services. How, how did it happen? How did you get to where you are? [00:14:16] Speaker A: Yeah, I actually started as a raider myself. I was home with my, my two children when they were very young and I was looking for an opportunity. This was felt like, feels like a million years ago before, before AI models kind of took over the world. And I saw an opportunity to, you know, start labeling data and thinking about it. And I, you know, I had a background in law, a background in history, and I said, oh, I have something to contribute, like, let me see what this is all about. And started doing rating at a, at Appen and then worked on that for a long time. Really fell in love with it. Really loved the idea of how do people think about judgment, how do people work with models? How do we impart sort of human, human knowledge into some of these systems to make them successful for others and worked my way up through operations. I spent five years at Amazon working on their customer service models and their help center bots there and then at Google working on some of the first generation Gemini products as well as some of Nano Banana and all those fun name products there. And I've been over here at Wheelo for about a year and A half running our operations and delivery teams. And it's been, it's been a great journey and I'm really passionate about bringing jobs to people who maybe are looking for something on the side or looking for something to do full time. This is really interesting work and requires a lot of thought process, a lot of different skills, a lot of exciting opportunities to write, to read, to reason. And I love providing those opportunities to folks across the industry. [00:16:00] Speaker B: Thank you for indulging me. And of course, I'm very curious always about the values, the principles that drive the professional for umk, what drives you as a professional. That, of course, I'm looking to also understand that aligns with, with Willow as a company and helps you be successful [00:16:20] Speaker A: at what you do. Yeah, absolutely. I mean, I, I really love the idea that AI models will be representative of cultures across the world and I think that that really aligns with Willow Data's principles and mission. I would not want to work with an AI model that's all the same, like, you know, all west coast tech people. Like, it just responds to me exactly like that. I've had enough of that in my life, for sure. I get that every day. I don't want the models I'm interacting with to necessarily reflect that. I want them to reflect, you know, the wonderful cultures that we have here across the world. You know, make it accessible, make it reasonable, make it something that is safe to use, something that people enjoy using and is reflecting to them and also helps them get where they're going. It's an incredibly powerful tool. I, I wasn't always sure that it was something that I was going to adopt and now I use it in my everyday life because it's incredibly helpful. And I think there's tons of opportunity as we get into more specialized spaces to give people accessibility to legal healthcare. Really incredible things within their countries and within their cultures. [00:17:31] Speaker B: And that's wonderful to hear because it can help us understand a little bit where you also see opportunities. You've mentioned healthcare, you've mentioned access. Where do you think the work that you're doing with Willow is going to be much more impactful? [00:17:45] Speaker A: I think the physical AI is sort of the next big thing and that's going to be really interesting to see how that shapes from a multilingual perspective. We have a lot of folks who work in the healthcare industry who aren't necessarily doctors, but are, who are healthcare professionals, who are care assistants or who are working, you know, with older populations or who are working with people who are in hospice and these Physical robots are going to make a huge difference for people in their lives. And we're going to have to figure out how to how they work with those groups effectively, sort of what those interactions are. And there's going to be a ton of work in that space, both from a language perspective, but also just how do you perform tasks? What are the tasks that you perform? And I'm really excited about that space. [00:18:36] Speaker B: We are very excited. It seems also that humans are really great at adapting and the world today looks very different from what it looked like two, three years ago. And that's very exciting. Looking into those opportunities, MK and from your experience, for language professionals listening today, what new career paths are emerging that perhaps didn't exist a few years ago? What opportunities do you see for professionals out there? [00:19:02] Speaker A: Absolutely. There's two major ones that have been coming up and I touched on them a little bit. But one is cultural safety reviewers. So I can't pretend to know what is sort of all of the cultural harms that could happen for any model in a country. But if you're planning a wedding in the US or if you're planning a wedding in Turkey or you're planning a wedding in Iran, those are all going to be really different approaches. And you want the model to reflect cultural sensitivities. When you're asking about, you know, what is, you know, do I need to go and speak to my family, do I need to, you know, rent a venue? All of those things are really specific to those individual countries and culture. And that's not about harm, that's just about making sure that your model is reflective of the culture and sensitive to the culture. So that's a big one that we're seeing come into play because these big model builders don't want to see their model saying something culturally insensitive at any time, especially if they're bringing them to enterprise customers. The second one I'm excited about, which plays into physical AI, but is also something that we're seeing across the models, which is Agentix Systems tester. So we talked a little bit about this, which is we need people to be able to tell us what are the reasons that they made choices or made judgments or made labels or chose this action. What are the little subjective things that happened in your mind that you made a choice to send that email on a Friday at 4:45, whatever it is, we need people who are able to tell us exactly what's going on in terms of their subjective judgments. And that's that agentic systems tester. It's almost like a red teaming person who's going in and checking for safety. They're going in and seeing what happened at each of these steps and can I articulate why it happened? And those are both really exciting careers right now that are coming up a lot and I think are going to be critical for multilingual professionals. [00:20:57] Speaker B: And mk, you've been very generous your insights with your time. Before we go, any final messages for our audience and for those listening from your perspective, let's say if you're a [00:21:11] Speaker A: multilingual professional, you know, go out there and seek new opportunities beyond localization and translation. Those are exciting fields, but there's even more you can do. And I think there's so much talent out there in the multilingual community in terms of what are you passionate about beyond language? Bring that passion. Don't discount your own domain expertise. If you love shopping, if you love travel, those are things that are really needed in this space and so highlight those for yourself. Really advocate for yourself for those special skills that you have and show how being a multilingual professional plus a domain expert really brings us to the next generation of AI models. [00:21:50] Speaker B: That is great, mk, thank you so much for talking to us today. [00:21:54] Speaker A: Absolutely. Thank you so much, Adi. [00:21:56] Speaker B: All right. And for all of those of you listening, thank you. This was Lang Talent. Again, a big thank you to MK Blake for helping us rethink the role of multilingual professionals in the AI era. Not as workers being replaced, but as experts shaping how the system functions across languages and cultures as AI continues to scale globally. Conversations like this remind us that technology alone cannot replace judgment, context or human understanding. Catch new episodes of Lang Talent on Spotify, Apple Podcasts and YouTube. Subscribe, rate and share so others can find the show. I'm Eddie Arrieta with Multilingual Media. Thanks for listening and we'll see you next time. MK Goodbye [00:22:47] Speaker A: Land Talent is supported by Global Search rc Translating recruitment into results.

Other Episodes

Episode

May 13, 2026 00:27:02
Episode Cover

Empowering The Invisible AI Workforce

Behind every sophisticated AI model is an invisible human workforce. In this episode, we explore who actually gets to be part of that workforce...

Listen

Episode

October 08, 2025 00:33:38
Episode Cover

From Localization to Global Experiences

What happens when you retire the “L word”? In this episode of Lang Talent, we sit down with José Palomares, Director of Global Experiences...

Listen

Episode 4

December 02, 2024 01:07:26
Episode Cover

Fostering Well-Being in Remote Language Work

As remote work becomes the norm, maintaining well-being is crucial for language professionals. In this panel, we’ll address the unique challenges and opportunities of...

Listen