What can we learn about language processing from computers? | LIVE English Show

Hello. Hello, everyone. Welcome to the live English show. It is Thursday and today it's the live English show. How are you all doing? So happy that you're here. And I'm going to wait for everyone to join in. So when you are here, let me know in the comments and tell me where you are from. And welcome. So today we have an exciting conversation for you because I know many of you are going to be so intrigued, but by what we have to share. And I have a good friend and a colleague who's going to be speaking to us and talking about artificial intelligence, language processing, and language learning. And it is going to be so incredibly good. Also, a lot of hot tips about pronunciation. Okay, so we have Manas.

Hello.

And Salma from Spain. Hi there. Diego from Ecuador. So good to see you. Valentina is here from Argentina. And Yusuf from Paris. Hello, everyone. Say hi in the comments. So everyone knows you get comfortable, take off your shoes, get something to drink, coffee, tea, a glass of wine. Because we're going to be here in the neck for the next hour or so and. And that's it. So I don't want to wait a lot before we get into the conversation. So please allow me to introduce Yazzie, who is an English teacher and she's also a linguist and she works with artificial intelligence. And today she's going to share us, share a little bit about the work that she does. So let's bring her on. Hello, Yazi, how are you doing?

Hi. I'm very happy to be with you here today. Thank you for inviting me.

I'm super excited to have you here. We've known each other for a while and I've been actually waiting for a good opportunity to let you talk to the beautiful people here and share all your beautiful knowledge and everything that you have to share. How are you doing today?

I'm doing fine. I was actually waiting for this conversation a long time ago because I've been following you way longer than you think. And I think that by talking to you, maybe I can also talk to other pronunciation coaches, other people who teach English and who feel that they are very insecure about teaching pronunciation because, you know, it's. It's a difficult topic. It's a difficult thing to do. So maybe I can provide some tools. And while I'm talking to you, I will also talk through you to other people out there directly.

You can speak directly to all the people, the teachers and the learners trying to make sense of teaching pronunciation in a simple way. And since we have such incredible tools out there to help us. There's no reason why they shouldn't know about this and use it in their teaching. So if you're a teacher, you should definitely stick around to learn more about what. What we have in store for you. So tell us a little bit about yourself.

Well, I'm Mexican. I've been a teacher for almost 20 years. I never want to say that because people start guessing my age. So, yeah, I started when I was 10 years old, of course.

Of course.

And then three, actually. Three, you know, three years old. Yeah, of course. And yeah, after being a teacher and, you know, getting certifications and being trained and an BA and then a master's in teaching English or teaching with technology, I thought that I needed something else. I didn't think that it was enough. And I wanted to know what it was like to learn a language when you travel to another country. So I decided to travel to Germany and another m. Yeah, I know it was a drastic change because I didn't speak any German at all. And yeah, you know, like those cliches that people go like, hey, if you travel to the other country where people speak the language, you're gonna learn it the best. And that's not always the case, by the way, so. And yeah, I love all those beliefs.

Like, you know, if you do this, then that. It's like, no, doesn't work for me.

Yeah, that's not always the case. And then I was, as I was in Germany, I had this class about pronunciation instruction and everything. And I. I learned that there's a lot of ple. Chase out there. If you are not a native speaker, maybe you're not trained, or, or you don't have the gift, or you don't have the enough experience and knowledge to teach pronunciation. And the first thought that came to my mind was like, okay, maybe I cannot do it, but will I be able to bring a native speaker to my small community in this remote places in the world where they have. They don't have this lifestyle, they don't have the things they're used to, Will they actually be willing to come and teach pronunciation to my students? And then I thought, hey, technology is out there for me. Maybe I cannot do it, but technology will help me. So I conducted a series of experiments and research throughout my MA Working with automatic speech recognition and artificial intelligence. And I found, well, it was two year experience, and I've been working with it after that for four years. So I've been able to observe the behavior of computers and the behavior of learners. And I think that these insights are going to be helpful for many people. Also, after. When I was in Germany, I was also a part of the linguistic laboratory of the university. So I had the chance to help other researchers, PhDs, MA students with their linguistic data. And, you know, the more you're in contact with it, the more insights you get and the more you make relationships between the phenomenon and. Yeah, that's. That's why I came from being a teacher to being very interested in what computers can do to help teachers. Right. And then jumping into artificial intelligence. And now my job as a linguist also has to. Well, it comes from there.

Yeah. It's like it's such a. You know, having all that knowledge is so. It's like having superpowers because, you know, all aspects. It's not just like the intuitive human part of pronunciation, but you also understand, first of all, like, the tech, like, how technology can help, which is amazing. Like, if you have. And not a lot of people have access to it, like we said at the beginning. But I think that the research really gave you a lot of insights as to how the brain works. Would you say that that's true?

Yeah. Well, just sitting there in the laboratory and talking to the people, my peers or my teachers, who had a bunch of data that they collected and they wanted to analyze, like, even just sitting down in there and helping them, learning what they wanted me to do. I'm not an expert, and I have never said I'm an expert at anything, but I've been with experts and I know what they look for. And sometimes you don't even realize there are things that you have to look for when you're talking about languages. So there are so many things out there that have to do with language and linguistic analysis that we are not aware of and we just take for granted or. Yes. We have, like, we have the feeling that something's going on, but we don't have. Or we don't have the data to prove that it is right or wrong.

Yeah.

So just being there, it was an amazing experience. I fell in love with research.

I can imagine. I can imagine. Even though it requires a lot of attention to detail, I would say. Yeah.

Like, and every time people are at the lab, you. I also had some wiping napkins in there because, yeah, people get really frustrated and. Yeah, I wasn't there. I've been there as well, so I know it's amazing. I respect research because it's a hard process. It's a really.

Yeah, absolutely. And it seems like an incredible experience to live abroad and to kind of like immerse yourself in a different culture, working and, and communicating in English, just like, on a personal level, I. I'm assuming that that was really worthwhile as well.

Yeah. And, well, before being to Germany, I had the opportunity to travel to the US Also to get the teacher certification as well. But, yeah, Europe is a different thing. People in Europe speak so many languages, and it's hard to find someone who speaks only one language. So when you arrive in Europe and you go like, hey, I speak two languages. That's my superpower.

Exactly. Seriously, who cares? Yeah, I speak five. So.

Yeah, and the ones you speak are so generic, Everyone can speak themselves. What's the new thing you bring to the table? And yeah, it was very eye opening. And also it gave me a lot of confidence because before being to Europe, I thought that my English was not intelligible enough. Maybe I still had to work on that, but then when I was in Europe, I met people from all over the world, and just finding out that all of them understood what I wanted to say was enough.

Right.

And I was happy. I was like, I'm ready to go.

I love that. So let's begin by, you know, the first question. What is artificial intelligence?

Yeah, well, there's. There are many definitions out there, but I will try to give you, like, the simplest one that everyone likes and that everyone understands. Artificial intelligence is the process in which humans can make machines behave like humans. I mean, that's a very broad one. Right. But if, for example, if you have a camera and you should and, you know, shoot a picture and you take a picture, the camera will do processes that humans don't do. Right. I don't. I adjust my eyeball to. To focus or anything. I mean, it just does it. My eye does it.

Luckily, luckily, we don't have to do that every day. We look, wait, let me adjust it.

I have to wear my glasses. So maybe, maybe cameras can do it better than I. But anyway, so when a camera can recognize shapes and focus and do what we do naturally, that's when we call it artificial intelligence.

Right, I see.

And then. Yeah, and then artificial intelligence intends to give the machines the processes or provide all the steps so that machines can do things that humans do, for example, see, speak here. And they are working on touch and smell.

Wow.

They cannot work. And still they have a lot of trouble working with taste because machines just don't care. But, but, yeah, yeah. There's so many aspects of the human nature that are trying to be mimicked by artificial intelligence. Now that, yeah, that's what makes it so amazing. It's like, how do you have a machine do what you do that. The thing is that the combination of all these processes is what makes it more interesting. So it's not the same to just have the camera like you. You're an Instagram, and Instagram can just recognize your features and know where your eyes are, and then, you know, apply a filter. But then do that. Do. Do that and still answer or say your name, recognize your face, and then say, hi, Hadar. That's like, oh, God, that's what. It makes you feel bad. Oh, this is impressive. Like, the camera is not only recognizing my features, but it's also saying my name. Because that's something we do, right?

Yeah.

When I see you, I say, hey, hi there.

So a lot of people have like this fear, maybe an unexplained fear around artificial intelligence, where, okay, the robots are going to take over the world or our job. And, you know, can you, can you speak to that a little bit to that? And why do people, I would say, resist like the. If, if you, if you've seen that in your field, right? Whether it's in Langu or artificial intelligence in general. And. And what you think about that?

Yeah, well, yes, there, there are so many people out there who feel that first, their privacy is not being respected by AI, and they also fear that they are giving away so much information that AI can do something against them. And also the other people who feel that AI is going to get to a stage where it's going to be so intelligent that it's going to be way more efficient than human beings replacing.

I mean, we already have that. If we compare it to a few, like 50 years ago.

Yeah, exactly. So there are international organizations right now working against that. So, for example, in Europe, there are very strict regulations on the use of private, private data. So companies cannot just go and use whatever they want. You have to give them access. But it's not only the user, because sometimes we users don't know what we are actually giving access.

Absolutely.

Yeah, yeah. So the institutions, organizations are looking after all rights, and they're going like, no, you're not using this. By no means. Not even if the user says, yes, you're not this and that. And also, artificial intelligence hasn't reached the level to be dangerous yet. What artificial intelligence does right now, which I think is the matter, is that it reflects the actual world, so it amplifies what is already there. So there were so many people, so many, like, years ago, Thinking of. Yeah, but Google Translation will translate bad words or chatbots. They will say bad words at you. And it's like, it's not the chatbot, because the chatbot doesn't know what is uttering or what is writing or typing, but it's just a reflection of all the data that is out there online for free, all that, you know, the banks of data. So the machine just looks for the most frequent sequences, the most common, the frequent patterns, and it will bring you that. So you can either use it to your advantage, but you can. We can also see that there's a lot of inconsistencies, but it's just reflecting the world.

It says so much about us, you know, and about. Yeah, yeah, yeah, yeah. Wow, that's really. That's really cool. I never thought about that. So talk to me about artificial intelligence and language processing and learning. So how do you. How does that work? I know that this is what you do, so I know you can't talk a lot about what you do, but a little bit about how it works. Like, how does. How do you teach a computer to understand a language?

Okay, so as I was telling you, we are trying to teach computers how to hear and how to speak. Okay, so you are a pronunciation coach, and I'm going to test you today.

Yeah. So you have a little presentation also.

Prepared for us for that. Yes, but just like in a general view. Yeah. So computers can explain, extract sound, decode it, get the gist, codify. Like, get like, what. What is this person asking for? So they can get the keywords. Right. And then they look for those keywords in banks of dictionaries that are built by linguists and computer engineers. And then computers grab the meaning from there. And then the engineers and the linguists told the computer, hey, when. When you grab this, reply this. So then the computer will look for the answer and try to generate an answer to give you. Like, either even if it can be an action, if you ask a virtual assistant to do something for you, or it can be a reply just in a conversational artificial intelligence. So there are different paths, like, of what language is used for, if it's for you to give directions to a virtual assistant or to have a conversation. So there are different branches in there. But what linguists do with computer engineers is to do that, like codify all the banks of data and annotate and go like, okay, this is a noun, this is a verb. The person is asking you for a thing. The person is asking you for information. So we find which words From a language convey, asking for something. Oh, okay, all these, this, this and that. And then linguists give all this data to the computer scientists and they go, they code it in such a way that the computer can retrieve it when, whenever it's necessary and just bring it to you and display or do whatever they want. So it's like this is, this process is called natural language processing. And it has natural language understanding, which is the part of the computer that will learn to hear, and natural language generation, which is how the computers will.

The production.

Exactly, exactly.

Do. Have we reached the level where a person can have a real conversation, full conversation with a computer?

Yeah, but actually, yeah, I was reading a paper, there was some latest releases from Google. Let me tell you something. Ten years ago, computers were really bad at understanding and replying. Five years later, they improved a lot. And then two years later, they amplified their databases and now they are bringing many more nationalities, accents, data, linguistic information. They are hiring linguists now. So companies are competing really, really hard to bring the latest, the best advances in language processing. So right now there are some robots. Are there like very famous robots that can have a conversation with you? But we also need to analyze the type of conversation. Like how likely are these conversations to happen? So for example, if I go like, hey, Hadar, how are you? And you say, hi Jesse, I'm fine. Well, that's such a frequent conversation that everyone says in all the languages, yeah, it's not surprising that, that a robot can do it. Right. But when we talk about other very skeptical, philosophical, difficult topics that are not very frequent, that's when the, the virtual assistant or the, the va. The robot will likely give you weird or somewhat not very accurate response. Yeah, right.

Yeah, yeah, yeah.

This is machine learning. So it's, it's a matter of how frequent something is and the sequences of frequency of a topic in a language. And that's what the, the artificial intelligence just goes out there, brings what is out there, what people already do, and just brings it and provides all that knowledge to a computer. And the computer will just look for sequences and frequencies and just reply, what is the. What the most common answers are.

So the more exposure to natural language, the better, the better.

Yeah, and that's something that I really, really like because, like, well, I'm gonna maybe burn a little bit about the presentation that I have for you. But like, it's, it's so interesting to me that many people go like, oh no, I want to learn English with a native speaker because I want to have their accent because I Want to be able to understand them. So let me tell you that artificial intelligence, the biggest banks of data out there are produced in English. So because everyone out there in the world, if they want to say something, they most of the times translate it into English. So, so that is available for everyone around the world. Right. So the biggest banks of data out there are in English. And then you go like, I want to learn and sound and speak this specific variety of English. And then artificial intelligence learned many years ago that that was not enough because you lose a lot of people. Because if computers are only able to understand native speakers and there are so many non native speakers of the language out there, that's a lot of market, that's a lot of people who vas cannot understand. So what did they do? Well, they exposed the computers to so many different accents out there so that the computers could become more effective to give an answer or to provide a service. So why is it that computers and companies are investing so much money to do that, but we limit ourselves to one specific variety or one specific thing? So learners out there.

Such a good point.

Yeah. So learners, people. If you want to be more trained than a computer to understand people, I suggest try to understand so many accents out there and try to. Yeah, that will make you more valuable for the market, for the job market in this case. Because if companies are spending so much money, that's. Because that's a trend. Because in the future we all will be one. Right.

I can definitely recognize a big difference in the voice to text features. Let's say 10 years ago when I was just. You started teaching and using and it was so strict, which was, you know, like, if I tried to teach a proper sound, then there was no deviation. But now, which I love, you know, there's a lot more freedom in the. And it recognizes, it doesn't recognize it word by word. So like it really understands the entire meaning of the sentence too, which is incredible to me. That's mind blowing because I was deliberately mispronouncing words in the context of the other word to see how it behaves. And it definitely detects, you know, the. My intention. So I can definitely see that it's much more open to sounds and, and mistakes and definitely accents. So I think it's, it's, it's great.

It's amazing. Yeah. Yeah. So do you want me to share what I have for you?

Yes, go ahead. Okay. Yes.

So let's see if I can do this again. If I can't, just please let me know if I'm doing it right. So you're supposed to be able to see the presentation now.

Yes. Guys, let us know in the comment if you can see the presentation.

Yeah, I would really appreciate it because I cannot see how they are there now. But you're.

I'm here.

The presentation. Yeah. I trust you. I see. I hear your voice. Okay. So here, let me just go back because this thing.

Okay, thank you. Everyone can see the presentation. It's good. Great.

Yeah. So my point for artificial intelligence and English learning is this. Exactly this. So when you are exposed to many accents and to different ways of articulation, that's what research calls high variability phonetic training. So you can detect, for example, if I say the word tree and if I say it with one accent or another accent or another accent, you will listen to differences maybe in the sounds, but you will also find the general gist and your brain will recognize what doesn't change. Right. So that's what we call high variability phonetic training. It's. It's better to be exposed to many accents so you can get the gist and keep what is useful for you to learn and just, you know, discard what is not that important.

Okay.

Artificial intelligence promotes autonomy. So I don't know. You Hadar, tell me, how often is it that students want to have you there in front of them for two hours listening to the many repetitions just to tell them, hey, you're doing it right?

Absolutely. Feedback is key. Like, I, I'm not. I don't necessarily think it's key in learning, but I. It's definitely key for the students to know if they're on the right track or not exactly.

So. But sometimes we don't have, like, teachers, we don't have two hours to be there with our students listening to every single repetition they do. And sometimes they're not sure if what they are repeating or practicing is actually right. So if you have a computer listening to all those different trials and mistakes and giving you an immediate feedback, then you know, hey, I'm starting to do it wrong. I need to modify this. I need to do that. But of course, you only do this when there's a teacher or a professional trainer behind you telling you what to change or what to keep. Right. But it can help you feel more confident. It's not the same to make a mistake in front of a computer than in an actual conversation, in a job interview, or in an important. Yeah. In front of important people. So it promotes autonomy. Like, it allows you to do so, like, to commit so many mistakes and still be okay because no one around you is annoyed, your mistakes or anything.

Absolutely.

And the last part is automatic speech recognition. And here is when where many teachers just don't really trust that computers can do something for them or for their students. And yeah, many people get frustrated. So let me explain a little. So how do computers hear? So this is a four step process. So first the computer gets the speech signal, right? So you speak in a microphone. And then the computer will collect what you're saying for a few seconds. And then the computer will extract the features. In this case, for example, in this example that I have in here. So it can transcribe it from speech to text and it can also tell the phonemes that you're saying. For example, I'm using Pratt in here. That we will talk about Pratt later. Yes, but the computer extracts the features of your speech, of your way, of the waveform and the spectrogram. This is the waveform. This is the spectrogram. And then the computer will process this information and transform it into. It will decode it into instructions for the computer. And then after it's decoded, it will process what's going on. Like if I need to do something, if I need to reply, like the computer will post process that. But then here's the problem. And Hadar, I want you and your beautiful American accent to read this thing in quotes right here.

Okay, but will the new display recognize speech or will the nudist play wreck a nice peach? Okay, wait, let me say it again. But will the new display recognize speech or will the new display recognize speech? Oh, I love that. It took me a second.

Is it hard? Okay, if you were not. If you were not reading that and someone said both sides sentences, would you be able to tell the difference at the beginning? At the. Like the first trail?

Yeah, like the second. The first option would automatically come up. Like, I wouldn't even think of the possibility of the second option because it's. Doesn't make a lot of sense. Or it makes. But like less frequent, right?

Exactly, exactly. But both sound almost the same.

But the computer is not going to.

Exactly. The computer doesn't know that these words are common, commonly, frequently said by anyone in the world. Computer just doesn't really know about the context.

Wait, I'm gonna, I'm gonna. For those who missed it, I'm going to explain again. What happened here is that Jesse asked me to read the first sentence, so I said, will the new display recognize speech? Now, as you know, everything is connected in English, right? I'm Connecting all the words. And this is just a collection of sounds. The second sentence here. I. I'm going to say it again. Will the new display recognize speech?

Right.

If you just listen to the sounds and you don't look at the sentence, it sounds almost identical. Right. So this is. This is what Jesse's talking about here. Yeah.

And. And actually, there are experiments that have been, you know, carried out with human beings. So, for example, let me give you another example of this phenomenon. So I speak Spanish, and for me, I can say, ba, ba, ba, ba. Can you please tell me which two letters I said when I look at you?

No.

Okay, which letters did I say?

B.

Okay, but tell me the names of the consonant.

Oh, B, A, O, P, A.

Okay. So there are. Yeah, there are experiments that say if you read the word PA and I say ba, your brain will understand.

Yes. Yeah.

Because we connect the written information with the sound. Right. And it's hard even for human beings to tell the difference if we don't see anything.

Yeah.

And if we don't have any context. Well, why do you expect computers to learn everything?

Absolutely. Such a good point that even the human brain cannot detect, you know, everything. There is also the McGurk effect, which is pretty similar, where you see someone saying, ba, ba, ba, Right. And then right after they're saying. But you hear the sound of the ba, ba, ba, and you think that they were saying the same thing or something like that. Like they're pronouncing something else and they're. The recording is of a different sound, and what you detect is the sound. That is what you see, actually. Yes. I think I might have confused it. Right. You hear fa, even though it was. So you have to take into consideration so many elements and how you process sounds and how you perceive them. Like, if we need to do that. Definitely. Like you said, computers.

Yeah. Let me tell you something else here that I have. And this is the. The greatest opportunity for me to actually talk to you, because I wanted to talk to you, like, four years ago, but you didn't know that I existed. So it had. Computers can make three types of mistakes. Substitution. They can substitute phonemes, they can delete phonemes, or they can insert phonemes. Do these Sound familiar with L1 interference?

Oh, absolutely. Absolutely. 100%. This is, like the main challenge when it comes to speaking a second language. If the phonology of one language does not like, or the constraints in one language is not. Are not the same as the constraints in another language.

Exactly. So let me tell you, here are three Examples that I got from what I've been working on. So the user, a native speaker of English says trip but the computer transcribed trick. The native speaker said means and the computers transcribed means. The native speaker said sit and the computer transcribed six. What's the problem? Can you tell the problem or the pattern and the errors of the computer?

Well, I think it cannot recognize the difference between certain consonants.

Exactly. Like for example, at the end, voiceless stops, like. Right, trick.

Right.

Like exactly. There has to be an aspiration so that the computer can actually get if you're saying a P or not. Right, right. In connected speech, we normally omit sometimes the aspiration so the computer cannot tell exactly when, when you are seeing P or C. So for example, in the case of means and beans, the spectrogram. Yeah. Like if, if you see that in a, in the voice in the spectrogram, you will know that the position of the M and the B is so close that the spectrogram confuses the. The like. How do I say that? The strength of the lips, if they are being like relaxed or if they are being like actually they're pushed like the, the spectrogram cannot see. See that? Right. Or sit and six. The problem with this too is that the spectrogram with the, the aspirate, aspirated T and X will show a lot of noise. Right. So.

Right. A lot of times the aspirated. A lot of people, speakers might add an S trying to pronounce an aspirated T. So they'll say time like ts instead of time.

Right.

And that. I think that's the same situation sit. That air, you know, the like burst of air is detected as an S. Right?

Yeah, yeah. Again, because of the air. The hissing sounds always will make the, the spectrogram a little bit darker and not be. I mean the computer will not be able to actually see the formants and see what's going on with, with the intensity of the sound. So those are hard for the computers. So yeah, computers have these limitations, but they can tell when something's going on. So let me tell you, this is another thing. And here I am talking to the expert in pronunciation instructions. So yeah, so the learner. These are. Now this was a native speaker, this is a language learner. Okay. So they have to read sit and the computer produced seed. What's the problem here? Hazard? Can you tell me the nationality maybe of or the L1 of the speaker?

Well, several. We have several options here, but we can definitely think of people who have probably a five vowel language. Right. Where there is no difference between the high E and lax, seat and sid. And in some languages, there's also a devoicing of voiced consonants. So that could be it. Or just like a, you know, misdetection. But still, like I would say someone who was trying to say seed made it a little too short.

Seed.

Right. And then it ended up being. Or the other way around. Was he saying. Yeah, coming up.

Okay, so the first. The first words are the words that were written. And the second words, what was the text? Retrieve exactly what the computer heard and retrieved as a text.

Yeah.

So the second one, this, they had to read desk, but they said the computer detected desk. Is this something that is common in language learners? Can you tell the problem?

Yeah, I can already guess what language is it. Is this the same speaker?

Okay, the first two. Yes, the first two.

Okay. Okay.

Yeah, yeah, yeah.

Okay. So it's the same idea. Disc lacks a. The learner was just dropping the jaw too much, saying desk, which is closer to an A in many languages. Pet. And bet the voicing.

Right? Yeah.

So the learner was trying to say Peter pet. But there was probably. There is no P, maybe. Or the P was not aspirated. So the computer detected it as a B.

Right, Exactly. Yeah. The next one. Leg, left. Can you see that? That's a difficult one.

Let me think.

Leg. Left.

So maybe turning the last consonant into a fricative.

Leg, like in some languages. Some students, instead of saying, like g can also sound like. Instead of getting leg, they go left.

Yeah. And then it detects it as an ft. Yeah, exactly.

And then the last one. The last one.

A fricative is a sound that. Okay, so there is a stop sound. Is a stop sound.

Right.

The. The air is blocked in the throat. And a fricative is. Is a sound where the air passes slowly between the articulation organs. Like a sound. Right. It's gradual. So that. That is what we said. And sometimes there are some phonological processes that occur in some languages where even though there might be a stop sound at the end, it would turn into a fricative, which is what probably the person or the learner was doing here. These and Cs. Yeah. So miss. Like a different pronunciation of the th. Where the tongue was not sticking out and produced in. Inside the mouth instead of. Right, yeah. Also devoicing, I would say.

Yeah. So as you see here, like, computers can maybe not. They are not able to. Well, I won't say that, but. Because they are really advanced now. But in the past, they were not able to understand Non native speech because of these details. But in here, if you are a pronunciation coach or a teacher, you know what to focus on if you detect a pattern. Right. Because you know that. Okay, the computer is getting this. I am the expert. I know why. As just described all the phenomenon that is going in here and you didn't even hear the person, did you?

Right.

You gave me a whole diagnosis of what needs to be worked on, but you didn't even hear anyone. So that, that can help you. And also remember that if you hear something, your brain will try to connect on what you know. You are an expert. But people who are not experts on the topic Ba Pa may sound exactly the same.

Oh, absolutely.

Yeah. So when you, when you see it, when you can visualize in a way what is going on in there, then it's easier for you to start moduling and working on what needs to be worked on. Yeah. Right. Let me give you the last example. So the user needed to ask the virtual assistant to zoom in. And after 1, 2, 3, 4 trials, the virtual assistant. God, all those answers that we have.

Oh, wow.

Can you. This is a difficult one. Right. But what happened? Can you tell what happened or can you figure out?

Yeah, I think. Yeah. Instead of this, it was pronounced as a J for sure. Because it was detecting like the maybe J. And then maybe there was like a vowel insertion. Then zoom in. Two syllables. And jewelry. Three. Yeah, all the rest are three syllables. So there is like a vowel insertion.

Right, insertion. But there's also a liquid near the liquid. Just as we have intrusive Rs, we can have intrusive Ls. Liquids can be intrusive as well. So there's. There's an intrusive L here.

Absolutely.

Isn't it interesting? And then you want to know like.

Hey, totally missed the L here. Yes.

Yeah. What. What happened? Like the person linked zoom ling or something like that. Since that's not a word that exists in English, the computer will bring all the ones that look like or something.

Yeah. Which is like how the brain works. When we hear words that we don't recognize, we simply search for the closest word that the brain knows. Right. This is why we want to know how to pronounce words. So we're. So people detect our message clearly.

Yeah, that's my insight. That was exactly my insight when I was working with this.

Amazing.

Okay, now let me go on here. As you know, as a teacher that this is the way street, you need to train perception and you need to train production. Some people are very good at perceiving sounds, but they don't have to have it, so they don't produce the sound. Right. Some other people can produce the sound, but they're not aware of the existence of the sound. So here is what. Natural language processing can help you a little bit. So there are tools online. There are so many tools out there. We are not sponsored by anyone today, by the way.

No affiliation here, nothing.

Yeah, right away. But all the ones that are here are just free access for everyone. There are others who claim to be more specialized, who, which have a beautiful interface and which actually collect data and try to give you more. A more personalized answer. And those actually. Well, you're going to pay for them, of course. But if you're looking for something for free, that is good and that can be trusted here I've collected a series of tools and this one is the one I want to talk about. Google's how do you pronounce? So I placed it in production because when I was working on my thesis and I was doing all this research, you hadn't issued the Google how do you pronounce tool by then. So you actually released it right after I defended my thesis. So it was like too late. Hadar.

Yeah.

Why didn't you tell me to wait?

Took a long time to start working with Google on this. So all the bureaucracy.

Yeah, but how do you pronounce is a great tool. And it goes, I think that you're. Your point there was to work on perception. That's why you have the lips, that's why you have the sound. Right.

You mean to teach the computer perception? Yes.

Well, what, what I mean is that, yeah, what you did was to work with engineers so that they could code the pronunciation in such a way that the computer could reflect it.

Yes.

So you also have work with artificial intelligence.

Oh, absolutely. Like big time. Because I had to really explain to them how I think this is like I just had to. How I process a sound, like literally. So for those of you who don't know, I was consulting Google when they produced this feature that how to pronounce, where you type in how to pronounce and then a word, then you can, it shows you this phonetic transcription, but you can also record yourself and get feedback. It's not perfect. There is like, you know, doesn't always detect your voice properly. But generally what I was doing there, well, I had two hats there. One was basically to explain how I think and how I work and what feedback I would give. Right. So like all of that, I was Literally teaching the computer how to listen. So I would listen to sounds and transcribe it, except explaining what I'm hearing. Right. And then so. So all of that process, I think they were learning and of course, other things that I. I can't talk about. And then.

And then, and then in.

Addition to that, like how to present it to people in a way that is clear.

Right.

Because that is also. That also needs to be intuitive and, and clear to understand what a person needs to focus on. If there are several things, for example, and what feedback they get.

Exactly. So I placed it in production because at the time, I remember that when you first release it, when Google released it, it didn't have the option for the person to record themselves. It took it a while. And yeah, because I was going to use it with my students instead of the tools that I had, but it didn't have the option to produce. So it's okay when you train someone to perceive, but also when you link it to production. And then Google had this little microphone there where you could record it. And it came like, I think it was a year after or some months after that.

Months after.

Yeah, it took it a while, but it was. I mean, that's the perfect connection. And that's like when you close the cycle and that's the important thing. And another question that I had for you, since. Since we are actually talking about how important it is for people to see what their voice is producing, why didn't you. Well, not. I mean, I know that there are many opinions about the IPA out there, but I also couldn't use the tool because it used somewhat English orthography and you didn't use the ipa. And I want to know what the conversation was like.

Okay. I think they in particular were looking for something that is not ipa so it's more accessible and that the spelling for people who use, you know, the, the Roman language, like, you know, the, the. The Alphabet, that familiar Alphabet, it's not as intuitive for all speakers. Right. Like, it really depends of what your native language is. But like to use a language or the English language also, it has to do with what, what they can present. Right. Because up until that point they had these features and characters that were very unintelligible. That means they had like double O, but this with little curvy line above it. And you know, people did not know how to use it. So they wanted to do something that is a bit more intuitive and clear for people to understand how to read it even without the, the ipa. Because ipa, everyone can go to the dictionary and find out. But, you know, they wanted to do something that is, that is more accessible. Because that's what Google is about, right? To make it accessible for everyone. Because to read ipa, you need to know ipa. And we're talking about a very small percentage of the people using this. And also, I think I, I. The hardest thing here was really to represent, it was easy representing consonants, but the hardest thing was representing vowels, especially because they change. Because they change. Yeah. And then like, you know, double O, it takes them. You know, a lot of people see double O and they are, they already associate it with their native language, not their native language or the, the common spelling in English. So double O is usually the lax cook look book. Right. But then when you use it for something else, so it could be very, very confusing. Which was a constant conversation that we had there.

Yeah, I, I knew that there. This was a conversation, definitely. And if you're really nice, if you could add the IPA besides what you already did just for other people out the ipa.

Yeah, I agree. I think, I think having both is, is great also so that you can, can compare it. But, but yeah, and I think also, like, they couldn't use the characters of the IPA to show as easily, you know, so because the characters of the ipa, that's like a different script. And it was, you know. Yeah, well, I don't know.

Yeah, maybe. Yeah, yeah. Those are the technical issues that we don't want to go into. And also because those technical issues that actually make some virtual assistant or some app more or way easier to understand is what make them have a cost, an extra cost, right?

Yes, yes.

This is why I advocate for the use of technology that can be inclusive. But also, I'm not saying that technology can replace a teacher. So you understand how if you are a teacher, if you're an expert and you can decode what the computer is doing or predict how the computer will behave, you can use it as a tool and you can, yeah, you can use it. For example, I know that many people, many teachers, for example, they have an audio of their students, and the students produce a complete paragraph and they don't know where to start, Right. They go like, oh, okay, we can start with vowels, consonants, and there's usually a pattern. And then automatic speech recognition can show you the patterns, or you can see in the paragraph the words that are most commonly misspelled. And then you can tell, okay, this is a highly frequent word. And the sound is a sound that actually makes a difference in meaning. So you can as a teacher or as a pronunciation coach, you can prioritize and you can actually see how you're going to work first and what you're going to tackle later, et cetera. And, and in such a way that the computer or the student can dictate the same paragraph three months after and see a high improvement. And it will see it because you tackled the sounds that were the most problem, problematic. And if the student can see what they dictate at the first month and then after six months, three months of training, they see a difference, then it's because, I mean, it's not, it's not easy to go like, hey, yeah, I taught you how to pronounce things. And to my perspective, you sound like a native speaker or you sound great. It's not the same as if the student actually can see what a computer or an AI can do for them. Now, I'm not saying also that artificial intelligence is like the greatest thing, because I know that it still has a long way to go and it still has some mistakes that it has to correct, but there are so many people working on that, me included. I'm working with virtual assistants that have to generate an answer for. For the user's utterances. So if the virtual assistant does not understand how the. What the person wanted, but the person wanted to zoom in, then I'm going to tell the computer a. This pronunciation zoom in may not look like zoom in, but computer, even though you don't like it, the person wants you to zoom in. So if the person is saying jewelry. Ghibli. Ghibli. If that sounds like to you, maybe the person wants you to zoom in. So the computer will take that as another option. Right. And then it will increase its database in such a way that, yeah, maybe in 10 years computers will be able to understand us all in a better and more effective way.

Yeah. But then it generates. Then they become smarter than people in a way or less biased, let's say computers. Less biased. Yes. Which on one hand is, you know, an ideal world. And on the other hand it's just, it creates a false. False. Yeah, exactly. Representation of what, what things. How things really are and how people are viewing very, very rigid and how they hear, especially if they weren't exposed to a certain accent or dialect. Right. And I wish we, I wish we could turn on the button and, and teach people how to listen like we do with computers. That could be amazing.

Yeah. Yeah. But that's another thing that Jonathan texted me and he was like, hey, can you please Ask, talk to Hadar and talk about testing like this, toefl, ielts and those standardized tests which are using automatic speech recognition and artificial intelligence to process the speaking part. And it's like, okay, it's a good thing because you take the bias out and we all have a filter. And even if you are a highly trained examiner and you have to give a grade to the speaking of a person, if you don't have like strict guidelines out there that take you out of the bias, and it's like, yeah, the first thing is not hearing the accent. Maybe, maybe you don't hear the accent and you only read what the computer was able to retrieve. Then as you had, I mean, you didn't tell me the nationality of the people, but you can tell what the L1 is and you can tell what the interference is. Maybe you can speculate, but you don't have a bias. You didn't see color of skin, you didn't see eyes. So I think this is what we can rescue from the, the current stage of artificial intelligence. Did I convince you?

So beautiful. Absolutely.

Yeah.

I mean, with that last sentence, of course.

Yeah, I know, I know. And so I've been using Pratt for linguistic analysis and speech analysis. And I have also a tutorial, a free tutorial in my website, how to use Pratt. But I have to update it because Pratt just released its latest version last month. So I'm going to have to update all the tutorial. But teachers, for example, if you want to visualize what students are saying and maybe you find it hard to perceive because we are all biased by our own perception skills. So when I started to use artificial intelligence, my claim was that, okay, if science, because yeah, I found so many quotes out there in literature, scientific research saying, yeah, maybe non native teachers are not a good model. I was like, okay, if I cannot be a good model and then everything my students say will be, will have to be filtered by my own bias, then how am I going to be objective? How am I going to be able to help? But then when I have automatic speech recognition and AI and computers to my side, they help me take a little bit of that bias out. If I use them in such a way that they help me do my work or do my job in such. Yes. So that students don't feel that, hey, the teacher just doesn't want to help me because she says this sound is not important. Like for example, when I was doing the research, I was working with vowels. This is something important. Research and literature says that we work with minimal pairs. So for example, in Spanish, the kid vowel and the fleece vowel are a problem. Right. But then students, instead, the kid vowel, if you tell them, hey, it's different from the fleece vowel, then they will go all the way to the dress vowel. And then you have to teach the dress vowel. So literature won't often tell you. They. They go minimal pairs. But you know that sometimes we have to bring an extra ally so that. So that students can create a new category or just distinguish, understand?

Because, like, it's. It's. You know, it's like a game where you have holes in a ball and you have to fit it in a hole. So if it doesn't go here and it doesn't go here, the.

The.

Like, they'll find a way, like a hole to put it.

Yeah, right.

Well, no, but. Or they'll place it in.

Yeah, yeah, yeah, yeah. If they.

They're aware. Yeah. They'll. They'll automatically go to a different sound. But what you're saying is the ultimate solution to create a new category, like, or at least know that it exists because it's hard to create or impossible to create, like, new categories, but consciously become aware of it.

Yeah, exactly. And go like, hey, it's not this one, but it's not this one either. So you, as the expert, you bring, like, you. You. You provide the student with enough samples so that they know. But it also. It was also. Also difficult for me because my perception is already biased by my L1. So I was like, okay, let me look at that. And then when Pratt showed me the vowel chart and how close the kid vowel was to the dress vowel or to the Spanish, A was like, oh, shoot, like even, you know, hey, they are super close.

They're even closer than to the high E. So there was a research done. Israeli speakers also. Also have five vowels, just like in Spanish and speak. Learners were. Learners heard the E sound in isolation, and they were asked, which one is it closer to the E, the Israeli E or e? And like, 98% said, eh. However, in pronunciation, everyone pronounces it as an e because spelling plays a significant component here. So if you listen to it in isolation, a, they analyze it as a. But when they read it, they'll say, seed, sit, kid, lift. You know, put a, take it. So. So definitely, like, all these elements play a significant role in determining someone's. How they place it in what category.

Yeah. And. And when. When Pratt can tell you what to see or why is it that they are so hard even for you to distinguish, then you Go like, oh, you see the Bible chart and you see how close they are. They were like, this is why. Yeah, now I get it. They are exactly close. So, yeah, and sometimes you, as a teacher, as a pronunciation coach, you have the feeling. And if you are a native speaker of one language, but you are not aware of what's going on in all of other languages, you go like, hey, why did the student say jewelry instead of saying zoom, zoom in? And then you're like, oh, how can I help this person?

Absolutely.

See what is there. And then you start tackling, like, in order. Or you. You get your priority according to your own process and how you decide to tackle it. Right.

So, Yazzie, how can people get access to all those resources? Do you want me to share in the description? Do you want them to go to your website to find out about it in.

In the description? Well, not. If you can share my website and the description of the video, it would be really nice.

It's already in the description. It's already there. So you have her website. You can click on it if you're watching this on YouTube. On Facebook, just.

Yeah, well, yeah, YouTube, Facebook. I have a website that is. Well, a web page that is Jazzy Addis. And in Instagram, Jazzy Addison. I'm not very creative with marketing names, but my website.

Your name is your brand. Like, yeah, yeah. Jazz is also an ETA member in English Teachers Academy, our community of teachers and membership. So she's like, we've known each other for. For, like, in all aspects, not just the teaching aspect. Yes. Okay. So you can find her here, and if you're on YouTube, click the link to see her website.

Yeah. And. And my website, you're going to see some English courses, but I also have Pratt Elan. And I'm. Yeah, like, I am planning to update the Pratt tutorial that is for free over there. I don't charge it because I know how hard it is to work with Pratt if you are so hard, if you're not familiar with it. But I also. Sometimes I open courses when I have, like, many people who have actual data and they don't know how to process it and they want specific things. Then I open a course. Course, and then we go over each ones and stuff.

Amazing. So teachers, pronunciation teachers, if you want to learn how to use those tools, then definitely contact Jesse and. And go to her website and find out more about what she offers. This was an incredible conversation. I was so happy. I. I could have. You know, we could have continued for another hour or so.

I think I have so many other things that people ask me for, but we will have to have another life.

We, we, we will have a follow up, maybe on Instagram for sure.

Yeah.

Amazing. So thank you so, so much. And you guys, if you have any questions, just ask them in the comments. We will be in the comments of, you know, both on Facebook and, and YouTube and answer you and, and that last thing you want to share with people.

Well, first, thank you, Hadar, because these questions that I asked you and the game and everything, I had wanted to have this conversation way before this day.

It was my favorite part.

Thank you so much for having me. And yeah, people out there, just the thing that I get from the current state of AI is first, being classive and try to learn and expose your ear to as many accents as possible. Possible, because that's the trend. And another thing is that pronunciation coaches, you can visualize data. You don't have to always rely on your intuition and be afraid if you're doing it right or wrong. Just you can visualize data and, and tackle it more effectively or quicker. Or maybe find and introduce the use of automatic speech recognition and AI in your process so that the students can also see like some, I know that some pronunciation coaches use Pratt and then they produce a vowel chart and then they, they show the students, hey, you're pronouncing it in here. You have to open your mouth. But they use the vowel chart and personalized vowel chart. And sometimes it helps, sometimes it doesn't. It depends on the speaker and it depends on the specific necessities of the client. But you guys are experts, so pronunciation coaches, you know what that, that person needs and, and if you can get tools to visualize and help them see their process, maybe it will be you will sound more reliable or maybe you will take the intuition and subjective part out and your bias.

Yeah.

And try to be more objective of what you're teaching.

Yeah. And, and confidence is everything. You know, we talk about it in language learning. Definitely in teaching, when you have that other, like something rely on, I think that really helps you build your confidence. And, and you know, it's a confirmation for what, whether or not you're on the right track. So definitely use that.

Yeah. Yeah. Because there are so many people out there. I was listening to a podcast and, and this people are, they are experts in tefl. And the, the, the presenter, she, she says that she receives lots of hate comments because people go like, hey, I don't even understand your accent, accent. So she might feel afraid and she might feel that her accent is not good enough, but when other hundreds of thousands of people understand them because she's in a podcast. Right. But if you're not in a podcast and you want to know if your speech could be understood and you don't want to feel afraid of judgment, just dictate to Google something and then you will see, like if the computer is able to understand you, then you shouldn't feel bad and it's not your problem. The problem is the listener who hasn't exposed him or herself to enough accents and they are not good or efficiently trained for the new trend coming. And maybe computers will take their jobs if they don't. If they are not willing to hire other teachers or ask accept other accents.

Yeah, because what if, what if like global companies stop training their employees and like, you know, pronunciation or accent training or let's call it what they actually call it accent reduction. Yeah, right. Let's lose your accent. Okay. What should I speak with an accent as an accent like it. And instead train people to listen better. Right?

Yeah. Yeah. If you see that the enterprise says we only hire native teachers, you have to be aware that that will be a problem in the future. If you're only used to native speakers and there are so many non native speakers out there and the computers are able to understand them better than you, then you will not be ready to be effective in the job market. So be aware. Just take that into account. Companies are spending so much, many millions and billions of dollars so that computers can understand everyone. Then you should better try to understand everyone as well.

My.

I said it.

That is so good. That's it.

That's it.

I'm not gonna add anything. Just going through it. Yes. Okay. Jesse, thank you so, so much. It was such a pleasure, everyone watching. Thank you so much. Reach out to Jesse. And I'm gonna put the this again. And all the links are in the description. Thank you for all your wisdom and insights and for that little game that I loved and. And I. I will see you soon. Thank you so much, everyone. Thank you so much. And I'll see you next week. Bye.