DeepLearning.AI NLP Learner Community Event ft. Nikesh Bajaj

Hello everyone. Welcome to this event organized by NLP DeepLearning AI for NLP learner Community. My name is Nikesh Bajaj and I'm very excited to be your guest speaker today. So I'll start with my presentation for 15 to 20 minutes and then we'll have a question answer session. For rest of the time I'll be answering your questions on Slack. So don't forget to post your comment and question and upload the question on the Slack and wait. I would like to thank you all to join this event. So let me start my presentation. Yeah, so I'll be starting with this. So the presentation, the plan of the presentation that I'll start talking about how I started my journey to AI. Then I'll go into detail of my PhD work and current work, what I'm doing right now and then we go to the question answer sessions. So I come from a very small town in India called ITRC and it's kind of a very center of India. I did my Bachelor's in electronics engineering from IED and in 2010 I completed my master's in signal processing from EMU. And since then I worked for five.

Years, almost five years in north of India.

And I started in AI from in 2012 with the course and Rangy's course Machine learning which was on Coursera. I enjoy that course very much. It was quite interesting course and I still remember that it was a start of the course Coursera. That time we used to have the ML hyphen.org as a link for the course and it was so exciting that I still remember when they announced the course I announced that there will be many more courses coming from any other university. Me and my roommate, we literally celebrated that news. It was so amazing too that we will be learning now from many more professors. So that was my first course. Since then I have done several courses from Coursera, Udemy, Udacity and edx. So and still I have been just doing many courses. Apart from that I started working on some Kaggle competitions, though I didn't win any competition from there. But I learned a lot from that. It's good point to start with also because you can in a course you learn and you start with the basic examples and you can actually go to the Kaggle and you can simply apply those approaches to the real world problem. And in terms of learning, you learn a lot from many other people who are using who are on the Kaggle and they share their code sometimes so you can see how they are approaching the problem. So you learn not a lot of new tricks and tips that how people are approaching it, people who are winning, what kind of approach they have. So it's kind of a very good platform to learn a lot of new new things. So. So I participated in few competitions, I didn't go further then. And then in 2015 I joined a PhD at Queen Mary University of London with University of Genoa in a joint program.

My thesis title was Predictive analysis of auditory attention from physiological Signals which I'll.

Be talking in detail in next slide. And during PhD I was also involved in a lot of other activities like.

I did the specialization from Deep learning.

For Deep Learning from Coursera. I attended so many summer schools courses, offline courses and give presentation on the topics for AI and ML. And I participate in data study group which is organized by Alan Turing Institute. And it's kind of a very good event where companies, they come and they post their problems and there's a team of people who work on that problem and at end of the week we show the one solution that we have. And I won also two competition as a second prize winner with a team. So there were a lot of activities I was involved with in 2019. Just last year I finished my PhD in May and joined University of East London as a postdoctoral research fellow where right now I'm working. I'm working on a project called Deception Detection and Conversation. While the official title of the project is kind of bigger, but the essence is the Deception Detection in a conversation. So that's what that I'll be again talking in detail which you all might find more interesting because this is more related to the NLP that you all are already learning in the courses. So about my PhD work, the title was Predictive analysis for auditory of Auditory Attention from physiological Signals. In the whole project we call it FIAT as a physiology of auditory attention because there is not much work about the physiology of auditory attention when it comes to the speaker attention to the speech rather than on the audio signals. In this PhD we designed one experiment. That experiment is based on dichotic listening task where dichotic listening task is one of the famous experimental setting in psychology for evaluating the attention. So our experiment was based on that, but we didn't have the two stimuli for that. So experimental setting look like this model where we have a listener, we have a different auditory condition which we change for the auditory stimuli that we have. Auditory stimuli is just audio message and we got three different kind of physiological signals which is EEG and GSR and ppg. And we evaluate the auditory attention based on the transcription of a listener about the audio message. And the auditory conditions that we change are noise level. So we change the different like background noise level as a crowded noise, how much, what's the level of that background noise which is the semanticity of the audio message. So audio message, sometimes it is semantically correct audio message and sometimes it's not semantically correct and the length of the message. So these are the three different auditory conditions that we had. We were changing it and we were presenting it to the listener and we were recording all the signals. So from the experiment we collected data set conducting with the 25 subjects. We recorded signals 19 stream of the signals at the rate of 128 Hz, 14 channel of EEG, 3 signals from PPG and 2 signals for GSR. So collected signal looks like this picture at the right side. So it looks like it's labeled properly. So you can see that there are signals, all the signals, eeg, GSR and ppg. And this green light, green background is actually the listening segment. White background is the writing segment and there is a resting segment. And for each listening segment the attention score which we calculated manually is labeled for this this score. So overall objective was to actually evaluate and predict this attention score based on the physiological responses. So as a label we had every signal is proper labeled with the task that what task the subject is performing like listening, writing or resting. And what auditory conditions are there? Like what's the level of noise level was that mass is semantically correct. What's the length and what's the attention score. So from this collected data we found four different predictive tasks. They look like the first first task is kind of a very obvious that we wanted to predict attention from the responses. So if you remember this model, so we want to predict this attention from this responses.

That's what the first predictive task.

Now our statistical analysis shows that this attention score depends a lot on the auditory condition. So for the for example, in a very noisy environment, attention score is very low. So this attention score is very much depended on these all auditory conditions. So we had a kind of a significant analysis that which factor is affecting how much. But in practical situation we don't have access to these conditions. So we formulated other tasks that can we predict these auditory conditions based on the task given by this. So can we predict these attention these conditions from the physiological responses. And if we can, then we can actually predict attention from that prediction. So it's kind of like we can multi level prediction and hierarchical prediction thing that we can use one prediction to break another thing. So these are the different four tasks that we formulated. All the resources, material and whatever publication that we have till now, everything is shared on this link. We created this website where we have all the libraries and all the functions that we need to start with this, we have already provided some of the script that you can just start with playing around to see if you are interested in that. So this was all the PhD work and in currently. So currently I'm working on this project.

Which is called Deception Detection and Conversation.

So we, I'm working with the University of East London. We have partners like intelligent voice trainers and this project is funded by Innovate uk. And so the problem with the deception detection is very specific in this. So the idea of project is that we want to detect the element of deception in a conversation where this conversation might be, let's say the operator of a financial company and the loan applicant. Or it can be any other situation where customer from the bank customer is calling to access his information or account detail for the bank. So there's operator and there is a customer and there's a conversation. So we just want to identify can we find out that if the person who's calling or customer is being deceptive. So that, that's the whole idea of the project. What I like about this project is that this, in this project we are again going to the looking into the what's happening in the brain. Because this, according to the linguist and cognitive psychology, there's a lot of thing that goes in brain when you are being deceptive or when you are lying and knowing those things actually help you to identify it. So there is a kind of a psychological compulsion for you to behave in certain way, choose certain words. So what I like about this project is that we are looking into again what's going on in the brain from what you are talking about. So the assumption here that if you are being deceptive you will leak some information in your conversation, in your choice of words, in your way of speech and everything. So that's what I like about this, this project. So the high level idea of the project looks like this. So we have, let's say the conversation here, we have the transcription of the conversation and we have the audio. We go with the acoustic feature from.

Audio, we go with the linguistic features.

From the transcription and we have designed.

Like some model to predict the score of the deception.

So that's a high level idea of.

The project and it goes much more detail into that.

So if I talk about the simple use case about this project. It looks like this. So if you can see this script right now. So this is a conversation between a customer and operator. And you can see that operator and customer. So it's a customer calling to a bank and asking about the debit card.

Information, his debit card information. An operator is asking can you verify.

Your debit card number? And he says I don't have that right now. And it goes like that. So the first thing that we do is we identify the linguistic markers there. And this is a list of 16 different linguistic marker. So in literature there are many more linguistic markers. Like negation is one of the common one which where we can see, no, not can't anything which is denying is negation and causation where you actually try to define some reason behind it. So because effect enhance and hedging is one of the interesting one which, where you try to be not very certain about something. So where you try to not give the full information, try to be not very accurate about that. So that's, that's kind of a hedging and like maybe and I guess and about. So it's called hedging. There's a slight connection between the hedging and uncertainty. So that's the one of the markers. So these 16 marker and the detail of that. We have already one published paper on this. So I'll share the link, you can go through that. So here's one of the simple model that we presented in the that paper. So in this paper we just use these 16 linguistic markers and the sentiment of each statement. So you all have now you all are very well equipped with the sentiment analysis. You all have done that expand that.

Exercise in week one, I guess.

So here is a model and this decision engine, this decision tree is like a very simple approach that you can see actually it's. It's simple showing that how we are deciding that if it is a fraud.

Class or it is a non fraud class and which feature is very important and all.

So we combine both of the features here. And so the best accuracy we got for this particular approach was with SVM. So 69% testing accuracy. That was our. Thing. Okay, so this is a link for the paper that you can see actually. So now interesting thing is about this paper is that there are a lot of challenges in this particular project. And the challenges are first of all, the first challenge is that deception, it's not a very well defined concept. It's more complex than simple labeling of being deception or not. It goes a lot behind the one sentence. So looking into one sentence, you can't be just very sure that it was deception or not. So it has lot of things which is going behind it. Another challenge goes is that ambiguity within the linguistic marker. So you like there's a marker called hedging where there's a term about. So about is one of the hedging term, where if you say the exact sentence like I have about $10 million, it's a hedging because I'm not being very precise about it. So I'm using about. But if you use about, let's say that we need to talk about it. So here about is not hedging because here I'm not being any. I'm not hiding anything. So there's a lot of ambiguity in the linguistic marker. So it's not like a bag of word or the list of regex operation that you can apply and you can get the linguistic markers. So there's a lot, a lot of context. It need to actually you can evaluate it properly. Other than that, the most challenging thing in this project is the dependency of the area like. So every, like this problem depends a lot on the context. So what context we are talking about. So is the context is for the financial services or is it for the loan application? Is it a simple interview, job interview, or is it trial like some criminal trials? And so it depends a lot on that context. And there are papers in linguistics that they suggest that the significance of marker not only just change in the different context, but sometimes even the polarity of significance changes. What it means is if there is a marker, let's say negation in one context, if the negation is very high in a class A and very low in class B, which is significantly different. So if you change a context, it might that these not only just a significance, but it might just change the entire polarity that negation in class A might be very high than class B. So it might just change the entire polarity. So it also depends on the incentive of being like if you want to be deceptive, what is the incentive that you will get or what kind of penalty you will have? So let's say that if you are playing a game and you have to be deceptive, for example, there's a very famous game called Mafia where you have to be deceptive and there's an online version of that called Mafia Scum. So you have to be deceptive on that. So in the game, if you are being deceptive, then incentive is that you will win and penalty is at Most you will just lose, lose a game, it's just a game. But if you're talking about the loan application, there's a penalty and the incentives are different. And it goes even to the extent that if you are lying on a trial, so it's a life and death situation where the penalty and incentives are very different. So different penalty and incentive actually trigger a different stress response of every marker. So it's kind of changes a lot of thing. Other than that it also depends on the media of communication. So in particular this project we are working on the conversation. But conversation can be oral conversation or the written conversation. Now even in oral conversation, is it on the phone or is it face to face? Because if you're talking to somebody on face to face, then you might not have that much time to delay or think if you want to lie or if you want to think about the your answer or if you, if somebody is being deceptive. So person has to be like very instantaneous, right? And if you are talking on the phone, then you're not looking into the person's eye and then you can actually have a little more comfort of line. And if the communication it changes from oral to written, it changes a lot of other things there. So for example, if you go from oral to written and in written also is it an email conversation or is it chat, online chatting or is it written letter? Then everywhere it changes a lot. So if you're writing an email, you have time to proofread it, you can read it again and again and think what everything is meaning and how it gonna affect anything. So you can, you have time to redo your email and you can think about it and properly write it. So, so email is more prepared 1. Online chatting you might not have that much of delay, but still you have a little bit more delay than oral communication and you might just avoid to answer that also. So online chatting is a different one. And if you have let's say a prepared answer that you might have seen that some of the CEO of the company, they have the well prepared answer for their interviews. They are kind of very, very well prepared. So scripted and kind of being weighted same for same goes for the politicians. So their communication has been properly vetted and prepared for the, for the broadcasting. So they know where, what kind of words they should use, what kind of words they shouldn't use to leak to avoid any kind of information or misrepresentation. So media of communication is one of the another challenge that you have. So you have to kind of select very one which you are actually working on. Then it also depends on the per every person. It varies in person, it varies in which region of the people that we are talking about, which language we are talking about. If person is a native speaker or like, like me, the first language is not English. So if person is like that. So. So the use of language for the people whose first language is not English, if you are talking in English, then choice of the words are very limited. So you have a very limited vocabulary in that case. And then evaluating it against the people who have whose first language is English will be little difficult because the distribution of the data will be different. So there are a lot of challenges in this particular problem. And then you kind of have to choose a particular problem that if you want to use this context, this media of communication, this region and this language and then if you change anything, then lot of thing changes. So you can't actually just use a model trained on a one context and apply on another context. It gives you. It doesn't give you that good performance actually. So other than that challenge now looking into these lot of scenarios that we have, the big issue comes with the data availability. So currently there are not many data set available for the deception detection. The one which is famous is the CSE deceptive speech data set. It's a lab setting where interviewer and interviewee are there in the lab. They are like there's an interview conduction going on and interviewee are being asked to lie about certain things. So it's a completely different context and it's a lab setting. And there's another data set which is a mafia scum. It's online chatting room. So it's again it goes with the different context. Again it's now chatting chat room and then there's the mafias who are trying to be deceptive. So there are not many data set available. That's another challenge for this particular problem. And the last one, you want to use this kind of model for any application like legal services or the financial services and Avenue want to implement them into the system. You want these model to be very explainable because you just, you just don't want a black box just giving you prediction or the score. You also want to know that how it is predicting and which part of that it is predicting as a deceptive so that they can dig into much more detail there. So explainability is another problem with the with such models. So these are the all challenges. I think I covered almost everything. So if you have any question, I'll be answering your question. On the slack now. So.

Yeah, I'll be ready with the questions.

All right. So there's a question from Richard Walker. He's asking, I'm wondering if Nikesh can talk about the involvement of signal processing in what task is signal processing applied such as translation nervousness Also how can the data be related with the very small data set observation and how is translation be done Existing framework how important is a useful has this study signal processing have been this project. So as much as I can understand the question is that you're asking that how we can use the signal process, how we can use the signal processing for the application on the for the small data set. If you're asking the signal processing for the deception detection or if you're asking about the signal processing for the auditory attention. So for auditory attention I would say that signal processing it's kind of a very much important because when you're collecting your physiological signals, there are a lot of features that you need to actually come extract from that. And there is a lot of noise in particular in EEG signal like electroencephalographs. There are a lot of artifacts that you have to remove. So in that kind of cases, when you are extracting such signal, they are always noisy and it requires a lot of signal processing that that you need. So another question is how did you overcome the bias in the model? So I'm assuming that you're talking about the bias in NLP model, the model for the deception detection. Well, in deception detection right now what the way we are working the question was asked by Saurabh Thomley and he is asking the question that how did I overcome with the bias in the model? Currently the model that we are working on, first of all, we don't have a large data set for that. We have a limited data set that we are working with and we are trying to generalize it. We are trying also to create new data sets. Bias is a problem that we haven't encountered yet. But that's one of the objective to actually go into that. Once we identify what kind of biases we have, we might actually come for that solution. We haven't come to that solution yet. Shreya Sharma is asking a question that can you talk more about the explainability of AI especially for NLP model? Yes. So for NLP model there are lot of explainability techniques for models like when you go with the model such as BERT model or any deep learning model or any LSTM model. So there's a technique called the occlusion that's that's widely used. You can also use a method like lime and shape kind of tools that they, they can tell you much more detail about the input and output. The occlusion model is very similar approaches, very simple. Like you might have seen the example for images that they in the image, let's say there's an image and it's detecting phase. So they try to hide a small part of the image and they try to see how much prediction value has been changed and they keep doing it to identify which area is affecting the prediction score higher. And that area is being highlighted as a more important area for that particular image to be detected as a face. So a similar kind of model is used in the NLP approach. It's called the occlusion method. Again, so let's say that you have a sequence of words and sequence of word and you want if you let's say for the sentiment there's a sequence of one tweet and then you there's a sentiment is 0.9 or 0.8. Then you go with the occlusion method, you hide each word, you remove each word and then process it again. And then at the last you see what how much variation from the original sentiment value you are getting. And the words which are very much important for the that sentiment, those words will actually be stand out. So if you there's one paper on that explainable AI which is from our lab only it's on inter speech you can go through that. It's kind of a very, very good approach to go through the explainable AI for the NLP models. So yeah, so there's another question with Richard Walker asking all usage of signal detection. So okay, so you're asking about the all usage of signal detection in terms of your previous question. So yes, when, when it comes to the signal processing and signal detection it's if you're in a very simulated area environment you might not get a lot of lot of noise. But when you come to the like if you are capturing a signal from any device then there are a lot of noise that you have to think about. Like even going for the communication system we have a lot of noise in the communication that we have to overcome. So in terms of usage of all signal detection there is no one method that we use for every particular area. There are different approaches for communication. There are denoising methods, there are wavelet approaches to design. First, first thing that we do is we just filter out with the frequency band all the noise that which is not relevant. Like for speech we just reject all the frequency above 4kW, so which is not relevant for the speech processing. So in terms of signal detection, there are a lot of algorithm depending on the again, area which signal and which area you're talking about. So it goes. The approaches are different. There's another question from Rajan Garg.