Analyzing the Temperature-Varied Syntactic Structures of GPT 3.5

Hello, everyone, and welcome to season 2.5, episode 3 of the Wolfram Student Podcast. I'm Sam and I'm your host today. This is the Wolfram Student Podcast, where every fortnight we dive into a new innovative project done by high schoolers using the Wolfram language. For this episode, let's welcome Tianyi, who will be discussing his project on the syntactic structure of GPT 3.5. Hi, Tianyi.

Hi, Sam. I'm Tiany. I am a high school junior from Phillips Academy Endeavor in Massachusetts.

Yeah, that's great. So can you give us some context, like your experience with Wolfram, what camps you attended and such?

Yes. So initially, my first kind of run in with Wolfram was actually this project that I did last year. It was at the Wolfram Summer Research Program. So the one that happens over about two and a half weeks in the summer. And that's when I did this first project. And then kind of after that, I also was a part of the Wolfram Emerging Leaders program Whelp, where I did a project. And I'll be returning actually to the Summer research program as a TA this year. So it's very exciting.

Yeah, sounds great. I was a TA last year, so it's great to have people coming back and being TAs. So I'm excited that's working out for you. And then our classic question we like to ask people when they come on, what's your favorite Wolfram function?

Yeah, I mean, there's a lot of different functions. I think for me, it'll have to be one that was quite centric to. To my project, actually, which is LLM Synthesize. So it's for large language models. It's how you kind of create text. And you can feed it different prompts, you can feed in different parameters as well. That kind of affects the type of text that is generated. And I think that was something that I base my project largely on, but I think also has just a lot of versatility in general.

Yeah, definitely. I think it's really cool how people generally pick their favorite Wolfram function. Few people go for, like, general mass use functions, but a lot of people always have, like, very specific things to their project. So it's fun to hear about, like, a very niche kind of function that was really crucial to your project. Wicked. So let's dive right into your project. Before we start, can you give us a summary about what all this means?

Sure. Yes. So my project focused a lot on kind of GPT 3.5, which was at that point the newest model. That was large language model that was used for generating text. And one of the things of one of the features of all of these different large language models, whether it be kind of chatgpt or like Google Quad or anything else, is that it has a hyper parameter that you can put into it. And this parameter is, it's called temperature. And what it essentially is, it's a parameter that, that influences kind of the randomness and also the quality of the text that is generated by all of these large language models. So in particular what I was looking at was how varying the input of this temperature parameter and kind of the resulting impacts that had that had on either the content or like the grammatical structures of the text that these large language models produce.

Yeah, that's really cool. So I'm excited to see what your project's fully about. So let's just dive right in.

Yes, for sure. So I'm pretty sure as you all know and so many of us have used ChatGPT like one of the newest kind of innovations is really the advent of these things are able to generate a lot of text and oftentimes like the ability is actually very, very impressive and very, very human like and sometimes it can be like difficult to even differentiate from human generation. So kind of going into the research program, one thing I've always personally been interested is kind of writing in English as well as kind of language. So my project naturally kind of centered around how this machine generated language, maybe varied different characteristics of it that were different than human language. And it was really just delving into that. So to begin with, my project initially was supposed to center around kind of GPT2 which was an older version that had come out from OpenAI. So here are just through the Wolfram language kind of initially I tested generating some text with GPT2 just to kind of get a sense of what that looked like and kind of the type of content that came out depending on the prompt that was given. And you can see here that I'm just generating a sample that's starting with Albert Einstein was a German born theoretical physicist. And then it has like 250 tokens. So essentially characters at a normal temperature parameter of 1. And interestingly what I discovered very early on into my project was that GPT2 is actually very quite unreliable. And this is because it came out earlier in 2019. So a lot of the text very quickly either becomes off topic or just doesn't really make sense in it. It generates a lot of garbage content which is kind of hard to analyze. And so My project transitioned then to focus more on G 3.5 Turbo, which was the newest model at the time and is still used on, for example, if you go to the ChatGPT website and analyzing that kind of more up to date content, so that uses this function llmsynthesize. So in this case I'm asking what are large language models? And then I'm also passing in the temperature parameter, which is what my project is focused on, which varies from an input between 0 and 2, so kind of anywhere in between there, which will then naturally get you some sort of output generated by GPT 3.5. So here you can see this is generated with a temperature of zero. This second example down here has a temperature of 1.75. And there are a couple things that you might be able to initially notice. One is this first one is split into paragraphs. The second one is obviously very, very long. And if you look further down, you can actually see that it very quickly kind of degenerates down into some nonsense as well. You can see that there's almost no punctuation. There's some kind of garbage words as well generated when the temperature gets up to the higher levels, such as 1.75. And so in order to kind of look at these sentences that were being generated in the text, there's different ways that we can kind of visualize and think about the sentences. So for example, one of the things that we can do is we can draw out the text structure in constituent graphs of different sentences. So for example, if I feed in this sentence, my favorite food is pineapple, you can see that it splits it kind of into the different clauses, the different phrases, and then all down to the different parts of speech as well, pronouns, adjectives, nouns, down to very fine granularity. So this is for a sentence I inputted. And then we can also run this function on text that is generated from a large language model. So you can see here the response, it's running the text structure on the response to this question of what are large language models? And you can see that it splits it down and basically breaks it down into all of these different phrases and clauses, which makes it a lot easier to kind of think about. And these kind of graphs, these constituent graphs of the sentences are actually tree graphs. So it's kind of like, it's like a tree, it's branching down into different things. So one thing that we can actually notice is that when we are taking two of the same sentences, for example, I jumped today and you danced yesterday, that even Though they have different words, if they have the same grammatical composition, we can consider the graphs, the trees of these sentences, to actually be the same. Whereas sentences that are different will obviously have different structures. And now that we have these constituent graphs of the sentences, there are a couple things that we can do with it to quickly kind of evaluate, because they are considered trees. So a couple things that we can easily see are like the vertex and the edge count. So vertex counts would be all of the kind of points on here, including all of the words, and the edges are kind of all of the connecting parts. So we can think of that as kind of like connecting the different parts of speech. Because this is also a tree, there are a lot of other things we can do. So, for example, the graph radius is the longest, kind of up to down. So here you can see sentence, verb, phrase, noun, phrase, noun. So it's 4, which is the largest pass. Another thing we can do is look at the mean graph distance, which is kind of looking at how far down each of the words breaks into. So it looks at the length of all of these chains from the top to the word, and then it averages this out. Also the distance matrix, which shows us between each of the different vertices, kind of the distance, how far apart it is from the other. So that helps us kind of quantify some of the things that are initially hard to when we first look at a sentence. Right, because when you read a sentence, it's hard. But these things can help us think about it more mathematically. There are kind of a couple other different ways we can visualize the differences. So now that we have these different measures, there are a couple things we can do to check the similarity and differences between these different measures. So one way to check the similarity between two vertices in the same graph. So in other words, two words within the same sentence is using a measure called vertex cosine similarity. And essentially what it does is it's taking these vertex projections of each letter found from the systems matrix, and then it's projecting them into space, and then it's looking at the angle difference between the two. So the larger the angle difference, the more different kind of the word constructions are, and the closer it is, the more similar they are. And there's also a lot of other similarity measures that we can use. So, for example, here we have the sentence from before, my favorite food is pineapple. And you can see that the vertex cosine similarity between the first pronoun and the sentence and the first noun in the sentence are the same. So my and food. And that actually makes a lot of sense because when we're looking at this split down of the sentence, the pronoun, the first pronoun, and the first noun are at the same hierarchical level, whereas the difference between the pronoun and the adjective is different because they're closer together. So the similarity value will be higher. And now we also want to be able to kind of visualize differences between graphs between different sentences. So one way that we're able to do that is to kind of take these graphs and put them on top of each other. So we can take the union of graphs from different sentences and then highlight the parts where the structural components differ. So, for example, here you can see we're taking two sentences, time flies like an arrow and fruit flies like a banana. And when we show the difference between them, we can quickly see how the structure of these sentences can tend to diverge. And this makes sense. For example, we can see that, like, time flies is like a noun followed by a verb, whereas fruit flies, it's like a proper noun, in a sense. And for sentences that. So these two are quite similar in length. For sentences that differ in length, however, we can see that the branching is much, much more dramatic. So, for example, here we have the classic, my favorite fruit is pineapple, shown in red. And then we're comparing it in this case to something that is actually generated by that large language model. And it's the response to what is the meaning of life? And we're just taking the sentence to that response. And you can actually very quickly see that the sentence structure is completely different. And the response from the large language model has kind of an entire different clause, a connection to that response in that sentence. So if we wanted to kind of see the differences in the initial kind of method that we're looking at with the parts labeled, we can again show here time flies like an arrow and fruit flies like a banana. And we can quickly see that, for example, it's identifying the proper nouns, the verbs, and also, like the prepositional phrase, which might not be there in the other sentence, fruit flies like a banana. So now I wanted to. Again, we're looking at kind of how these things vary over the temperature, the range of temperatures from 0 to 2. So the next thing that I did was I created a manipulate function which essentially lets us kind of play around and toggle with the input of the temperature that we're putting in and then see kind of the resulting graphs right away. So this here is the manipulate, and this actually won't work in the project as there's an API key and it doesn't work when it's published locally. But the idea is that you can change the temperature input for the first sentence that's generated by the large language model and then put in a different temperature input for the second sentence that's generated, and then it'll juxtapose these two sentences against each other, and you can very quickly see kind of the structural differences. So, for example, here you can see that the second one has a much higher temperature, and in this case that's the red one. And it's. It's branching off in places where the first one with a lower temperature does not. Now, another thing is, so I decided to put this on the cloud and then I kind of built on this manipulate to further show some of the differences. So here again is something that is showing us in the constituent graph form, in the tree form. And the red parts are the nodes that are highlighted that are different. So when we toggle them, we can see that when two temperatures are the same, it'll result in trees that largely don't have any red, which means the sentences are actually quite, quite similar in structure. Whereas when the temperatures are different between the first one and the second one, it'll create more parts that are red. So essentially it's either adding or it's subtracting different grammatical structures, syntactic structures, to the types of text that is being generated. Now, I also wanted to visualize this in kind of a more systematic sense because the large language model text generation is largely predictive. One of the things that from time to time it'll also look different. And also I want to get a sense of the broader trends between changing the temperature. So here is kind of a 3D plot of something called the graph edit distance. And what the graph edit distance does is it lets us look at the similarity between tree graphs. So between two sentences, it lets us really quantify. It gives us a number of how similar or different these two graphs are. And the measure is essentially how many graph edit operations. So either deleting, inserting, or substituting either an edge or vertice will have to get from one graph to the other graph. So from two sentences, how to get from one sentence to the other sentence in the least amount of changes possible. So here is that 3D graph and on the x and Y axes are the variations of two temperatures. And then the z axis is the amount of the graph edit distance. So one thing we can see is that, for example, as you have high temperatures for both of them, the graph edit distance is actually lower, and the highest is when one of them is high and the other one is low. So when one sentence is generated with a high temperature and the other is generated with a low temperature, also, we can interestingly see that when the temperatures are kind of the same, it's very, very low, actually lower than if one was high and the other was low. So now kind of going further down, one of the actual problems with this is that when we initially look at these tree graphs, it's counting the words as different nodes. So it's adding that to the graph edit distance. So, for example, if I had my favorite food is pineapple, and my favorite fruit is apple, Even though those sentence structures are the same, it still has a different graph at a distance. So in order to kind of fix that, I essentially just pruned out all of the words that were included. So it only has the structure. And then when we run this again, we can see that this actually makes a lot more sense. Although again, it doesn't follow what we might intuitively assume. So, like, initially, when we're just thinking about it, we might think, oh, the highest graph edit distance will be when we generate one sentence with a really, really high temperature, maybe like two, and the other one is really, really low. However, it actually turns out, if you look at this graph, that higher graph edit distances are found when one has high and one has median, and they're medium temperature when it generates it. And the reason behind this is actually because when we have a medium sentence, there's a lot of times when the structure is different. It actually has to break down this medium sentence through graph edits and then reconstruct a more complex one. So there's actually more operations involved. Similarly, here with the mean graph distance, which was a measure that we included earlier, which looks at kind of the average distance between all of the vertices, all of the words in a given graph, in a given structure of a sentence, we can look at how that varies as the temperature increases. So again, here you can see as the temperature increases, the graph edit distance, the mean graph distance actually will become higher. Which makes sense, right? When the sentences are generated with a higher temperature, they tend to be more random, they tend to be longer, and such the mean graph distance also tends to increase. Again, because text generation is random, we decided to generate a lot more points, kind of trials to just make sure that this trend is something that is consistent. And it really is. You can see that the range of values, either low or High actually increases a lot as the temperature increases as well. Now, one thing that I was also interested in is, alright, we've looked at just a sentence on its own, but what about sentences that build up a paragraph? So between sentences, how do these things vary in large language model generated text? So here it's generating a paragraph. So before we were only looking at singular sentences, now we're looking at entire paragraphs, and it's looking at kind of how the mean graph distance varies over the paragraphs, over the sentences that make up the paragraph. And interestingly, regardless of the temperature that's used, it all follows a very similar pattern where kind of different dips and rises as it varies from sentence to sentence. So here, like around the second sentence, a lot of these models are actually having the least mean graph distance, whereas in the third sentence, a lot of the generated text has a higher mean graph distance. And this is again regardless of the temperature, which is really, really interesting, as this is a pattern that's kind of similar for all of the GPT generated text. And I was curious to see whether this pattern was also able to be found in kind of human generated text, human written text. So I looked at a collection of various speeches, novels, essays, the Declaration of Independence, Alice in Wonderland, JFK's inaugural speech, and as you can see here, within each paragraph, from sentence to sentence, the mean graph distance varies dramatically. So while GPT in that sense is a lot more predictable, the human written content has a lot, much more of a random pattern, which is actually an interesting way in which we might be able to differentiate the two. So again, here, when we overlay them with all of these solid lines being the GPT generated paragraphs from sentence to sentence progressing along the X axis, and the dotted lines being the human generated ones, we can very clearly see that all of the solid lines follow a very, very similar trend, while all of the dotted lines vary dramatically. Interestingly, again, the average of the mean graph distance across all across all of the sentences in the paragraph is actually quite similar between different temperatures, which is quite interesting. Now, when we get to higher temperatures, on the other hand, when we get to values such as like 1.75 or very low values, so any one of the extremes, it will also tend to become random. And one of the things that I actually had to include earlier on was a character stop limit, because when the temperature reaches around 2, you can see that it becomes very random. It just rambles on. And so there's like kind of a max character limit needed for that. So when we reach extremes either above 1.75 or below 0.25, it will tend to kind of go crazy hallucinate. However, in the middle range of those values, the temperatures will create very noticeable patterns and consistent patterns in all of GPT generated text. Another thing that I was quite interested in was looking at sentiment analysis analysis. So kind of we can quantify the positive, neutral and negative sentiment for any sentence, whether it's human generated or whether it's from a large language model. And when I plotted this across different temperatures, I actually discovered maybe not much to, maybe not to too much surprise, that sentiment doesn't really have any correlation with temperature. And that is something that is quite randomized. Now one other thing that's quite interesting that we can look at is something called the grammatical structure frequency measure. And this was something that I was looking into more near the end of my project. But so what, what this measure of grammar frequency familiarity is called is it's able, it makes us able to look at kind of grammatical structures and common grammatical units that occur in text. So for example, I first ran this over just immense wealth of Wikipedia data to kind of look at the patterns in human generated text. So I picked the most commonly visited, the 23 most commonly visited Wikipedia articles. And I kind of just kind of split that down so into all of the different sentences, then into all of the different words. So there were 14,000 sentences, over 300,000 different words that we were training this on. And then again trimming down these trees of all of the sentences and actually it trims, I trimmed them all down to a third level parse tree. So these third level parse trees three units down are what I'm taking to be kind of a grammatical unit per se. And when I ran it over all of the data that I have, I actually found that it followed something called a zip F distribution on a logarithmic scale. And what this essentially is saying is that over all of this text data that I've trained it on all of the common structures, all of the common kind of third level parse trees. So this is an example of one in the text in the sentences will occur very, very frequently, whereas uncommon structures will be drastically more infrequent. And this also kind of gives us a measure, a GSF M measure in which if we take a particular grammatical unit, we can put it on the scale and we can see, see kind of how common it is. So for example, taking a sentence here, female domestic cats can have kittens from spring to late autumn, with letter sizes often ranging from two to five kittens. When we put it on the scale, it gives us kind of this value of 4.09. And when we look at this in turn, it's all the way up here, which tells us that it's actually a very, very common grammatical unit. So this type of structure to the third level is very, very common. So finally, I of course, wanted to apply this same kind of frequency measure of grammatical structures to large language models and that kind of generation of text. So here are kind of iterations of four different temperatures from 0 to 0.75 and then also increased up to 1.75. And these are kind of across the different sentences, plotting the value of the gsfm. And interestingly, we can see that, for example, on the third one here, all of them are very, very similar. However, when we get to larger values or lower values, the grammatical structure and kind of the patterns, the consistency and the frequency of it devolves. So that was kind of most of what I did. I think one of the parts that I was really interested in near the end actually was looking at the main differences, differences between GPT generated text and human text and being able to quantify some of that differences within a sentence and also in between sentences.

Yeah, that was a lot. But I think it was definitely really cool. Thank you so much for sharing all this. Yeah, I don't know, I think just kind of getting a sense of not just using ChatGPT, I guess, for work, but also knowing, like how well it does it and like analyzing that is really cool. So, yeah, good project. So now that you've done this and, you know, with all the other future experiences with Wolfram that you have ahead of you, especially, we know, with being a TA and other things, the research program this year, what do you think is your future work with this project or with Wolfram as a whole?

Yeah, definitely. So I think in this project I was able to kind of look into a lot of the details and quantify a lot of the different patterns that we notice in the syntactic structures of large language model generated text. And actually, one thing that I discovered near the end that I didn't quite get into is this idea of probabilistic labeling. And the idea behind this is the generation of text for large language models is probabilistic meaning. It's essentially looking and picking the highest probability words. And one thing I was actually quite interested in as well near the end was looking into kind of how this confidence measure changes how and how it changes depending on the different part of speech that is being looked at how it changes depending on kind of from paragraph within the paragraph, from sentence to sentence as well, which gives us kind of a lot more insight into the specifics of how it's deciding how to construct it and kind of what types of sentences have the highest confidence. Perhaps those are the most common, maybe not. And I think kind of one of the potentials of this really is a lot of people nowadays use ChatGPT in all sorts of things and oftentimes a big problem nowadays is trying to differentiate between ChatGPT generated text and human generated text. And that's something that's very, very difficult to oftentimes use. Humans reading it can't even tell. And kind of one of the things that came out of my project is noticing these patterns that persist in ChatGPT generated content. For example, kind of looking at how from sentence to sentence, the mean graph distance within a paragraph for all of ChatGPT generated sentences across temperatures was very, very similar, whereas humans was different. And I think one very interesting future area of pursuit is kind of fine tuning this, focusing on specifics and being able to develop a sort of large language model generated text detection type of program, which I know is a very big interest field currently.

Yeah, I think that's really cool. You know, good luck with all your future projects and yeah, we'd love to. Or at least I'd love to hear more about whatever you managed to do with this in the future. Yeah. Thank you so much, Tiny, for coming onto the podcast and sharing this today. It was great to hear about.

Thank you so much for having me.

Thanks for listening to this episode of the Wolfram Student Podcast. I hope you learned something new and took away insights from today's discussion. If you want to be featured in a future episode, fill out the form@tinyurl.com WSP S2 interest and be sure to tune back in a month's time to hear our next episode. Once again, thanks for listening to this episode of the Wolfram Student Podcast.