Live CEOing Ep 736: Language Design Review of TextSummarize

Okay. Hi everyone. We are doing Wolfram language design review of a new function called Text summarize, which is a function that has been proposed multiple times over the last 20 years. And I think now, thanks to LLMs, it finally becomes realistic to make a reasonable version of it. So how do I run it? Can I run it?

Yeah, if you evaluate init code. Just the whole cell.

Okay, so if I run this, am I going to get the same answer every time?

Well, maybe. I mean usual LM stuff. The temperature is quite low, but it's not zero as it's not recommended.

Summarizing text piece. What the heck is a text piece?

Yeah, we can, we can. I mean this is all temporary text. Basically, this is splitting the text into chunks. And so if it has multiple chunks, I need to improve the message.

Be a computeroid and say one out of one. Just put the.

Yes. Yes. If it is only one, I have to just skip that part.

Just.

Yeah, those are all like fine details.

Chunk one slash, blah, blah, blah.

What do you mean one slash? I mean if these three is one out of three. Two out of three, Third of three. Yes.

Yeah. Right.

I just need. Need to skip.

Why don't we say text chunk one of three. Right. I believe that's consistent with what we do elsewhere.

Okay. Yeah, noted.

Okay.

Okay. So I experimented a bit with prompts.

What's that?

Yeah, yeah, I expanded a bit with prompts. You were last time you suggested to have something that doesn't sound much like. It's like third person. So if you say summary and I believe it's probably is in this agenda. If you scroll down, there's one that has the same tone as.

I don't have that one on the agenda.

Oh, you don't have one. Okay, so you can. You can. You can try. Oh, actually it is in the.

This doesn't look like an extract. This looks like multiple extracts.

Well, extract is sentences from the.

Okay, so then call it extracts if that's what it is.

Okay, sorry. That's because I'm not native speaker. I thought it was a correct definition of extract sentences from the text. Okay, I can put extracts.

No, if they're separately quoted, they are multiple extracts, plural. Okay, if it was in one quotation mark, which wouldn't be Right. This is better.

Well, actually, this should be a list of sentences, but that's also some of the. When I finalize a prompt, I have to figure out why sometimes it works, sometimes it doesn't. That is it should like stuff like keywords and extracts should be a List because you ask for list of sentences or list of keywords. I need to kind of ballot proof the fuzziness of the LLM output and make sure that in every case I.

Don'T think that the default should say the text provides.

No, exactly. You mentioned this last time. If you scroll down the ref page notebook, I have added the easy on the ref page for the Declaration of Independence. I've added another one which is called Summary. We should discuss all these names because I'm quite bad. One is a paraphrase abstract and says the text blah blah blah and summary actually keeps the same tone. So I put it in scope so you can compare the differences between.

But that should be the default, presumably summary.

Yeah, yeah, I can switch. I left. Which one?

The one that is the default right now.

The one default is Abstract. I've implemented that. Not. No, no, yeah, I will switch it.

But I mean, saying the text is. By the way, this is not the right format, right? We have colons and things here. For goodness sake, let's make this.

No, no, I'm using the default. I'm using the default. The only thing that is custom is the text. No, you don't have. In the code that Avery sent you, if you scroll down, you see it in the ref page, you see the result without the. This text provides blah, blah, blah. It's in scope where I use the same input for all of them. So yeah, this one is in this text and this one here, you see this one keeps the same style and tone.

If I want a different length, if I want a different summary, if I want a shorter summary.

Yeah, I'm playing prompt engineering. You see, we have this method compression medium. We can package it in some ways, but it's a delicate thing because you need to describe to the LLM how you, you want it long. So I'm kind of experimenting with this compression level and I don't think it should be too fine grained, something like high, medium and low or something like that. Because you could try specifying the number of lines, but probably won't work. I mean, it ignores it if it doesn't understand that sometimes it works, sometimes it doesn't. For some number it might work, for some number it might not. So I think it's just that promise that we cannot really keep if we put like a specific number of characters or lines.

Well, I mean, we could enforce it and we could send it back to the LLM and say it's wrong.

Yeah, I'm always hesitant to do those things that can possibly involve an infinite iteration with the LLM. I'm not sure about that.

We can always give up after a while. But.

But that's true.

But yeah, but also, you know, the LLM is going to get better at doing those kinds of things and we don't want to design that. You know, I mean, I do believe that's the type of thing that will actually improve. Many things will not improve much, but that will.

True, but we can. I mean, if we have a setting which is, which has like discrete intervals, then we can switch it to a continuous interval like 0 to 1 with not much effort if we notice that it got improved.

All right. You know, I mean, I'm thinking about what this would say if it didn't say the text is. What's it going to say?

Did you see the one. The one that I have in scope?

But that's for cats. And this isn't.

No, no, no, no, no. I have. In summary, it's in scope. If you scroll down to scope, I have it summary for the Declaration of Independence. The, the second one. You see the comparison.

Okay, do other people have comments on this, please? I'm. I'm concerned that in some situations you want a description, the text is. Do you see what I'm saying? In some cases you want an actual summary. We have lots of other people on this call, so somebody else speak up here please. Anybody else have an opinion or not?

I mean, I think that the default you want is a summary and that the abstract version is. Should be something where you're asking for abstract.

I agree with which one is the summary? The summary? The text is or is the summary in the course of human events.

I think the second one is closer to this being the summary. It's got a little bit too much extract in it in this version, if you get me.

Yeah.

You want it to be higher level rather than trying to actually use the language. So I don't think so.

But if it's supposed to be in the style, it's going to probably use some of the language.

Yeah, I did several experiments. It was difficult to fine tune because I also didn't have a super clear idea about what it is to keep short, but also keep the style and the Persona of the text.

I mean the one that's listening to our summary looks to me more like X. You know, its extracts, it's reduced by. But it's still extract nature.

It's very nice. Hopefully should be verbotin. This one is free to change.

It may be free to change, but it's not changing very much.

Let's look at the design that you're proposing here.

Yeah.

Okay, first point is, where's the source text? Okay, source text, fine. It's. Yeah, those seem fine. List of objects. So it's going to produce a single summary of the list of objects.

Yes.

Okay.

Look, so here we, we have.

A big saying that. I mean, text summarize generates a textual summary, you know, generates a summarized form of a piece of text or something. I mean, in. I don't know. Okay, let's go down to this.

Okay, we have one. I mean, at least I have one.

Where's the. Okay, the prompt is something. Go ahead. What were you saying about.

Staying with the input? So one thing that this could digest is the vector database. And solutions is using vector databases for this specific problem.

My proposal was Ken for what problem?

Okay, so you see that the third usage line, I say topic goes to specific.

Yes.

And this is a way to kind of pinpoint you have a lot of material you don't want, like extracts or a summary of everything. You just want about the stuff about the white rabbit in Alice in Wonderland. You don't care about the rest. So that's extremely useful. Because you don't care.

The syntax is crazy because you might just want a summarization of that topic of the text. So I strongly suggest, instead of doing it that way, that the thing to do would be to say text, summarize of text, arrow, topic or something, comma, spec.

No. Okay. Yeah.

Because that way you, you don't. If you don't give the spec, you don't have to worry about it.

Yeah, makes sense.

So then that form, without this, without the second argument is going to be a default version of.

Where am I putting?

Where.

Where am I putting this? Guys, I'm going to put ILD and you're going to. You're going to put it somewhere, the right place.

That's fine.

Okay, I'm sorry, go ahead.

Okay, so the thing is, if you have a lot of text, the default state of the art is instead of doing something like our text search, which is query based on the actual strings, like bag of words and stuff like that, is to embed pieces of this obvious detects and then do matching with the vector database. So my proposal would be to kind of for this, like release this now as it is, because it's very useful. And if you have an external vector database, you can still use a list of strings, syntax and immediately use this function and then work to support the vector database in some format. You Know, one possibility would be to integrate it in text search. You know, have a different backend for text search. I would like to have it like something available language wise.

Yeah, I agree.

Not something inside text summarized. Because it sounds like a broader thing. But yeah, this requires something like we discussed, like fuzzy vector, nearest neighbor, like something that we don't have in nearest. So like there's a bit of technology to be implemented first. So yeah, this I think is kind of a separate design.

I mean there are a bunch of helper functions here. Because what you're basically saying is take a piece of text and is it sentence by sentence or what chunk is embedded?

As much as your LLM can handle.

No, no, no, no. I'm asking, you're saying. But if you want to find the topic, a particular topic for the topic one.

Sorry. Yeah, yeah, sorry. Yeah, for the topic. Yeah, I guess the granularity, if you want it, very granular, maybe. Jason, you had your team as some experience with that.

We did page by page because the LLM can happily read a whole page and find the things on it. So that's typically.

So then you're homing in on pages and you're then going to. Yeah, typically several prompt. So wait a minute, what you're doing is you've got a thousand page document, only 10 pages have everything even vaguely relevant. So you're using the embedding vector database story to, you know, to prune it down to 10 pages and then you're using prompting to say on those 10 pages, tell me about this topic. Is that correct?

Basically correct. And we, we have a parameter for how many things you want it to retrieve. Often it's three or sometimes it's five. You know, it can be as many as 10, but it's not determined. It's not determine the topic that if there's anything relevant on those pages, it's instead there's a fixed maximum number of the most relevant pages from highest to least most relevant to least relevant. And it reads that number of pages and summarizes the subject based upon what it saw on those pages.

Okay, but hold on, hold on.

So if you're, you're reading the top three pages, the top five page. You're reading the top three pages, the top 5 pages, the top 10 pages, depending on how it was called. Right. What, what second argument was given when it was called. That's the number of things that the LLM reads and then it's the prompt engineering is that having read these, you know, answer the user's question about this topic. And that's what the LLM does.

But I mean, so at a lower level, what we potentially want is, you know, something like this. Right. We want something which says. And this would give us somehow. I don't know quite what the right return is, but some kind of thing which is some sort of list that is these vectors are associated with these chunks of text. I mean, it's very similar to the. To the text search index object. Search index object.

And this is why it's futurized in the text possibilities. I don't know how Nick feels about that. We discussed that a bit. But I would like to explore the idea of making this a method around that function.

A search index object. Well, I mean, yeah, I mean right.

Now it's just Lucene. Yeah. But I know I have to see. It is too incompatible.

Maybe not just spent months making this work for embeddings. So maybe he can say something here.

So Swastik is here too, but I'm here as well. I can speak to this. So what Swastik was doing for embeddings related search was. It's not tied into text search, though. It certainly could be.

Why not?

Why isn't it tied into text search yet? It's on our roadmap. It's just, you know, we're trying to get the, you know, we're trying to make it work first before, you know, messing with established functionality. There's no reason that it can't be tied into text search. I mean, obviously that has to be designed and that's what we're talking about. What's that?

I said as a method option.

Exactly. Yeah. I mean there are a lot of pieces to that, like talking about where to store the vectors and things like that. But you know, that's kind of what we're here to talk about.

Right, but that. That can be part of the search index object, can it not?

Sure, I would think so.

Yeah.

Like it can be in memory or out of core and it should probably be out of core. But yeah, these are technicalities mean we should be able to handle that.

Yep.

Okay. But so the idea is to somehow have that be called. So the actual thing that will be called here is you would do this. So you would be saying essentially you're saying, I don't do this. I say create search index of the text method, arrow, whatever it is, text embedding. All right. And that will generate a search index object and then that will. That, that's the thing that's happening inside text. Summarize Is that correct?

Well, you know, if you provide a search index object as an argument, I think it's better to keep this in two steps because it's otherwise extremely long computation.

Okay, I understand that, but can we tell if, you know, if you just gave it a random search index object that wasn't connected to that text, does the search index object have enough tagging in it to know what the text actually is?

No. I mean, there is no text. You give a search index object in place of text. I assumed that stands for text. I see.

Does that work, Brian? Or Swastik? Does that work? Does the search index object contain the text?

It would have to. It does not contain the full text. It can't, otherwise it would be, you know. The search index object for our docs would be the size of the docs plus.

Well, it probably is.

It's not. The doc. Search index is like 20 megs.

Yeah, but why should it contain. Why should it contain the text?

Stephen, usually the indices. An index is about the same size as the text.

No, but I mean, if this is out of core, it just matters. It has to point to some database where the text is. So this would make everything extremely light.

Yeah, I understand, I understand, but that's not the architecture right now for search index object, is it? Or is it. Can somebody just say that, please, search index object here. He's the one who's supposed to be working on this. No. Okay, well, he was invited, but I.

Mean, search index object is. It can certainly live without the corp. Without docs corpus in the same place. Right? You don't need. I mean, we. That's how we ship our product, right? There's a. There's the full index goes with the product, but not the full docs. So you don't need it. It's not just a list of pointers pointing to the text. It is self contained. Now, you can certainly text search and create search index and things like that. You can make it such that there is a field in the index that contains the entire plain text. It's extensible in that way. That's not how. I don't think that's the default. Still there?

Yep.

Okay.

And this is still different from how the vector database formulas works, obviously. I mean, in that you. You do have all the information in the original text, but you've encoded it differently. You've already passed it through a, you know, an embedding layer. And because most of those vectors are sparse, they can be handled relatively efficiently. But you haven't thrown away any of the information.

Right, okay, well, so there is the.

Question of, you know, create or, sorry, create, Search index and search index object are on disk. Those represent, you know, out of core indices. Do we want text summarize to be able to work on, you know, you just throw a piece of text at it and it works in memory?

Hell yeah.

Because if so, then, you know, it's. It's something a little higher level than CreateSearchITex or Search Index object, like search, for example.

This was a proposal Text summarize just sees the text, but the text can be fed by different actors.

Yeah.

Okay, so in this case, the search index object is just the dj. I mean, you ask for the topic and then that provides you with the right piece of text, either with a vector database. I mean, we could even hook it up as it is now using the scene. That will still fit in the same.

Design, but you still want. You just want your design to be that by the time your actual text summarize function sees it, it's already been turned into relevant pieces of text.

Yeah, exactly. But this can be done by the current text search as well. Like it's maybe less powerful than LLM on the whole pipeline, but. Right, yes.

All right, so let's just go through the design here. So I mean, this question of if you already have a search index object, what do you do is a separate question from making this work for a, you know, fairly small piece of text, couple of pages, you know, all in memory, etcetera, etc, etcetera. Right.

Yeah.

Okay, so this is, I mean, to what extent is that topic thing a, you know, a question, you know, can this were the British rotters? Is that what this is supposed to be? Whether that's Declaration of Independence or something.

Or what?

What is that supposed to be and how is it going to be prompted?

I mean, I don't have a definite answer to this because I think that this is mostly what we want it to be. In my mind, this was what and how the topic and the spec find.

Textual answer already has this mechanism for ask a question text comma question. Right. So I think that's a different thing than this. And perhaps the prompting, you know, a text here. What about, I don't know, taxes.

Yeah, this is a lot more how I see it.

Yeah, okay, fine. But then find textual answer can use the same technology.

Oh yeah, to pinpoint. You mean a topic before asking the question.

Yes, for example.

Yeah, they should be parallel, but they mostly differ in what they return whether or not you want an actual answer. Or whether or not you want a longer summary that can be taken from more parts of the text.

Yeah.

Also one thing that we discussed briefly when I started looking at these four possible extension using LLMs. This design is very tight on the answer that is in the text.

Okay.

If the answer is no, you see that word there?

Experimental.

Okay. Yes. I've been told that if stuff experimental is being experimental for too long, we need to maintain background.

This one is so bad, honestly. But I don't think there's a problem. I don't. I think it. You know, I think we should just change it, you know, sorry, new technology came along. We can do something better now.

Okay. We can come up with them with the. With a new proposal for this.

Now, some of this is still useful and will be good to try to get out of the LLM.

Exactly, exactly. I think we can still ask the LLM to provide some context about. And this would be the main difference in fine textual answer.

You know, like, I cannot believe syntax will work out to have this be an arbitrary prompt. I find that very hard to believe. Don't we have a wrapper.

That works? You mean LLM prompt?

Well, it goes that indirects to the repository.

You can use string templates if you want or template object that's always supported by LLM function. So I can disable the. I mean right now all the strings that are not explicit specification, they're interpreted as.

Yeah, I think one's going to want anywhere a string template. Because I don't see how you can do this without. You know, it's gotta be applied to the text.

Yeah, yeah, it makes sense. Again, we can just drop to the text.

It's not doing anything.

That's true. Actually every prompt I have is applied to the text, so.

Yeah, but if it's. It's just a. It's a text and summary.

Exactly.

Guardians of the Galaxy just says Groot.

Which is. No, actually is funny because I forgot actually to hook up the argument. So if you do like, I don't know, LLM synthesize and you say title, which would be the same thing as doing this without the text. You actually get a title for something. It's kind of this LLM hallucination. I don't know if you've ever played with that. So you can say title summary and it gives you something. Or even empty string and it gives you something back.

Oh, it tells you some kind of title for it.

Yeah.

If you see LM synthesized title, then you get a title.

Great. Magnificent.

I thought you knew this is the type of thing that you like. Oh, I got this mental health before.

It likes you had that one before.

Yeah.

Impact on social media mental health. Yeah.

Oh, what's the world coming to?

Try empty string.

I assume these guys have kind of dealt with these corner cases. What's it going to do? Is it going to write? Is it going to write a speech?

It's going to write. It's going to write stuff. Yeah. Stories or things.

Should.

Should still be, you know, checked by the old alignment policy, so it shouldn't be too bad.

Well, at least it's writing about itself. It's kind of like. Like lots of humans.

Oh, this. This is random. This is random. I got stuff like, not related to the AI trying this other times, like stories or things sometimes very specific. Well, sorry I distracted you. We shouldn't be doing this.

Okay. In any case.

Yep.

It's the Dreaming LLM. Okay, let's come back to this. Okay, so the question is, if you say text, summarize a piece of text, I think this is a reasonable syntax for giving a topic. Now the question is, what on earth should the spec really be? Okay, so I claim this isn't right. That. That. Well, I don't know. I don't know how to. You know. Okay, an abstract. I don't know. Most abstracts just start off. I mean, they say things like we show that AI will not destroy the world or something. It does that. AI. What's that?

That's from papers. But yeah. What's the abstract of the Wikipedia? I mean, it doesn't say. We show.

No, it's. I don't know, just a summary. So that's a good. That's a good example. If it's the, like, I don't think, you know, a summary is a summary. That's the type of thing you see at the top of Wikipedia. An abstract is a different kind of thing, isn't it? It. I mean, okay, do we have anybody who's kind of knows about the kind of. We have the wrong group of people for this. The kind of English literature angle on all of this.

Do you want me to see if.

Michael or Alan are around? Allan might be a good person for this.

Okay.

Alternatively, Maybe Mark Long.

I can check in with Mark.

I think that's what we need here because I don't really know what this. How. Okay, so keywords, by the way, this text, you can't say in one of them, verb paraphrase, blah, blah, blah. Another short version of. See what I'm saying? Paraphrasing. Of text, whatever.

If it's okay. Oh, yeah, yeah, sorry. I see the style.

Yeah, yeah, okay. I mean, you know, we could. If we wanted to be cute, we could say TLDR and things like that. But I think we don't want to be cute.

I mean, we also have. We also. We also have the prompt repository.

Yeah, I know.

So we could have LM prompt tldr.

Yes. Which is fine.

Which is. Yeah, is.

I would call this one line or something.

Yeah, I mean, this is more like a suggestion. I feel very uncomfortable doing this because this is not my language and I'm kind of. I tried to read the stack exchange for English about the differences between abstract, synopsis, summary. There seems to be a lot of synonyms here. So the fine details are really lost on me.

Yeah, right.

One common way to add to another additional potential thing could be like an itemized list if we are thinking of adding things here. Otherwise I think that can be handled through the prompting.

Yeah, why not? I mean, itemized or something. Or something like. Yeah, a bulleted list is a nice idea.

Exactly. Sorry, my bad. Onkar already suggested that. I forgot to add it here.

Okay.

Yep.

All right, let's. We have. Mark joined us. Okay, here's the context. We're looking at a function called text summarize, which summarizes text using LLM. What we want to talk about is in kind of English writing usage, the difference between an abstract, a summary, a. I don't know what else you call it. I mean, the. The kind of. The way you write an abstract. Mark, do you have a. Do you have something you can say? About what? What. How would you characterize the difference between a summary and an abstract?

Well, a summary would actually cover all the text or all the content that was in there in a concise manner. An abstract is more like an executive summary in technical writing, which basically hits on what your thesis or point of investigation is along with your terminal conclusions. But the summary would have everything in between too, in terms of all the evidence and line of logic.

Okay, so, okay, so that difference in usage is. An abstract is what's the main point? And a summary attempts to cover everything in some way. So, for example, an abstract, which we have here maybe of the Declaration of Independence would be kind of, you know, wherever it is, somewhere down here. Okay, I don't know whether that's a reasonable abstract, a difference in the summary. I mean, this is. Okay, the other problem is under what circumstances should we be saying the text is. How would you characterize the difference between something that Says the text is blah, blah, blah, versus in the course of human events, blah, blah, blah. So I'm saying one is a descriptive is about the text, a summary of the text from the outside saying this text is. And the other is a summary kind of from the inside that just says this is what we're saying.

Right. Well, the Declaration of Independence is written in first person plural. But I wouldn't necessarily write a summary like that though. Yeah, well, I just think that the summary, in essence everything that's in your abstract is in your summary. It's just that your summary will address every single section. A major point of evidence.

Remember, we've only got, let's say we got three sentences. You got to fit whatever you got to fit in three sentences where we're pegged on the length, let's say. So then, I mean, you don't get to, you know, if you have a thousand page book, we still have a, you know, four sentence summary or something. And whatever it is, similar length, abstract. So if, you know, we got to be able to define what's the difference?

I mean, okay, so if you were talking about a book, the abstract would be the point that the book makes. The summary would be the narrative structure of the book.

Okay.

I mean, in a general sense.

Okay, all right, so then, but in that distinction, okay, which hopefully is understood by the LLM, that distinction, I think there's a good chance it will understand that. So then the question would be. In, you know, which one of these, or is it a different thing entirely says descriptively like a review, this book is garbage, as opposed to another one which would say, you know, would you just launch and say what it has to say?

A review. A review would be different than a summary or an abstract.

No, I understand, but, but you understand the difference, the distinction I'm making. Does it start this text is. Or does it just launch into, you know, I mean this isn't a very good launch point here. But, but you know, or even, even just take off the words in this text and just start. The representatives of the United States, blah, blah, blah. In other words, which statement of fact. Yeah, okay, but what is the, what is the thing that starts in this text, blah, blah, blah, like a book review would say in this book, blah, blah, blah. How do you, what do you call that as distinct from a summary or an abstract?

I would call it a description, but.

Okay, Mark, do you have any comments on that? I mean, it's some kind of description review, something?

Well, I would say that a summary starts with the beginning of the book or the article where I would say an abstract would almost start with your conclusion, because executive summaries, which is kind of like an abstract, start with your conclusion so that people know exactly what the whole thing is about.

Okay, so wait a minute. You're making a distinction between an executive summary and a summary?

Well, technically, yes. In terms of writing technical reports that begin with executive summary, which is different than just like a book report summary. Executive summary tells you what the. It actually tells you what the conclusions of the paper are.

So.

And I'd say that's an abs. That's in essence, an abstract.

Okay, but so you're, you're claiming executive summary and abstract are. Should they be considered synonymous or are they. Should they be distinguished?

I would use them as synonyms almost, but I would differentiate between a summary, because a summary is like a book report that just walks through step by step from beginning.

But I mean, we're relying on the LLM understanding these distinctions, which I think it will. But I'm coming back again to my question. Does it say in this text or in this book, blah, blah, blah, or does it simply launch into, you know, you know, the summary is, you know, cats are wonderful animals or something? Does it say in this text?

I don't think it has to.

No, no, I understand. But I'm asking, is there a form in which. First question for folks like Jason and so on, is it useful to have something that says in this text, or is that always just fluff?

I mean, I think it's mostly fluff, but if you specifically ask for a description, you want somebody describing it from the outside. I can see wanting that. But it's a specialized use. It's not the usual case. The usual case is you want it to, as you put at launch with the subject matter itself in the long summary form or in the short abstract form. But if I want from outside would be description.

But by the way, I mean, in. In this thing, like short summary, you know, that that's not a bad way for us to name these things. Instead of that whole, you know, thing with short, long, Whatever, whatever, whatever. I mean, we could just say, you know, short summary, long summary, short abstract, long abstract. What do you think, Julio, as a way.

Yes. Yeah. Possible. Possible. Yeah. Even easier. More like more clear, less obscure.

Right.

So also because some of those, they are like inherently. I mean, they have a length already. Like, you know, the, the like title has. Is. Is one sentence.

So we've got, we've got title. We've got. Now keywords will Be in order of their, you know, keywords will be roughly in order of their importance.

Well, we, we can ask right now. It is in order of like how the text reads. So depends on what you want. That's also as a usefulness because you just can basically imagine what the text is about by just reading the keywords. We could also try to solve that different.

Again, that would be like a keyword summary.

Okay.

Keywords would be just the list of keywords, I think. You know, if the, if the thing starts with cats and ends with dogs, but there's a lot more about dogs in it, I would argue you should put dogs first in the list of keywords. See what I'm saying?

Yep, yep.

I know where the keyword summary is the right term, but that would be keywords that are, you know, kind of sort of, you know.

Does it make sense to have a, a version of that function which would have like frequency attached to it so it's already ready for like word cloud.

Well, like keyword and score something like.

This or scored keywords, something like that so that you have a list of them with a, a keyword score pair that's suitable for people.

Oh, yeah, yeah, sorry, it wasn't the string suggestion. It was. Yeah, yeah, yeah. So in this case we would have like a list of rules as an output or something like that.

Maybe, maybe I'm, I'm a little bit concerned about output forms that aren't the same.

Okay, so you, you would like for all of these to be a single string?

Well, actually that's not realistic because some of these are lists. So I, I take that back. I'm, I'm retracting that statement. That means this would be possible to have scored keywords that are different.

Yeah, this is something that really I envisioned it will require like most of my time to make sure that the outputs can be coheres in the. Into the right. Into the right shape. Because.

Anyway.

So, okay, so fine, so back up here. I mean, we could also say.

You.

Know, given that we're having short, long summary, we can say one line summary, which might be different from short summary. I don't really know. But. Okay, so what will we call? Okay, so you've suggested itemized list, but I mean, I don't know. What, what is that called? Is there some better term than bullet list? Mark, Anybody on call who suggested this? Anybody else?

Bulleted list, highlighted list highlights.

This could be.

One of the most useful things that you can. If you can have highlights with a number. Right. I want that I Want five bullet things or no more than five bullet things. Some of these prompt engineering. But if you can ask for it to be summarized in this many bullet points.

So. Right. So I mean, so if we say numbered items, if they're numbered, I mean, I don't know, do you actually want the number in the text or.

I don't particularly need. I don't really need a number of the text. The main thing is that you can coerce it to have only the number of.

But we don't have a mechanism for that in what we've been describing so far. So unless we have a. Unless what we do is we say that this thing says text summarize of. You know, again, the syntax would be text or some way of specifying text. Possibly text, arrow, topic. And then here, you know, spec or potentially we could do spec, arrow size or something. See what I'm saying? And that could be. See, I don't know. I mean, what we could do there is say by default an N, which is sentences. And we could have a quantity object there. You know, quantity, you know, whatever it is, 10 words, 100 words or something. And the LLM is not going to behave itself every time. Why didn't that work? I thought that was a valid unit. What the heck? Okay, there's a bug there. Somebody report that, please. Yeah, I can do that. Okay. So, you know, there is a unit there. People understand what I'm saying. So. So you could say a spec, like abstract, for example, arrow, 200 words. Does that make sense?

Yep. Related question is, can you have a list form of topic? Can you have it like three topics? Two topics?

Do I not. Yep. I mean, it's going to. What it means by that is I want stuff in the text about either elephants or walruses. It's not like I want only the section that's about elephants dancing with walrus.

Sources.

Correct.

If you give two. If you want to give.

No, no, no, no, no, no. You've got to decide what the. What it means if you give two. Right.

I think we. We had done in the past that least was. And. And alternative was or.

Well, that's for some things. But I don't know whether that. That doesn't seem to make a lot of sense to me here.

Yeah, especially because your. Your actual topic there isn't forced to be something atomic.

Right.

You could have your individual topic be something which is a word of your thing that says, you know, only dancing with.

Yes. Okay. I mean, by the way, Sandra is pointing out on Live Stream, what about Things like, you know, passive voice, formal, etc. Etc. Etc. Those are things where you've got to be able to add in the prompt. Right? Those are, those are modifiers effectively. Right. You should be able to do the modifier prompts.

Okay, so you want, you want to chain the prompt with the specification, right?

I'm asking for the syntax for a modifier.

Yeah.

What is the modifier? What is the way you do a modifier prompt in LLM synthesize?

They are chained. If you put a list of them, they are chained together. I see, so you could say LLM prompt passive voice.

Right. So that means the spec here, which you're saying could be either abstract, blah, blah, blah. So what it would look like is it will be this and then it would say abstract comma LLM prompt formal, right?

Yeah.

And I mean as a courtesy, we might try to look up the strings. I don't know if that's a good idea.

Yeah, we might time our hands there. Like it's very clear. If you use it through a LAMP prompt.

Yes.

I'm thinking there's another, another possible usage for daily syntax, but it would, you know, supporting multiple specs and returning like a association of those would still involve separate codes, so I don't see any immediate benefits of.

All right, so Flask on the live stream comments that the executive summary should be something capable of being read on its own, which I would agree with. Sorry.

Should be something that capable of being.

Read on its own. You don't need the document there, obviously.

Oh, okay.

For tech summarize in general, the whole point is give me a summary so I don't have to read the document. I mean the TLDR story. Okay, all right, what else do we have to figure out here? I think we made some progress in defining this. What else do we do? We need to work out.

Well, this is already a sizable amount of stuff to update. So I, I think it's, it's a good, very good direction for this function.

The only question I suppose would be, are there, you know, people going to understand summary versus abstract? And we're going to have to say in the documentation some, some summary of what we've been saying here, namely, the summary carries the whole narrative. The abstract is only the punchy part, so to speak, is only the, you know, what the point is supposed to be. Summary slash abstract slash executive summary is what the takeaway should be, not what the narrative structure is or what the supporting, you know, the fact that it has. You'll just say, you know, walruses are the best animals in the Arctic. That's the executive summary. And the summary might be, you know, a study is done of all the different kinds of animals in the Arctic and it is concluded that, that, you know, walruses are the best because polar bears, you know, don't, aren't capable of doing this or that or whatever it is. Does that make sense to people?

Yes.

Not the part about the polar bears, which is irrelevant. But, but the, but that distinction.

You know, I think it was Mark who brought up description and I like, I, you know, I like description as maybe something different. You know, if you threw the Declaration of Independence at it, wouldn't it be nice if it could tell you this is the Declaration of Independence? This is, you know, this were, these were the ramifications of this document it was written in. Blah, blah, blah.

Well, yeah, but that's asking it, that's asking for the LLM to make random stuff up.

Yeah. All the information needs to be in the text. Right. But it would be bad if we make things up.

Isn't such a bad thing. That would be. This text is blah, blah, blah.

Right?

Yeah.

I mean in, in the use case where you're just feeding it like top three or five pages. Right. I don't see description being a very useful thing.

I don't agree with you. Think about a corporate setting where, you know, you've got a bunch of documents and it's like this document is, you know, it's something says this document is a thing sent to the HR department complaining about this type thing. Right. As opposed to, you know, Joe said the plant in his office was trying to eat him.

Right.

Which would be a different. You see what I'm saying?

I think the description, description used does make sense there. I think the thing Ankara is trying to get at is if you're already feeding it only extracts may not be able to do the descriptive summary. The whole thing.

Well then that's, but that's the whole point. It's. This is what a super function does is if you, if it, if you're asking it for the description, it's going to find what it needs to find. Yeah, but I do like the idea of breaking out the concept of a description, which is the meta thing saying this text is the Declaration of Independence or whatever. As opposed.

Yeah, I agree.

You know, getting into the meat of it now. I mean, let's just think about use cases here because you know, for example, let's say I've got email coming in, a whole bunch of email coming in actually description might not be a bad thing to ask for. You know, this is a piece of mail from somebody who wants to invest in a company. This is a piece of mail from somebody who's trying to sell us something. This is a piece of mail from somebody who's trying to buy something from us.

Sure.

And you can definitely imagine using that for sorting and so forth before you even look at the summaries.

Right. And then, I mean, for summaries, you know, we know we're already using something, and I assume we know what the prompting is for that. The summaries we're using for the. A digest of LLM preprints that Michael is generating. Julio, do you know what the prompts are for that?

Not. Not really. I'll ask.

You know what I'm talking about, though, right? You see these every day.

Yeah.

And they're quite good, in my opinion. Right. I mean, I don't know. I mean, I try to read them.

Yeah. Not. Not. Not all of them, but. Yeah.

Well, I mean, let me pull one up. Let's. Okay, here we go. So this is the actual summer abstract from the paper. Oh, wow. So that is probably a generated keyword list. Notice also this fields list here.

Yeah.

That's like a categorization.

Yeah. I would call it tags, but yeah.

The point is that.

The point is that. Go ahead.

They're a bit more computational than just keywords because they're. They're not arbitrary tags. They're taken from something like a structure of science.

I wouldn't count on that. That they're taking from anything, actually. I mean, this. But I do like this. This concept here. Okay. In any case, that. So go find out from Michael how he's generating these things. Because, I mean, clearly this. This thing that we're getting here and. And I. I know, you know, in generating these summaries, we're going from a PDF, I think. Am I making sense?

Yes.

So that's the thing that you got to know how to do. And I don't know, you know, I assume it's stripping all kinds of stuff with formulas and so on that it can't handle. I don't know whether we're importing.

Well, can't we import PDF as text? I thought we could.

We can.

Yeah, we can, we can.

Ideally, that's just really stripping everything that is not text or making it text.

Right. Okay. Well, anything else to go over in this meeting?

Maybe one thing is, like, I put this property argument here. I'm not sure if it is super useful the only two I couldn't think of is. I mean, it was to access like the thing is, if the stuff is too big for the context, you need to iterate. Are you interested?

Property. Where's property.

Possible values for Prop R just above the options.

Where. Where is the property? Where does it get put in?

It's just a third argument.

I see.

It's. Let me. I put it there initially because I, I was.

I understand. Yeah.

Realized we didn't have any.

Sorry, you're worrying about chunking.

Yeah. I mean, what if you want to get the intermediate results also for like kind of understanding what's going on and maybe fine tune your prompts or stuff like that?

Well, I mean, maybe there should be just a thing here that you only use if you're trying to debug. That's called text chunks. See what I'm saying? I mean in the, in the things you can ask for don't actually summarize. Just give me a list of text chunks. Not sure how useful that is. I mean, that's just going to depend on, you know, it's various. LLM ergonomics of how big the chunks are.

Oh no, no, no, no. Sorry, misunderstanding here. The thing is, if you ask for a summary and you have a very long piece of text, you need to summarize the chunks and then summarize the summaries.

I understand, but if the chunks are separately summarized, you can ask for text chunks and then map text. Map text.

Oh, okay, I see a point. Yeah, yeah, yeah, yeah.

I mean, if you really want to know what's going on, I mean, I predict that we're not going to do it that way. I predict that it's going to be more sophisticated than that very quickly and that we're not independently going to summarize every chunk and then summarize the summaries.

So to speak, that just kind of hopefully should be a fallback case if you put in a very big string and you are not using any search or topic like syntax.

Yeah, I understand. So that means that this kind of debug escape valve seems worth having but.

Not super highlighted, like as taking a whole argument.

Yeah, right.

Makes sense.

Because I mean it, it's because most of the time you're not going to want that. Okay, does anybody else have any other thoughts about things that would be, you know, immediately useful? I mean, I really, you know, this description thing is a really good idea relative to summary and abstract and so on, because it really is a different thing. So what other use cases, you know, what we've got, I mean, in the case of the, you know, the, the daily LLM abstracts is very clearly. I just want to know what the heck these papers are about. It's very clear point. And I don't want the description would be this is a paper that does blah, blah, blah. I don't want to know that they're all papers. Right. So what isn't. What are other use cases? Jason Ankar, anybody?

I mean, I think that the use cases that we want in solutions are going to be covered by this design. I don't see other things missing. I can think of some neat example to make using it after, you know, the whole thing is working. Ones we've had is use the text, summarize with bullet points with questions to automatically generate a slideshow. Right. So those sorts of things we, you know, do some notebook programming downstream of this. So you get sort of automatic document generated generation of the slideshow of what was our use case. All the legal proceedings that a company is going through in a particular quarter, for example.

Yep. But so.

It'S the same kind of text summarized with a question or set of questions. And you want to be able to do it as bullet points where you control the bullets producing.

You asked for something specific there with numbered items. And what you're now saying is you actually want something. If you want to put it on multiple slides. I doubt you want one sent. I mean, first of all, you do not want the stupid numbers embedded in the.

I don't want the numbers. The usual case we had was you're asking for each specific question, right. You're getting the five, you know, three to five bullet point answers. And then that goes on a slide with the question at the title of the slide and the answer is as the title. It's on the slide.

I see, okay. But let's be clear on what you want for the bulleted items. Do you want a list of items or do you want a thing where the text says 1, blah, blah, blah, new line, 2 blah blah blah.

Yeah, it doesn't, I don't want to say 1, 1, 2. I just want it to be the, you know, bulleted items.

You want it to be a list of items. You want it to be a list which is a bunch of sentences. So let's see what the name of that should be. Okay, what should the name of that be? It's definitely not highlighted items. That's definitely not it. It's not numbered items because that suggests there's a number there.

I mean, is it not Bulleted list.

What is it?

Bulleted list.

I don't know. Because that might suggest there are bullets there. Does Mark have a suggestion or anybody else? What would you call the 1, 2.

3 unordered list versus ordered list are used in HTML if we want to be.

What about topic list?

It's not topics. Yeah, that's categories, which is a different thing that that might be. That's the thing. Categories or, and, or topics are yet more different things you might ask for. I mean this is something we've asked, we thought about for a long time. Topic modeling. We, you know, considered that like 15 years ago. We didn't do it because we thought that the target was too mushy. Okay, so what is the name that you give the. You know, what is the thing that. I mean, is it called a bulleted list except that doesn't have bullets? What do you call it? Itemized list.

Yeah, or bullet items.

Well, you're describing the whole thing. So bullet items would be. Well, maybe itemized list is the best thing for right now. Hey, Mark.

Yes.

You're the most likely person to come up with something for this. What would you call this?

You know, to be honest, I think itemized list is about as good as. I mean, that works for me.

Is there a term for it in, you know, official technical writing, ish type stuff?

Not really. I mean basically in technical writing you're gonna have bulleted lists or numbered lists. Numbered lists are used for a sequence. Bullet lists are just used for unordered items. And that's about.

Is.

The biggest amount of structure you're going to get between those.

Okay, okay.

You know, here's kind of a different thing. Would we care about like year or any like time? Like when was this written or about what time period is this written or is that too.

Well, it's a different esoteric, different thing. I mean really, when we do care. Summarize. Go ahead.

I see. We do care about it, but it's usually a fine textual answer is kind of separate question about the document. Right. It's not part of the summary to me.

Right, Sounds good. Okay, anything else to cover here? Okay, thank you. The folks on the live stream for their comments. Thank you all here. Okay, bye.

Audio ship.