Enabling Mathematical Search inside Wolfram Mathematica

Hi, I'm Flavio Gonzaga and I'm from Search on Math. During my presentation, I'll be showing how to integrate a search engine for mathematical content using a file inside Mathematica. Good afternoon. I'm here to perform the presentation enabling Mathematical search inside Wolfram Mathematica. Thank you for being present here. As you can see, I'm from Brazil, so I'm still improving my conversation. If something that I said is not clear, please feel free to tell me and I'll try to find a different way to express myself. Okay? Talking about Search on Math Search on Math is a search engine which was designed to search for mathematical content, including mathematical formulas. So using Search on Math, you can build your own queries and mix textile content and math formulas. And this project, the Search on Math project started in 2008 when I was thinking about a theme to my PhD at Federal University of. Later in 2013 finished the doctorate and the first version of SearchMath was released. Later in 2015, Search on Math became a company and in 2021, Search for Math became part of the Head Start program from Amazon. And it was also when we released our first version indexing the Archive base. So this is our brief story talking a little about the search on Math functionalities. When you visit Search of Math, you can choose between using the web index or using the scientific index. In fact, if you visit Search of Math now, you see that our scientific index is temporarily disabled. Why? Because we released the archive version in 2021 Like I said, and this version was only in beta version. For this version we really downloaded the entire archive from the AWS repository and treated all of the latex file and things like that. But now we are preparing next version of the archive where we are indexing the archive for HTML5. So we are still working on it and I believe that our scientific version will be ready in a couple of weeks. But all of this is. But everything that is presented in this presentation is to valid the end considering also the URL to Search on Math for the archive. So basically, as you can see in the figure, you can include in your queries text and or math formulas in latex format. And the only thing that you must take care of that your formulas must be enclosed by these delimiters. So called Search for Math will treat this in a different way. This is different to Search for Math formulas in comparison to just for textual content. So we need to know exactly what part of your query is composed by what part of content. And by doing this, what will happen is that Search of Math will return the results page, and the results page are first ordered by formula similarity. And after that, if we have two different results with the same similarity, then Search of Math we'll use the textual content to define what came what must be showed first and what is the concept of similarity. Well, similarity inside Search on Math decreases as long as search on Math identifies the need of replace symbols. For example, if you search for like this, scene X, Search of math will return first scene X, but it will also return scene y or sin of any other variable or scene of any other maybe constant. So this is the main difference between what search of Math can do and what Google can do. So when we are looking for a math formula, we are looking for the structure of its formula, and we are not thinking about exactly the literal sense of each symbol. So you are free to use your own symbols and certiomath will be able to find good results. And so you can see our results page and here in the left is the formula which is most similar to the one that you have searched. And in the right is a brief abstract of this result. The scientific version, which is composed mainly by the archive papers, users can filter by year by author or area. And the error are the same errors that are already defined inside the archive. So here we have an example where we apply the filter considering year and also for only one author. And here as you can see, you can see the results for the search. What we are presenting here is what I am presenting here is a way to integrate Search for Math search into the Mathematica using a notebook file. The idea is that the users can search for math content without leaving the Wolfram Mathematica. So Search for Math have an API which is available at these address searchmth.com web service API and through our API, any app basically is submit queries and get results in JSON format. So it's pretty easy to integrate search on Math to any other application. And we'll show how to do this in our first version for Wolfram Mathematica, the basic params are query as you can see which is the query that you are submitting. The page that you want to get back from Search of Math, the first page of results, the second and so on Domains you can define in which domains you are searching or not. It's up to you. If you don't specify the domain, Search on Math will submit its for you to its entire index and username key are also mandatory fields that you must informed. So you can submit your queries and User and key. You will get them as soon as you. As soon as you subscribe to the search format. So basically what you need to inform is a query, the user and the key. All other fields are optional fields. And also if you are submitting to search to archive, you must use value X on the domain of the complete documentation page is available@socialmath.com doc and there are also even more params inside of our documentation. So basically what we are doing is that we created some message here using mathematical. As you can see, we are connecting to Search on Math and we are submitting a query here, the Taylor series. And this is the query I'm informing you what page I would like to retrieve. And here are my data, my user and my, my private key. And the same goes to the Scientific index. Well, if we take this and, And try it on Search on Math, I will open. Let me go back here, you can see that, you will see that the results will be the same, obviously. So inside Mathematica what we are doing is that, Well, we are here, like I said, connecting to search on that and we are only formatting the results here, including the title, URL and the common elements of a search result. And here we are informing the query and when we execute this. Yeah, sure, Yes, the backslash are only for escape the slash the backslash from latex because the formula must be from latex format. So I'm just escaping. Okay, and as you can see here, let me show this using Search on Math too. Taylor series, this is the query and. Let me only remove the scapes. And here we go. Okay, so basically this is what we are searching using Mathematica and this is the same query here in searchlnf. I'll log into my user to get the same behavior. Okay, so this is the query that we submitted using Wolfram Mathematica and these are the same results in Search of Math. And so the idea is that. As you can see here, you can see the results and when we go back to Mathematica, you can also see the same results here without the need of leaving the Mathematica notebook. And obviously this is like a normal search engine, so you can click in the result that you'd like to see more about it. And as you can see, you can see the result here in the browser. And also if the result is inside of a bigger thread like here, Search on F will also go right to the correct answer where the formula and where the formula was found. Talking about the domains in our web version, Search on Math is Able to search in all of these domains from Web Method, the LMF and Lab Physics forums. Well, all of Exchange Stack, Exchange Physics. Okay. And Wikipedia in English person. Okay. There is some interesting results in mathematics. As you can see here we have an error. I left these just to show here the first result Mathematica was not able to render here and it's because it's from the Socratic and Socratic uses ASCII Math as a mathematical language, not latex. So since we don't have a way to convert from ASCII Math to Wolfram language, at least as far as I know, we have a problem here on handling this the other results since they are being received in latex. So I was able to convert from Latex to open language and so you can see here the formula properly rendered. What we are working now is that when trying this and submitting multiple queries and using it inside Mathematica, I realized that it's pretty common yet to have some problem when converting from Latex to Wolfram language, especially when we are working with scientific papers and when we are using two specific libraries from LaTeX package from LaTeX. So what we are working is that SearchMath will start to return through its API the results also available as image in SVG file. So so you'll be able to see the results as image in any environment. So you will not need to convert anymore to both language or to think it's the original formula is written in for example asking Math, MathML or any other mathematical language. Okay, so can go through the results and search on method. We started to like I said, we started as a company in 2015, but only a couple of months, two months ago we started to provide to users of search of subscribing plan because obviously we need to expand the company and we would like to index more and more content. And since we started this, we are very happy because Search on Math has been used and subscribed and students and researchers from different universities are subscribing to our services. And here we have a brief list of the universities that students and researchers are subscribing to our plan and using the search engine. Well, the mathematical searches is still an area which we are still researching a lot and finding new ways to search for mathematical content. Like I said, what we know is that the algorithm that we have been used to search for textual content, they just don't perform well when searching for mathematical content. So I'd like to invite you first to try Search on Math obviously and feel free to share your thoughts and to contact us through social networks or even writing direct to me on LinkedIn or through email. Okay. Okay. Then I'm available to answer questions or any possible observation that you'd like to do. Yes, this is very interesting because when you look to all of those domains, they are so different when considering, for example, the size and number of pages, but also the number of formulas. And this is also a big challenge to us because when you are indexing a new domain, considering the mathematical content, there is no pattern or no default. Each site will be using its own, not its own, but different languages and things like that. But the first domain we decided to index in the past when we started search on math was Wikipedia, Wolfram Math World, BLMF and later Planet Math. But for now, if you have some website that you enjoyed and that is not listed as virtual math, you you can ask us and it will be a pleasure to index and to introduce it in our index. There are only one basic rule is that the formula must be be available in some mathematical language mathML or latex or math. We don't have the OCR serves. OCR serves to convert from image, for example. And since we now when we finish the index of the archive, we are thinking to move to maybe to journals or something. This in the scientific area. Okay then, thank you.