Import/Export: Updates, Applications, and Architecture

The intro to this talk is, well, I've given some variant of updates to the import and export framework, as well as, you know, any additions to formats, new formats, etc. Over the past six years. I think, and I can honestly say that this is the most optimistic I've been about the future of working with data in Wolfram language. So we've already seen the abstract. Wolfram language has been built to work with all kinds of data, and there's a lot of different data sources that we are interested in working with. The point of Wolfram language is to build everything into the language so that you're not importing all kinds of modules or learning how to set up some kind of isolated environment. The language is built to just work. And import and export has always been striving towards this goal of ultimate flexibility and consistency. If you look at working with data in other languages, things can feel fragmented. And our goal has always to build a sophisticated language that has, you know, technologies built into it that nobody else does, but that still is able to communicate with the outside world. This talk, I'll specifically talk about what's been going on in import export recently, but in the past I'll typically go into some kind of a demo and show an interesting application of one of the newest formats and one of the cool things that I've been able to do with it. This year we're going to dive even deeper to the inner workings behind the projects that I'm working on. I'm really excited about about and something that you've probably already heard tidbits of here and there, if you've seen any of the other talks related to data science. So let's get right into it. We're going to talk about new and updated formats and then what my current vision is for how we might publish and connect to data in the future. So right off the bat, document formats, I think that we're going to see some people that are pretty excited to know that we are introducing Docx. Docx is a format that has been, you know, it's the kind of format that you learn rather immediately and sort of kind of an introduction to computer course for, you know, a very young person. Because you're drafting documents, those are the kind of documents that you might submit papers in at school. But when it comes to actually semantically understanding that information, we wanted to make sure that we can release something that is practical and computable. You may have noticed that when working with Word documents that it doesn't always look the same across every Single viewer. And we did our best to address those kinds of challenges, but still make things computable. So you'll notice that the elements for Docx include a number of metadata related things, with a focus here on formatted text and plain text. So we have a Docx file here that I'm going to get the description of, which is some metadata in the file and the description of Jabberwocky. Docx is a famous nonsense poem written by Lewis Carroll. Okay, so that's a. That's a fun description. Let's go ahead with the default element and take a look at what we get for the plain text. Just go, let's grab the first 10 lines and this seems like a fairly sensible stanza to me. I'm not going to try to read it, but I do think that this is a cute example. And of course everybody is familiar with the Jabberwocky project. So next we're going to move on to formatted text. Formatted text is actually importing these things with style directives. I threw in a little function in here to make the text look a little bit bigger for the scope of this presentation. But the style directives inside the text objects will actually contain the exact font size and other information that was extracted from the Docx file. And we can also extract some images from it. In this case, this, this nice poem was about plums and there is a picture of some plums in the vein of document formats. Previous years you've seen dramatic improvements to particularly PDF in this realm of document formats. We want to also start looking at ebook formats. It's pop popular amongst the company and it's popular amongst our users. So the first format that we've targeted is Mobi. And to give you just a little bit of a historical reason why this is something that is widely supported and has a good online open source community for. So it was an easy target, but also something where our contributions we think can really make some sense and have some actual impact. We're going to continue to expand on these types of features to more formats. And in this version we chose Moby for the first one. So like we mentioned, we have some import elements here. As you can see, it's very similar to Docx where there's some kind of meta information that's particular more to an ebook rather than a document format. In particular in the sense that there's kind of a publisher associated with a book, where that might not necessarily be the case with every single document file or maybe not an ISBN. So here is a Moby File, which is just a short book with one of my favorite poems, oh, Captain, my Captain. And you can grab the text from a MOBI file and some images in a MOBI file, and perhaps even some meta information or the COVID of a MOBI file. And so this is just another document, and we grabbed the COVID image of that. RDATA and RDS is the next addition to our statistical data formats. So previously we released some formats that are able to work with the SAS and SPSS programs. They are exporting data in a particular format. And of course, you want to be able to use all of the advanced Wolfram language capabilities on your existing data. So we did that for that suite of software. And now we're also looking at the kind of data that's produced by the R ecosystem, which would be RData and RDS. So these are just a couple examples of an RDATA and RDS file where we can import that content. Now, moving on to another rather important format in the system, spreadsheets and tabular data is a big theme in import export, generally, because CSVs and Excel files are just kind of all over the place. We are also moving more in a direction of creating columnar data structures that are efficient, and we're going to get to that a little bit more. But we still have this existing support for Excel that is extremely popular and widely used. And so I've personally been the developer for this format and done everything that I can to make sure that we're targeting features that are actual needs. So let's move on to named ranges. And I see a question about dxf. We will get to that question towards the end of the talk. Now, the elements for Excel now include something called named ranges. If you're familiar with spreadsheets, named ranges are something that you can do in the Name Manager. If you adjust the formulas behind the sheets, you can assign some kind of scope that refers to a particular set of cells in the document. So in this particular sheet, I just took A screenshot of two named ranges, AB and BC, where AB is the values here, 1, 2, 4, 5, 7 and 8, and BC is similar for the remaining values. You'll see that while these could be scoped either to the workbook or the sheet, they still have a reference to a particular sheet. And that'll come into play here when we look at the output of named ranges. Because this refers to a particular sheet, these things are all grouped together in the same kind of way that it's structured in the actual file itself. And so while this has kind of a workbook level scope, that's not information that is directly relevant to particularly getting the range from a sheet. So the goal of this element is to get the range from a sheet and we can assign that range, you know, maybe ab from sheet one. And then if we're extracting data from sheet one, we can pass those directly as sub elements to import and get back just those values in that range. So named ranges are a popular feature in Excel users now. That's something that you're able to import. Another big improvement is missing detection. So again, because I developed this, I have a little bit more of the history here. This div 0 Excel file is just an Excel file that contains a formula 1 divided by 0. And in Excel, if you have some kind of an error in your formula that it's titled an error cell. And so those things have been interpreted as dollar failed because it's not interpretable. It doesn't actually return data of any type. So what do we do with this interpretation? I think that that was a slight oversight on my part in particular, and these things rather than an actual failure mechanism, but it's just something that wasn't able to be computed, so it's missing from the data. Now, if you import these things in future versions, those will show up as missings rather than as a particular dollar failed. So we're also cleaning up some messages. You're going to get a lot less messages using Excel in the future. And if we look at that, it's the same formula. Another big feature added to Excel is rich text. You can't see much of the interesting contents in this because import by default. Default is just looking at strings. But if we look at the formatted data, much like you could do in a. Docx file or an Excel file, you can, much like in a.docx file in Excel files, you can have text which has different styling within one word or one kind of like string of characters. And this support is something that wasn't even available in most parsers that are even out there. We've pushed for getting as much details as we can out of the file, constructed a way to put these things together and actually return these with style directives that are accurately representing the contents in your file. And we know how important styling directives are Docx and xlsx. Now, this updates applications and architecture. Usually I'd have a bigger example for you in terms of applications of these formats, but I recall a conversation that I had last year around this time with somebody who may even be attending this Talk, who mentioned that they were able to look at particular parts of a book that they had in a PDF. And because of the formatted data, they were able to make certain inferences about where certain chunks of semantic information started in the file and ended in the file. And that formatting information really does unlock an entire world of semantic analysis in terms of, well, it's. It's formatted to look visually to indicate that something is different there. So we have an emphasis on caring about those types of details and preserving them intact. So that. That's just my very quick spiel on applications. We care a lot about formatting data exactly as it's represented in the files, so. So that your resulting work is faithful to the content that you imported, but also flexible and does what you expect it to. I fixed over a dozen bugs in xlsx, and most of them were fairly minor, not things that we get. I don't think any of them have been complained about recently, but I've done my best to make this project as polished as possible. The video team has added some cool subtitle functionality. So in Video Combine, you're able to take this video of a bird. I would assume that it's a bullfinch bird based on the name of the file. And we can video combine onto the video some text and looks like we just took a table for the duration of the video. We're generating a random styled text that either says troop or tweet with a random color that's scaled at some range. And if we take a look at this video, nothing's there because I haven't turned on the subtitle track yet. But in the video GUI there is this. And I know that's a little small, I'm sorry for that. And now you can see these subtitles are overlaid on top of it. We got chirp, chirp. Oh, there's another chirp. And it's a cute way to add some content to your files. You can do the same things in import and export. So import. The video functionality has been growing and expanding into one of the most organic ways that you can interact with that type of data in the world. Lots of people are using FFMPEG because it's a good system for video editing and video transcoding, etc. But it's not always the most friendly user interface, especially if you have to have kind of a certain understanding of terminal commands or whatnot. The point of Wolfram language is to bring these computational concepts to your fingertips immediately. So if you want to do Something like export subtitles. If we have a GIF of frames that we want to add some subtitles to, we can go over those frames, find which interval in the video that the frame would appear in in terms of seconds, and overlay that subtitle onto the frame or not onto the frame. These are being exported to the video as an actual subtitle, so these aren't being overlaid. You'll see if I press play, they're not necessarily there unless I go and select it from the track. So now you can see these are actually a subtitle that is provided to the video object. And you can also import the contents of subtitles, including their styling directives. Okay, so this is already taken longer than I had hoped. I have quite a few more examples to go through when it comes to Excel setting properties such as cell alignment, cell borders, et cetera, et cetera. So you're talking particularly in export. I've primarily worked on import, Excel export functionality is something that we do know that there could be some additions in terms of flexibility to, and those are good suggestions. I'm going to go ahead and take a quick note of cell alignment, cell borders, and make sure that if it's not already there, that it's at least on an agenda somewhere. So, moving on to Publish. This is very work in progress, and if you have questions, feel free to kind of drop them in chat whenever at this point. But let's go into a bit of the future of how I see data exchange working with Wolfram language. So I'm working on a project that's tentatively called Publish. It's very work in progress, and anything that's shown here is completely subject to change. But the goal here is that we want to take content that was created in Wolfram language and share it with the world easily. We want to support a wide variety of services so that depending on so that you can get your content to the right viewers, essentially. And we want to build an architecture that is easy to add to, but also maintainable. We want to build a system where, because this is communicating with the outside world, this is something that we can keep up to date. If you've had a chance to see Julio's talk on kind of the future of API integration, a lot of those efforts are going to be leveraged in these upcoming super functions that are able to take a wide variety of Wolfram language types and also look at all of the different types of data that we already know how to communicate with and combine these things together in a new efficient architecture. That sets the groundwork for something that is always going to be consistent, but also controllable. And like you would expect from Wolfram language, it's going to be pretty smart on its own and give you the flexibility to describe exactly the type of content you want to publish. So the example that I chose for this demo was LinkedIn. And I have a selfish reason for LinkedIn. It's the only social media that I have. And if you'd like to connect with me, here's a barcode image of my LinkedIn profile, or you can just take note of my name there, it's in the notebook. But because this is the only social media that I'm even remotely familiar with in the year 2023, it was the one that I decided to use for testing. So we have some prototypes of this that I'm not able to share the actual execution of just yet, but I did take some screenshots of the kinds of things that you might expect. So imagine that you want to publish to LinkedIn some text. You'll get a text post, which seems pretty sensible, but we also want to be able to publish a wide variety of Wolfram language types as we add them to the system and have those things kind of automatically understood. So you could ask to export or publish some text and some video to your LinkedIn post. The point of publish though, is that it's going to have smart default behaviors. This isn't going to be the kind of thing where you need to structure an API, request yourself, or understand the intricacies of how to communicate with their servers. You know you have content, you know you want to get it to this destination, and we do the rest of that work for you. And it's built upon this premise of data is exchangeable and convertible. So it has built in type conversion in a way that is going to be mostly predictable and also controllable. So we should have sensible defaults. And if you don't like how things are being converted, you're always able to override and control the conversion functionality for your content before it's published. In this case, I took some text and I told it that I wanted that to be the subject of an image. For the purposes of this example, I just have any sort of text to Image conversion is done by rasterize. So you get a post with some text in it and also an image with that rasterized text. So publish is something where you can take data of all kinds. You can be specific or you can rely on the language to be smart enough to make the types of conversions that are necessary or construct the types of API requests that would normally require the deep knowledge of a web programmer. And lastly, in the future, we are talking about something called Connector Data Connect. So originally Publish would have been released in the next version, but we wanted to stop and make sure that as we're building this architecture, you're going to notice that I copied this bullet point from the last slide. Exactly. As we're building this architecture for connecting to data, we want it to be tightly coupled with the same types of architectures that we use to publish data. As we're moving towards these new systems and these new data structures in the language, we're going to provide tools that have the exact kind of flexibility you've always expected from Wolfram language, have the tools that nobody else was able to create, but now is also able to efficiently work on data, no matter where it's from and the point of connect. So import. You can import from a URL or you can import from a file, but once you get your content, that's what you've got in your notebook and you have to figure out some way to save it or whatnot. As we move to the future, data isn't just living on your computer anymore. Data is online. Data is in databases. There's so many ways that you can access tabular data that is particularly useful if it's structured in a column format. So if you've seen some of the earlier data science talks given by Piotr and by Nirmal, you've already seen some examples using the architecture that the new connection technology that I'm building is going to be based upon. In addition to that, I'm developing these types of connection architectures tightly with the type of live data visualization that you might have seen from. I believe Jason gave a talk on this. And so Table View is something to look out for. Tabular, or whatever it's being called for now. And subject to change is also something to be on the lookout for, because these are the foundations that we're building an architecture so that we can do all of those great things that Wolfram language has always done, but efficiently and in a model that is connected to your data live. In addition to this, every single thing that I'm building is with the notion that data is constantly changing. How we access it on the web is constantly changing. These tools need to be iterative and we need to make sure that as we build each step moving forward, we're not dropping performance, nor are we dropping the flexibility that you've come to expect from Wolfram Language. With that, I did grab a selection of talks here that I really do have to say that the teams that I'm working with on all of these systems that are coming together to make working with data in the world truly accessible at your fingertips has been a privilege. This is not an exhaustive list of the names, but I'm really glad that the right people are working on the right things for these projects, and I'm excited for what I get to show you next year in my next talk. Thank you. Got just a few minutes for questions. Otherwise, like I mentioned, there are ways to connect with me. I'll be around for the rest of the day on the you know, from interface. You can find me on LinkedIn. There are some questions here. All tables from text documents like a PDF in order to compute to do computation on the data tables is often messy or varies widely in style. Takes a bit of time to use string pattern matching to pull the data out. So PDF development is something that I worked pretty closely on and that PDF development is particularly complicated because so for a moment, compare kind of docx and PDF. Usually when you send in a resume, you send in a PDF because you want everything to be represented exactly where it goes. If you give somebody a.docx file and they open it in a different program, your images might move around, your fonts might be different. And the reason for that is PDFs are actually mostly implemented as vector graphics. There are text elements built into the document, but those tables aren't stored in the document in any sort of consistent format. A few years ago I gave a talk showing some methods that I use to use image processing and extract table data from a PDF, because what you really have is just an image at that point, the text wasn't even highlightable in that image. So it depends on the PDF and if the PDF is more structured then extracting those types of semantic information are exactly the types of projects that are on the horizon. And Google Firestore, I have not heard of this and this is another suggestion that I have written down in my notes and I'm more than happy to look into. All right, so we're right at noon. Thank you all so much for joining and I hope this was informative.