Creative Conversations Live from Fall X19: Omer Shapira

So welcome to our Creative Conversations podcast that we're live streaming from the Fall X Festival here in the Wisconsin Center. This is a partnership between the Brimful Life Career Wellness Company and NO Studios, Milwaukee's membership organization for artists and creatives. So I am here today. I'm so excited to talk to Omar Shapira. Yes, and you are. I had to ask you earlier, how do I describe you? Because you are sort of like an artist and an engineer, engineer and a tech person all rolled into one. And then you do virtual reality work. So just how would you describe yourself?

I don't know by my name.

That's a good place to start.

Engineering artists is typically what I call myself nowadays, in that order, because I work in tech now. A few years ago, it would have been the other way around.

Interesting. I love it. And so let's go back a little bit to the beginning. Sort of like your youth and you were a student of both mathematics and linguistics. It seems like you always kind of had sort of the tech side and the art side always interwoven. And it started with your studies. So tell us a little bit about that combination.

Well, it didn't actually. Like, I only went to school, and this is typical where I come from in Israel. I only went to school when I was like 25 and actually started being an artist a lot earlier. So I was a filmmaker for. Well, since I was 15. Basically I was working in film, like public access TV and things like that. And I sort of became a professional around the time that I was in the army. I became like a professional video editor, a professional visual effects artist, and dedicated pretty much. I decided that filmmaking is, you know, is worth my time as an adult. I spent a long time doing that. Started burning out of that, which is weird. At 25ish.

At the ripe old age of 25, you sort of like hit your peak there. But go on.

There's a good reason for that. Yeah. When you're young, as a person working in relatively mature industry, you're constantly waiting for people to go away so you can begin shining. And I didn't really have the time.

Yeah, hey, look, I'm old and I feel that way too. I'm waiting for everybody to go away, I guess, like.

Yeah, as a place grows, then, yeah, over here, over here, it's pretty much the same. I feel the same thing. Because America is what, like 80 times larger? But yeah, so I was looking for substance and I thought this, this is like the mid aughts, the early aughts, and I Thought natural language processing is going to be huge in the next generation, so I better do something.

So you were thinking that.

Yeah, I was thinking, like, how did.

That even come to your mind? Because I don't think most 25 year olds are like, huh, natural language processing. I see the future there.

I mean, it wasn't that like Google was just beginning to become impressive with some of their natural language processing. You could already enter some natural language queries and they would return. Interesting results. At the time, my mind was blown. I was like, hey, how come they get to do this? And how can that be used for art?

Interesting.

So I went to school for that. And as I was going through school, I became a little bit less interested in doing that just because my field, I was really drawn to my field. I was really drawn to visual arts. And so I started steering more towards graphics programming and I moved to art school in the United States. I found myself not really doing art, but mostly spending time in the computer science department doing graphics programming. And that's how I got into VR.

Nice. I like it. You know what I like about your background and your story is that you sort of just led where your passion and interest was guide you, right? And a lot of people, they end up in professions and careers not because they have a particular interest in it or a curiosity. It's more of like what other people expect them to do. Are you just kind of a free spirit in that way? Or did you fight against maybe what parents or elders at that time expected of you? Or did you just say, now it's art, now it's language, now it's engineering? I like it, I'm doing it.

No, it's totally. It's totally. I mean, like, I will preface this with I was very lucky. I went to school just at the right time. I went to school just when the economic crisis hit. I went into the army when the other like, earlier economic crisis hit. So, like, I was, I was shielded from the worst times of looking for jobs.

Okay.

Which pretty much meant whenever I had to, like whenever I had a major pivot in my career, there was some kind of job looking for, like, looking for a person to do it. But yeah, My parents are both scientists. They both worked in the weapons industry in Israel. So my, you know, like the path to become a filmmaker was very much not listening to them until you realize that you are kind of like them and like, my mom is a mathematician. Okay.

Yes. Eventually you do realize that you're more like your parents than anyone would care to admit, right?

And I think the first point where I was like, you know, like, luck had really struck was when I was. When I graduated from my grad school in the United States, and the world's first. At that time, VR studio opened in New York, and I happened to know them. So I just. I, by extreme streak of luck, just landed a job that happened to be in a visual effects studio with my old skills doing my new thing.

Well, I was gonna say it's just. It's almost like a perfect storm of all your skill sets coming together in VR, Right? So your video production, your artistic work, graphic design, computer engineering, linguistics. I mean, it's kind of like you're bringing everything to bear, right?

Yeah, but I guess a lot of people feel that, and that's about. Specifically about VR. And I think this is, like, if you look at the trend of artists right now emerging past five years, emerging into virtual reality, it's a bunch of people who are. This is like typical to a lot of computer arts. They were the rejects. They never really fit into a professional industry, which is, you know, film is pretty much professionalized now, right?

Yeah.

You go into film because you're looking for a job. You go into visual effects because you're looking for a job. People who went into VR were not looking for jobs at first. They were looking for a form of expression, and suddenly the jobs were around them.

Interesting.

Yeah. So it's not just me, polymath, who happens to walk into that. A lot of people found their calling that way. And every single person I meet in this field is fascinating by their own. Right?

That is actually really fascinating. So someone who's not sort of immersed in the VR world, that's just kind of like. Kind of an interesting description. I've never really heard it described that way before, but I like it. All right, so you do a lot of work in VR. What are you particularly really excited about? Either stuff that you're seeing or stuff that you're personally working on.

Yeah. Gosh, this is a hard one. VR is really at a really early stage. And it's funny to say this because, you know, this is like the third overall generation, the third attempt to make VR happen. We had one, like, in, you know, like the mid-90s. We had a little bit of it.

I'm assuming it didn't go well, or I guess it was the first stepping stone for what it is becoming.

It went away very quickly. But, like, people forget this nowadays. There were a hundred thousand, about 100,000, if I'm not wrong. Virtual boys Game Boys for virtual reality created in the 90s and it took a while for our current generation headsets to reach that count. And yeah, so that was bad. But nowadays when we're, when we're clearly doing the right thing and we're clearly in a path to people being productive and interested and immersed in more than one way in virtual reality, still the most basic human beings, kind of like the Maslow's hierarchy of people outside VR is beginning to manifest issues inside VR. We have like, we've had to say the first MMO in VR release, officially or not even released, just announced officially a week ago by Facebook and we previously had VR chat. And a lot of people are like being sort of, I wouldn't call it productive, but social in VR. And they're getting to a point where it's interesting enough as it would have been an MMO 15 years ago as it would have been second line. We're still not as expressive because our entire bodies are there. And no one has made all the stuff that, you know, we're used to in reality. No one has made that inside VR yet, right? Which is, think about it like you can't grab, like you can't grab anything to read yet there. You're there for a very specific reason. So you're there for a short amount of time.

But will it ever get there? And should it? But should it ever? Because then what is the difference with reality? Like, you need reality to be able to do stuff that you can only do in reality. Virtual reality by definition is virtual. Should they actually end up being the same?

No, and they're not going to be the same. But it's not that we still need tools of expression there, right? And one of the, you know, one of the biggest promises to me in VR, well, there's two things. The first thing is to be able to capture people an experience where you are really messing with their senses. You have control over it, which is something that film directors in a dark cinema or immersive theater directors have. That's one thing. The other thing is it has to be in order to be adopted, it has to be comfortable. It has to become the same level of belonging in your home as a television or a television. And for that it has to be useful, accommodating and so on. So you have to have to have a bunch of tools in there for your disposal. So I really would like people to be able to do their jobs in VR for a certain degree of job. You should be able to tell it operate a robot in VR, you should be able, and this is something I'm literally working on right now, be able to work on a film production in VR. So, you know, you should be able to be a visual effects artist or a sculptor working on a 3D model in there and collaborating with people. And that's. So that's not fulfilling all of our needs inside that space, but it's definitely making this, you know, all that it can be for us to be humans inside.

Right. Okay, so let's take the one example that you had, being able to like work on a sculpture within VR. And you're working on that now. So what's like the one thing either challenge that you need to solve or the barrier, the wall that you have to break down to get it to that point of much better.

Okay, this is a hard one, right? No, because if you think about any tool that you're using in your daily life, there is only the learning curve of that tool. If you're learning how to do pottery, you're only learning how to use your hand. You're not learning how to keep your balance at the same time or talk to your teacher at the same time while you're doing it. And you don't need to learn how to look around, see other people doing pottery class with you in VR. You need to learn all of these at once. Right.

So.

Having enough of your life available to you, around you when you're doing a task in VR is probably the hardest thing. But that's pretty much being handled. Everyone is attacking this because the industry's number one goal is to keep everyone there for as long as possible. But the thing that I am interested in and is very hard to solve is collaboration in real life, when we have rapid communication. Like, I see your micro facial expressions and I know when you're uncomfortable when I'm saying a thing or when I'm doing a thing, it is a lot easier for me to reach, well, consent and consensus about what something I'm doing. So suppose I'm like building a small city model or something like that, and someone doesn't like what I'm doing. They could be like, you know, like unvoice chat. They could be like, I don't know about that, but that would be like, that's a theory. Or you can communication. Right? You need to be able to micro emote to people there. So getting that level of collaboration is so much harder than just being able to have a tool. And that's something that I really care about and I haven't seen solved I've seen a lot of research in this area, but one of the areas that I'm working on is an attempt to solve it. There's. There's a media format by Pixar that we've adopted at Nvidia, which is where we work, called Universal Scene Description, which allows multiple people to have multiple opinions expressed in geometry about something. So I can be sculpting a character and someone else could be painting it at the same time. And I could say, I don't like this thing that you're painting. I have a better idea. Not layer that on top. And he's like, let me switch around to see which one is better. And they can do that really easily. And the art director can come in and she can say, I have a better idea than both of those. And she will make a hole in the thing and change it. And we will all be able to see all these changes and communicate with them at the same time. This is suddenly, you know, this is something that doesn't exist in the real world. Yeah, yeah. We can understand really easily because we're humans that we're able to communicate in abstract concepts really well. And so this gives us a level up in a virtual space that we.

Don'T have in the real world. Yeah, yeah, yeah. Because, like, in the real world, you could have those conversations, but you couldn't see the manifestation of it right away. But in virtual reality, when you're done doing your work on this, you'll be able to do both. All right, so this seems like a lot of hard work.

Oh, my God, it is.

What. What's your why for? Why you're focusing on this and why you want to accomplish that big, audacious goal that you just described. What keeps you motivated to keep figuring it out. Yeah.

Gosh, this is. Okay. So this is a really hard one. I always. Throughout my career, I found that if I have very good control over a tool or a discipline, I can be expressive and not a moment before that. So I can be really a visceral artist from the point that I have a very good skill. I'm very good with editing. I know rhythm really well like that. Computer graphics and computer arts in general. The barrier for being like a visceral artist is so high because you can't really do something from intuition and translate it to the exact thing that you want. Typically, what happens, and this is true for a lot of computer art, is you would try something, you get something else, and the result is beautiful, but it's not controlled the way you wanted it to be. And it often takes very large teams with very small goals to get some controlled results. That's why AAA games take five years. So my why is I really want to make tools that can get me to the level, the level where I feel like a visceral artist, not like an intellectual, as quickly as possible. Because when I'm an intellectual I can be a good engineer, I cannot be a good artist. It's just a different skill.

Yeah, that's really interesting. But then as you're working on that, it's not. The technology is not at the level yet of being a visceral artist. You're working on that and using your tech side. So to balance that, do you engage in art? Do you still do art as a way to get that visceral expression that you hope to build, but it's not there yet?

That's a really tough one because I have for a few years done that and like I had this modern dance experience capturing VR and I felt that was like really native to that experience. I had some other experiences in VR and some other digital art pieces, but for most of my time in digital art, I've always been like the person who feels like I can't do anything with this yet I just want to build the tool. And I guess that's how you become an engineer or at least that's one way. And most of my time now when I'm wearing my profession hat is spent understanding artists, talking to visual effects artists, getting what they want to do and then helping them build a tool. And my mind is always blown because most stuff that people I admire and make is so much better than what I could fantasize making, like in the current mindset that I am in. But like when I'm off that or when I'm doing something else, going back to rhythm based art forms, I'm still active and I like, I still get, you know, the feeling of like, oh yes, I, you know, I know what I'm doing and I'm like, I'm good at this, right? You know, like music, editing, things like that with games. I have made things that I would call art and have been in galleries and have been in museums, but they're like, it's just a nice prototype. I don't know, it's not something that I would put in my memoirs yet.

Interesting. You are still young and man, I'm super excited to see where you're going to take this next. I just love the way that you are thinking about these complex issues. And really integrating them and combining them. And it's, like, super cool to see where all this is going. So did you ever think that there would be something like Fall, Fall X that is really bringing together all these disciplines that you are so passionate about? Or maybe you did see. You weren't able to see some futuristic stuff that I would never have guessed. So maybe you did see Fall X coming.

No, I did.

You know what?

No. And I'll really be like, I'll share with you a few moments that my mind was blown since we were in the United States.

Yeah.

I think the first one is when I just got here. Like, actually when I decided to move to school here. I had met with. For the first time in my life in New York, like, 2011, I met with what I would call hipster hackers. This is like, a really bad name for both of the parts that I said. But I met with people who are like, you know, like, understand here and now, speak my language, and do tech work. And to this day, I also understand that people tend to, like, especially as they age, tend to go deeper inside their own communities, go deeper inside their own disciplines, and don't end up evolving the ecosystem. They may, like, you know, contribute a lot to one thing. And computer graphics researchers will often be brick, like, make some, like, amazing breakthroughs. Groundbreaking is the word I was looking for in one specific computational art. The people who use their works are often people they don't meet. And that's the insane thing. And, yeah, people who have both sides, or at least are able to meet the users or the audience and either hear them confused or entirely excited about things that they thought were the opposite are so valuable. So I'll be honest. It doesn't really happen in south by anymore.

It doesn't really happen, what, at south.

By Southwest anymore, but it does happen in film festivals and small tech festivals in the United States. There's one crazy thing about Americans which I have not seen in any culture.

Just one crazy thing?

Well, yeah, many. But one crazy thing that is, like, for, you know, like, all for the good is everyone is so excited to communicate, and everyone is so excited to change if this brings them success. So everyone wants to listen to audience opinions. It's not really a European thing. It's definitely not an Israeli thing. Interesting. So, yeah, the need to communicate between people and especially, you know, especially in smaller communities, because those people all have different experiences. Coming from different small places means you grow up with a different language, even if you're both speaking English. You literally grew up With a different language. As a linguist, I can say that. Yeah, yeah, yeah. And it also means that you grew up with, like, an experience that you can't translate. So your views of intimacy are different. Your views of how I use a thing or how I go about how I conduct myself and what is polite in the world are very different. And so, like, Americans have this urge to learn from one another. I know it's weird saying, saying this because I know, like, all the. All the, you know, many wrong things in this country contradict what I'm saying.

Yeah. But I think it does take someone with the outside perspective to it to see, like, we just don't maybe appreciate that in the same way that you do because we're in it every day and we just, you know, figure that this is, you know, how it is.

I will tell you this. I've been here for less than 24 hours. So many more people have asked me not, like, how I am, but, like, what do you think about this? Is this cool?

Really seeking your input. Yeah.

Than anywhere or want to know about me and my experiences more than it goes to be polite about something than I've experienced in New York in a day or a week.

Oh, that's interesting.

Yeah. This place is really thirsty for knowledge and experiences.

When you say this place, you mean, like, Milwaukee?

I mean Milwaukee, but I also mean this conference.

Oh, the conference, right. Yes. In Milwaukee. Yes. I like it. I like it. Hey, thank you so much for sharing your perspective. Really, really fascinating. I think. Sort of like your perspective on everything and your interesting blend of backgrounds. It feels like, you know, I'm just sort of really excited to see where art and technology coincide and sort of how much there is blending between reality and virtual. And so am I.

Let me know if you see it.

Well, you'll get there before I do. Thank you so much. Yeah. And I look forward to your keynote speech tomorrow. Right. Today at 4pM I will be there. Thank you.