Point Clouds as Representative & Performative Format for Landscape Architecture by Ervine Lim

Well, here's my presentation, so let's get to it. Just a very quick outline. I'll just talk about the background and motivation for those who are not familiar with the project. We talk about how I developed tools to work with point clouds and why I had to develop them. And then we'll discuss specifically point clouds as a representative as well as a performative format for landscape architecture.

So now for those of you who are not familiar with the Chilibung river, it's a river that runs through three major cities in West Java, Indonesia, Bogor, Depok, and of course the capital of Indonesia, Jakarta itself. And just from the photographs alone, you can tell that as the river goes along is 120km through these cities, the water gets degraded, the river is constantly being abused. And on the right you see that people are still using it for livelihood, but instead of, of fishermen fishing for fish, these are fishermen fishing for plastic bottles that float down the river.

I mean, there are a lot, a lot of issues with the river. But the salient one, or rather the most pressing one, and maybe the most politically charged one, is that issue of flooding. So these are photographs taken after the floods. We can't go into the floods because if you look at that stained line on that wall there, that is how high the flood reached. So basically everybody's first floor, sometimes their second floor gets flooded. And it's of course costing them a lot of money. The 2013 one is estimated to have cost Indonesia $3.2 billion. And if you read the news, flooding is not of course isolated to Indonesia. Singapore has it India, Asia and Pacific region alone $16 billion just in 2014.

Of course, this. The issue of flooding has not gone unnoticed. There are what I call infrastructural knee jerk responses, if you can call them that. There are the dams which have been proposed more than a decade ago, since 2004 talks about it. And only in 2015 supposedly some of the contracts were awarded. So that is one of the infrastructure responses. Another very huge and controversial one is the building of a, of a.

Sea wall. So basically trying to block off sea level rise and try to save the city from sinking. More obviously, if you go onto the ground is this issue of normalization, which is in fact kind of like a canalization of the river itself. It's an ongoing process. You'll see more images about it later. Sometimes we why they do it because on the bottom right image there is already a canal which they are now re canalizing to make it smaller, which is a little bit Counterintuitive. So.

Like Y mentioned, we are part of a team. That's kind of the great thing about being in fcl. We are part of a team that includes engineers, planners as well as landscape architects. And our main goal is to find not really a solution. It's impossible to find a solution just from the ten of us, but rather to start to think about whether there could be a change in the response that we take when trying to solve some of these issues. So instead of jumping straight ahead into engineering solutions, could there be a slightly softer approach? And this is of course backed by the engineers in the team. It's not just the landscape architects wanting more green.

Now. From the onset of the project, there has always been an issue with data. And a lot of the times our researchers had to literally risk life and limb to collect the data. Just to give you an example on the bottom right, to collect hydrological data, we literally went to the dam operator's house, asked him, could I have your data? He said yes. He'll photocopy 10 decades, sorry, a decade worth of data for us. And then we had to go and make it into, type it into Excel one by one. And of course, for landscape architects, we need vegetation, which is sorely missing. We need topography, which is completely contradictory as well as.

Completely unusable because the river basically goes up the mountain. So what we did, we partnered with the Institute of Technology Bandung, they were very kind in assisting us in getting a third party to actually fly the drones for us as well as the simulation platform here at fcl, we were able to create or rather capture very high resolution data of some of the sites that we had. So on the right you see one of the results of this. This is a high resolution auto photo of it of the site, about 20 centimeter per pixel resolution. But if you had already guessed, I'm not after two dimensional images, I'm after three dimensional point clouds. So this is a fly through, or rather flyover of the three sites that we documented using the uav. And you see, it's actually not a continuous surface. You see something to break up where the points are not enough to cover vertical surfaces. And this is quite.

A difference in terms of how we actually start collecting data. And you can collect them at any scale, you can collect them at the entire catchment scale, which is really, really large. Using satellites instead, you can capture them at the river corridor scale, or like what we've just seen using a UAV at a site scale, you could in fact have a million points just covering a Single doorknob if you so desire. And all of these can be nested within the same three dimensional environment. Now this is something that is very different from what we are used to as landscape architects. If you had wanted to work with three dimensional model, you would have started something like this. Started with a base map, contour maps, adding on building outlines and footprints, extracting them, and so on and so forth. Now, on a silver platter, you're given something as detailed as that. Of course, with that detail comes a bit of problems that we will discuss about later. So now that we understand that the starting point is this high resolution three dimensional point cloud data, my hypothesis I won't read through the whole thing is essentially to try to use it as a, simultaneously as a representative and performative format. Why am I doing this? Firstly, if you were to do landscape representations, that would generally be one.

By itself. And if you were to do performance testing, there's another school of thought.

Usually you had to jump through hoops, you had to convert from one format to another. If you wanted to try, for example, a landscape perspective, if you wanted to run a flood simulation through that, it is impossible, right? So in order to do something that is embedded.

Hypothesis at that point.

One of these formats that would allow you to do this.

Now, before I even begin trying to understand whether it is beneficial to go towards such a workflow, the problem was that although there are tools to work with point clouds, they are scattered across many, many different platforms. Argus has some, you have some from Bentley, you have some from Autodesk, et cetera, et cetera. And I needed something that would kind of firstly be usable for designers and secondly be something that would allow me to kind of manipulate the tools or even create the tools myself. So what I ended up working with is Rhinoceros. It is not a GIS program, it is a three dimensional modeling software. The good thing about it is that it already has a native point cloud object in which you can start working with as well as it has quite a, well, not really powerful, but quite a user friendly plugin called Grasshopper, which allows you to do simple programming without any knowledge in programming itself. So what I created was 28 tools that you see little icons up there categorized into three broad categories. So first category is the modification tools. These are basically the bread and butter tools that you need to start working with the point cloud data incision. You can cut it out, you can copy. In this case it is a copy and paste job from somewhere else. And why this is particularly good in point clouds is that you must understand, the point clouds exist as individual discrete elements, so there is no connecting tissue. So when you remove one point, the surrounding tissues do not disintegrate like a mesh or a surface model. In fact, the texture also is still completely intact when you start doing something like that. Now, what is novel about my approach is that while other disciplines use this a lot, to document the site, to analyze the S is, I wanted to inject something that is completely new, inject a design scenario, for example. And that's what designers do. I mean, there's no point in designers not designing anything. So in order to put in.

Scenario as a point cloud, I had to kind of jump through loops. The first, you have to, of course, model it, and you can't model directly in points. You have to model it in surface models.

To convert them into point clouds, which you can then texture to make it visually at least look a little bit similar to the original base data that you have around it. Now, the reason why I'm sticking or trying to stick within point clouds is not only because I want to visually fit it to the base data. I also want to, like I said, connect it to some kind of a performance.

Or analytical model. I'll talk about the tool that I use later. This one, you see, of course, is hydraulic model that we use with the help of our engineers team. So now, with the help of these tools that I kind of developed, I can start exploring a lot more options. So now it's not about whether or not you can do it. It's about, okay, now I can do it. What are the benefits or the pros and cons? So firstly, of course, is the enabling of scenario development. In this case, I pushed in the government's normalization, basically, the canal. I put the canal into the model itself. Like I said, you can do flood simulations and I can start separating the point clouds. Of course, when you look at the point clouds just now that fly over, you can see you can understand a tree from a building, from the road. But to the computer, all of that is just the same thing. It's just a point and some color information. Of course, there are much more advanced algorithms that will allow you to do this. But for now, what I am able to do with the tools that I had was to start separating that into at least basic classifications. So urban vegetation, water, as well as the strata in which they exist in so low vegetation, high vegetation, medium vegetation, so on and so forth. So you put all of that together, you start having something like this, you are able to start with the existing site, create your scenario, test it out, visualize it in this manner, as well as to prepare it for further analysis. Now of course is when you start comparing.

If you are looking at it as a form of realistic representation, I will tell you that with the tool it is a terrible, terrible way to go, simply because there is no way to light it, it's very hard to.

To texture it as compared to existing tools which have been developed over the past many, many years. In this case Cinema 4D with Photoshop, with lighting effects, with Loudbox, which is three modeling software. So if that was what you're after, then.

I don't believe that point clouds as a representative format per se is going to be of much use. But I think it's premature to kind of complet throw it out of the water because a lot of the representational formats that I'm comparing it with exist in two dimensional media, on screen, in print, at sections, plans, elevations and so on and so forth. But the value of the point cloud is such that it exists in three dimensional media. So how do you then bring out that three dimensional quality? I experimented with two options. One of them was digital fabrication. You take your reality captured model, make it into a mesh and then literally mill up everything that is there. Again, if you compare something that was done manually, that looks and understand where the on the right, you can't see anything at all. It kind of like a Swiss cheese kind of model. It's the exact same model. That's one that's lying outside. Now you must understand this took many, many weeks to create. And on top of that is missing a lot of important information such as.

And of course the are all estimated. Whereas on the right is directly captured from reality itself. And this lens.

Useful in an exhibition kind of setting in which we.

Use a projector information onto the model and it was reality captured. And because all the other simulations and analysis were done using the same data at the start, all of these will then be projected perfectly onto the model itself. So in this case you see quite a preliminary flood simulation that goes on.

Over the village itself. And this has been exhibited in Rotterdam, Zurich, Jakarta of course, and in Singapore multiple times.

Instead of trying to. So that was trying to bring the digital world into the physical world. So now we try to reverse it. We try to bring the physical person into the digital world. So in this case, for those geeks out there, you will know what this is. We are wearing an Oculus Rift, which is a virtual reality head mounted display. And you can of course if you can take the nausea, headaches and kind of motion sickness, you can actually fly over the landscape like Superman. It looks nothing like on a screen because you can start picking out details in the topography much more readily. You could see where the banks start to become a 90 degree instead of something that's smooth, et cetera, et cetera. I didn't go too far with this because I ran out of time, but I think this is another possibility that would.

More likely benefit of the three dimensional nature of the point cloud. So now that was for representation now as a performative format. So I did two.

Kinds of analysis. The first was, like I mentioned, with the help of hydraulic engineers, the use of flat simulations. The second was something that I found it to be possible. Yes, I can do it was the use of fragstats, which is a software to calculate landscape metrics. So landscape metrics, what is used for is specifically to quantify landscape structure. So for example, how connected the, the urban fabric is or how connected the vegetative patches are. And I was trying to, because.

There'S a current kind of a lack of the ability of landscape metrics to kind of tie in together with the dynamics of the environment, I was trying to see whether you could actually link the two together. So is there a correlation between what we get from the.

Flood models and what we get from the metrics itself itself?

So like I mentioned, the tools were created specifically so that designers could use them. And these are actually students works. We gave them the tools, we gave them a short workshop and then they went along and created their own scenarios. Some scenarios work better than others. For example, this one in the middle here completely failed. We weren't sure why until we realized that they didn't do any grading at all. So they changed the landscape, but everything was flat. So of course the model will fail. And the ones at the bottom look as if they can contain the flood. But if you were to dig deeper, you will understand that they are basically obliterating everything that is there, digging a five meter hole into the ground and then trying to salvage the situation in terms of the flooding. Of course, anyone that has worked on floods will know that it is impossible to solve flooding at an isolated point, especially one that is downstream. So instead of a design based approach, we went to a decision based approach. So we zoomed out, we looked at instead of a small section, we looked at a 40 kilometer stretch instead. And then we put in things that we thought were most likely going to happen. So first of course there's the original scenario. But there's also, if you look at the section, a full normalization scenario, which is basically the canal running the entire 40 kilometers of the stretch. Something that I think is more representative of what will happen, which is a partial normalization in which sections of it start to get normalized. And then, of course, our dream is that instead of using any of those infrastructural things, we start to use green infrastructure instead. So we have retention basins, we have areas in which vegetation is allowed to slow the water down.

In an attempt to try to mitigate the floods. So let's look at this. So in terms of it is extend the full normalization scenarios, of course, the entire stretch. And this is not something that is plucked out from fantasy. We actually use sections that they drew and then we embedded In, I think, 400 over different sections and then interpolated across the entire river. The partial normalization, like I mentioned, is the same thing, but a lot less extensive. And then, of course, the green infrastructure scenario is the one where we scatter little.

Elements across the entire stretch. So this map on the left is probability of indundation map, which our engineers did. And it's actually a very dangerous map for us to show to stakeholders, because if you were to take this map.

As the only result that you have, and if your only concern is the flood, you will go with the full canal because it is the one that will solve the floods. And we know it will solve the floods because that section, that cross section, was designed specifically to hold that amount of water. So theoretically, it should work. And based on the simulation, it does, at least most of the time, work. The other two work to a certain extent. So the retention bridgings flood like they are supposed to, the expansion inverse flood that they're supposed to, but the downtown area still has a little bit of flooding. Now, this is when engineers have to come in and kind of let us understand the graph a bit. So the full normalization scenario, the problem with it is we already know, of course, through experience from other canals that the flood is going to come harder. It's going to come faster, and it's going to hit downstream a lot faster. And downstream is Central Jakarta. So not only is that going to happen, it's going to have a lot more erosive power. Water is going to come in faster, harder. It's going to erode the banks a lot harder. So we don't think that's the right way to go about it.

We also have another two scenarios, one I call the urban scenario, another one is the vegetative scenario. Nothing was changed in terms of the bathymetry of the river. The topography is entirely the same. The only thing that changed was the amount of riparian vegetation. I did this as a control to see how sensitive the flood model was to changes in vegetation along the river itself. And unfortunately we could not see any difference at all. So even though I reduced the vegetation down to 30% or increased.

Didn'T look like there was much at all, which is a bit contrary to what we understand from other literature in that riparian vegetation does have a role to play in flood mitigation. So what I'm suspecting is that the flood model itself, at least the portion that controls how vegetation is calculated, is not fully incompatible the effects of vegetation. So, for example, it doesn't account for infiltration. So all it does is it accounts for friction, that's all. So as far as we could go with that. But the problem, of course, with that was that when I tried to correlate some of the landscape metrics back to that, it was impossible. So you could, I believe, completely change it into vegetation. I mean, like everywhere is vegetation, and then these numbers will be exactly the same as you see on screen. So that was a bit of a disappointment. But at least I know for a fact that it is definitely possible to run some kind of analysis, this kind of analysis.

On the models itself. Now, towards the end of last year, when all our models, all our data was almost in, we were contacted by an ngo, Chilubung Medica. So they were proposing, or they were counter proposing, some of the government's plans of relocation and eviction. And they wanted our help to kind of help them along with the flood simulations. And of course we tried to do that. So we got everything together. We made sure the floods were as representative as possible, to the best of our knowledge. And then on the 20th of August, this happened, or rather after the 20th of August, eventually this happened. It looks a bit more like a war zone now than anything else. The government came in, bulldozed everybody that was within, I think about a 25 meter radius from the, the river. And if your house happened to lie halfway between the 25 meters, they would cut a section through your house. And eventually it became an antithesis to whatever we wanted to do. We now have a river that is lined with. We have riparian vegetation that is now relegated to living in tires, and we have residents that are living still next to the river, but even more distant from the river than they ever were before. So at this point in Time, I think it was a little bit like the final nail in the coffin for us. We had been battling this for a good, I think, four or five years. And it was at this time that we kind of decided that, okay, we need to stop because we're running out of time, we're running out of money. And it became a little bit too sensitive to start talking about this anymore because of the evictions that were happening. So then back to again.

The research itself. So what are the findings, at least so as a representative format? Well, firstly, there's going to be an issue with obtaining the data. We're not talking only about technicalities in which vertical services disappear, but the use of UAVs is now going to be highly regulated. This was a flyer that was distributed to every single household in Singapore. Not just flyers, every single household in Singapore received that flyer instructing you about what you can and cannot do with drones. For researchers in Singapore, you need a permit. This is something that is new and enforced in law. In Tokyo, anything above 250 grams, you run the risk of getting arrested. In the states, everything above 250 grams, they are trying to get registered. I don't think that's going to happen, but that's what they're trying to do. And I already mentioned about the issue of trying to achieve realism in terms of a representative format, but I think if you're not after something that's realistic, I think this is quite a quick way to do it. And I'll explain a little more later. Now, another problem with point clouds is that they have this strange ethereal nature to it. The moment you want to, it's counterintuitive. I want to see something in more detail. I go closer and then it just disappears. So everything that is close to you in the foreground dissipates. Everything that is in the background is in focus. It's a little bit strange and sometimes a bit irritating when I'm trying to make more sense of the data that I have. And it is extremely static in nature because it is almost like a three dimensional photograph in that sense.

So there's no animation, at least in the work that I did. No animations, no people walking around, no atmosphere, no light, no way to cast light, even if you wanted to.

And of course it is static in time, meaning that if something were to change, you would have to fly your drone again. And of course that will entail more cost and more time. But I think instead of trying to force.

Point clouds into some of the existing formats that we have so we have perspectives and sections and things like that. It should be dealt with completely differently. It is a different class on its own. It should be studied as a completely different format on its own and not really compared with other existing ones. Now, as a performative format, there's of course difficulty in obtaining the data we saw. There's also a lot of difficulty in processing the data. A lot of this will likely not come from landscape architects, they will come from computer scientists or engineers in which they had to clean, filter, classify in order to start making sense of the. And the data that we have actually now, not really three dimensional, it's more like two and a half dimensional because it exists in the grid. And what this means is that we cannot have water flowing under bridges, we had to clear all physical bridges, we cannot have water flowing under canopy. And all analysis with regards to that three dimensional nature that you might think you need is not there. Of course you can recode everything, but then everything becomes a lot more computationally expensive.

And of course there are errors both in the way I did the classification as well as some of the flood simulation from our high school geography. You know that the convex, convex bank of a river deposition, in this case it has erosion. Why this is so is we're not really sure. And it was only when I started to visualize some of these results, I mean, the engineers looked at the graph, the graph didn't say anything. We started to bring this out. It seemed to have some issues there.

And of course.

When trying to relate some of these analysis back to design, if you could even understand what some.

That mean that you can design in a fruitful manner, because numbers don't necessarily inform you of something that you might need, for example, spatial quality or aesthetics or things like that, or even directly ecological qualities. Of course you can run other analysis, that's the whole point of this exercise. But in this case it was a little bit difficult in terms of the time it took.

All along. We have always wanted to have it like an iterate. So you create a scenario, you get the results and then you go back, change certain elements and you run it again. This was never done. We always only had time to do one iteration, got the results and then we ended it there. So a lot of the problems with that is because of the time it takes to firstly create the scenarios, test the scenarios, get the results back, et cetera, et cetera. And that loop was never finished.

So now just to round off what I see lies is firstly baseline data or the collection of it will never Be the same again. We are going to have much more accurate data that you can either just collect just a few guys of us going out into the field, or governments are already doing it. Singapore, I know for a fact that it has done the entire Singapore in LiDAR. By this year it will finish majority of the roads in lidar. And it's about time we start.

Using some of this information as well. And regardless of which format it is, point class will always exist in a table. You have your basic XYZ and then something that comes behind it. And it is in this table that I think.

We need to be able to then embed custom metadata, meaning classification information. You have maybe some kind of a.

Solar radiation factor, if you have such a thing, and then all of it goes into a table by itself. Of course, the problem is that that table can be billions and billions and billions of data entry points large, especially when you're talking about national data. And there's research being done in computer science on how exactly do you manage this data? What are the standards in which need to be achieved in terms of point cloud data? But I think it's a good time in which we start to highlight to software developers, large software developers, that there is a need for certain other tools to start coming together and using the point cloud data. Not just, I mean, right now a lot of it is just measuring, you know, simple stuff. Like I saw what NPARX was doing, trying to see whether the vegetation was too low so that you need to go and trim the trees. So a lot, I think there's a lot of value that needs to go in it. And I think we are at the starting point of something that is going to be completely different from everything that we have known in terms of 2D GIS. So now we are moving into a completely three dimensional world. And I think all these analysis and tools and all the things that we have been doing in two dimensions need to start to progress into this third dimension. And eventually when that all kind of comes together, I think this is where point clouds for landscape architects is going to be the most useful in early stage testing. So we do it for buildings, why not for landscapes? So instead of, of.

Just blindly designing and basing it on aesthetics, why not test certain scenarios? Why not have, of course you could have other performance indicators as well and running the loops eventually when you get up to speed of it. And this all needs to start being inculcated in our young landscape architects, not so much to replace whatever they know now, but to enhance it, to bring it to the next level. And for me, the next step. I will be joining the National Parks Board in about one and a half weeks time and I'll be specifically looking at trying to use the LiDAR data that was collected by SLA specifically for urban ecology users. For design. I'm not sure because they didn't put me in the design group, but. Well, let's see. Okay, so that's all I have for today, thank you very much.