Yves Ubelmann is a technologist, artist, and architect who is on a mission to digitally preserve the world’s cultural and natural heritage. He is the founder and CEO of Iconem, a company that creates stunning 3D models of endangered sites and environments. In this episode, Brad and Yves take a boat trip through Venice, touring the subject of his most ambitious project: an AI-generated digital twin of one of the world’s most magnificent cities. Inspired by his grandfather’s work to restore war-damaged heritage sites in France, Yves shares how he is using AI to memorialize the world’s heritage in 3D and raising awareness of the effects of climate change on our planet.
Yves Ubelmann is a technologist, artist, and architect who is on a mission to digitally preserve the world’s cultural and natural heritage. He is the founder and CEO of Iconem, a company that creates stunning 3D models of endangered sites and environments. In this episode, Brad and Yves take a boat trip through Venice, touring the subject of his most ambitious project: an AI-generated digital twin of one of the world’s most magnificent cities. Inspired by his grandfather’s work to restore war-damaged heritage sites in France, Yves shares how he is using AI to memorialize the world’s heritage in 3D and raising awareness of the effects of climate change on our planet.
Click here for the full transcript
Brad Smith: I'm Brad Smith and this is Tools and Weapons. On this podcast, I'm sharing conversations with leaders who are at the intersection of the promise and the peril of the digital age. We'll explore technology's role in the world as we look for new solutions for society's biggest challenges.
Yves Ubelmann: The nice thing with technology is when something becomes possible and you do it, you just don't know what you will really finalize with the result. There is so many outputs you cannot expect at the beginning.
Brad Smith: That's Yves Ubelmann, the founder of Iconem based in Paris. Yves's company is using AI to create digital copies of some of the world's most significant natural and cultural heritage sites at extraordinary levels of granularity. In the special conversation which we recorded in a boat on the canals of Venice, he shares what drives his passion for digitally preserving natural and cultural heritage sites. We discussed how the pandemic provided the perfect conditions to tackle one of his recent projects, creating a digital twin of the entire city of Venice. And how sharing these digital models with researchers helps shine a light on what we can do to preserve them and on issues like climate change.
To hear more conversations like this one, I invite you to follow or subscribe to the podcast wherever you're listening now. My conversation with Yves Ubelmann up next on Tools and Weapons.
Brad Smith: Yves, your work has brought you to Venice and around the world in recent years, but let's start with you. What brought you into the field of using digital technology, using AI, to really help people understand what's happening in the world? Where did you begin with this?
Yves Ubelmann: Since my childhood, I was in the field of culture and architecture, thanks to my family, because my grandfather was an architect. He was restoring historical monument in France, in St. Michel, and he did the restoration after the World War II and a lot of restoration of churches. So I was in this field and I was passionate with history, architecture, so I decided to do a school of architecture, but I was also really concerned by the destruction of cultural heritage.
When I started my work, I had the opportunity to come in Afghanistan to follow the work of archeologists that were digging the whole civilization of this place and this civilization is not well known, and it was the only track we had. And the thing is, from one year to another, we were seeing the disappearance of this heritage. So I was questioning myself what can be done in this context to save at least the memory of this heritage?
In Afghanistan, there is no possibility to preserve physically these sites because there is a conflict. These sites are remote in the mountain; it's a desert. So we find the only solution is to preserve the image of those sites. The memory of those sites. So I was interested. I was working in the imagery technology to find a way to be accurate in the documentation we can do on the field.
We started just with few algorithm at the beginning, working very experimental way, taking some picture on this site in the mountain of Afghanistan, trying to process it back in France. And we had very good results. I realized that we were able to recreate a full twin, a digital twin of a site. And a digital twin of the site is useful for archeologists because they can work on it. They can create knowledge and expertise on this twin. It's also interesting for the public, because the public just [has] never seen these sites and there is no possibility to access. So it was both a manner to study the sites, understand the sites, but also to share these sites to a broad community.
Brad Smith: And this led you to create your company, Iconem. One of the things you've done with Iconem is bring your staff here to Venice in the pandemic in 2020 to do something that had never been done before. What did you do here?
Yves Ubelmann: Yeah, it was during the COVID time, we decided with the Fondazione Musei Civici. It's the main foundation that manage the Doge Palace and the main palaces of Venice. We decided to partner together for large scale experiments. The idea was to make a full digitizing of the palaces and the city itself.
During this time, it was a unique opportunity to work on the empty palace. So I was able to bring my team room by room, places by places, square by square in Venice because there were nobody there. So we stand one month here with all the technical equipment, taking picture at different levels from the ground with the drone.
Brad Smith: You use drones in the air?
Yves Ubelmann: Yes, in the same place with the boat. We just fly the drone here to take all the details of the palaces here, the facade. And we use also airplane to cover the wall area of the city. And then when we were back in France, it was also COVID time, so there were not a lot of activity.
So, we took all our big computer and we processed all this data and it was a huge amount of data. It's more than 300,000 pictures, so it was the first time we did a digital twin of a city.
Brad Smith: You captured the city down to, I think, two centimeters. So you can see brick by brick, kilometer after kilometer of the buildings and this canal. And as you said, you captured it all literally in digital form.
Yves Ubelmann: Yeah. For example, you see this facade of the building just in front of us, there is eight kilometers of facade of high value historical buildings. And it's a lot. Eight kilometer is not possible. I mean, for an architect to make an assessment of each facade, to see the little cracks, to see the little damage.
And for us it was like four hours to take the pictures. Then we just make the processing. And then with machine learning, pattern recognition, AI can do the job to detect the damages on all these facades. So, it's opened really new perspective in terms of conservation for historical buildings.
Brad Smith: How many drones did you use when you were filming Venice?
Yves Ubelmann: We have two or three drones. And when we are in a place here... You can see this is the Doge's Palace.
Brad Smith: Right.
Yves Ubelmann: So, we did a full facade with the drone of the Doge's Palace and also the square. And it's just like few hours just to have a full representation, sub-millimetric representation of the Dutch palace.
What is interesting in this palace is you see the main room here. At the second floor, it's like 12 meters high and there is a ceiling with beautiful painting from Verona and from the ground you cannot see the detail of this painting. And what we discover with the 3D model of this palace, we discover all the tiny details of the painting and the brushes of the artists and also the cracks, also the damages.
And in this one, we realize how valuable were the 3D model of a palace like that because it's not possibly so rich, it's so big that it's not possible to have just a view of all the tiny details of it.
Brad Smith: Now, did you have the idea of coming to Venice before the pandemic, or was this something that came up when you realized that the city was empty?
Yves Ubelmann: I should say it was a dream, but I know before the pandemic it was impossible to do so. I just jumped into this opportunity and trying to convince the foundation here that was the right time to do this. And in few months it was done, so we succeed to do the work.
Brad Smith: Was it easy to persuade people here to do it?
Yves Ubelmann: Yes, because I think now people, they really... They feel the importance of these new tools. And even if they don't, they're not very technical people. We really can understand now the value of the technology because of the imagery itself. An example, when I show to the head of the foundation, all the tiny detail of this painting, they never saw in this way. It was evidence that the value was there.
Brad Smith: You also really, I think, shined a light on the impact of climate change here. Can you tell us about that?
Yves Ubelmann: Yeah, there is a long-term threat in Venice, the sea level rise, and that will affect the whole city. All the main monuments will be affected by this. And with an accurate digital model of the wall city, you can really have a clear view of what will happen in the future. And it's interesting to see how this right of level will affect area by area the city. So we can make a simulation and we can have a better understanding of the future of the city. And with this understanding, find the best solution to protect.
Brad Smith: You're sharing this data in so many ways, including with the public in museum exhibits. Where did that start? Was it here or was it a museum in Paris that the public first saw what you captured here?
Yves Ubelmann: The nice things with the technology is when something become possible and you do it, you just don't know what you will really finalize with the result. There is so many outputs you cannot expect at the beginning. So we started this project with the idea of protecting heritage, working with architect, working with conservator, and then we saw that we just recreate urban landscapes of one of the most beautiful city of the world. So we decided to share this result with partners, with museums, with cultural institutions. And we were discussing with L'Acropole in Paris and we were saying, why won't we bring Venice in Paris through this 3D model? And that was the idea of the exhibition. So we decided to make a large scale video projection showing the landscape of the canal for the visitor, diving the visitor inside the Grand Canal and also inside the building itself.
Because as we are also digitizing the inside of the building, you can cross the facade and see the different living rooms, beautiful artwork. So you have a general view of the beauty of this place. It was an interesting experiment because I think there is another threat for Venice. It's not only the climate change, it's also the mass tourism.
There is 30 million people who come here every year, and this city wasn't built for that. So we have to find another way to share the beauty of the city. We have to produce new kind of content for the public because the tourism is still growing. So we have to find solution. And the idea may be by building a digital Venice is also a way to preserve Venice from this new threat.
Brad Smith: You're not only bringing this technology and your techniques to a city like Venice, you're bringing it to other continents and sources of nature like the Amazon. What are you trying to accomplish there?
Yves Ubelmann: So, in this context of climate change and everything, the environment is changing is so fast. And so we were interested by studying the interaction between cultural heritage and natural heritage. And in the Amazon, natural heritage is changing very fast. The vegetation is changing, the spaces is changing.
So, we wanted to see how this change of natural heritage of biosphere could affect also the cultural heritage. We were working in Colombia. Archeologists recently found very beautiful painting from 8,000 years ago, from prehistoric time. And this painting were protected by the tree, by the vegetation before, but now it's open to the sky. So you imagine that the rain just affect today, this painting.
So, the idea was again, to make the full documentation of it and see how the different perspective of these sites and how we can in the future protect it. Maybe by also changing the vegetation around and help preserving this to the climate modification.
Brad Smith: We at Microsoft have had, I think, an enormous privilege to work with you and your team at Iconem. We worked at the Musée des Plans-Reliefs, the Mont-Saint-Michel exhibit there. We were able to take that not just in the museum in Paris, but around the world to Seattle, but now we're working on something new, something at the Vatican. Can you give us a sneak peek of that?
Yves Ubelmann: So for us, it's also a tremendous opportunity to start this work. So, it's the idea of having a better understanding of one of the most beautiful monument in the world, St. Peter in Rome. And it's a so complex monument that we really need to have the best technology just to explore the monument itself. There is a lot of different layers, historical layers that start from the St. Peter stones and underground and then first Basilica, second Basilica.
And the second basilica is extraordinary architecture of technology because they invented a lot of new ways of doing architecture to be able to bring this cupola, so big cupola so high in the sky.
Brad Smith: Yes.
Yves Ubelmann: So, with all this 3D dimensional data, we are able to have a clear view of this engineering method from Renaissance to cover such big church and big area. It's an interesting project because it's like an investigation of the 2,000-year history through one unique monument, and we can see a lot of new connection between the past of the monument, maybe the future of the monument as well, because this technology also will help all the people to have a better understanding of this place.
So it'll be also a way to share the richness of this architecture, the richness of this history for broader audience.
Brad Smith: I find it so fascinating what you're doing because you are using new technology from drones to cameras to AI and just an enormous amount of digital data. But what you're fundamentally doing is advancing human knowledge and expanding our understanding, the history of the planet in many ways, and the history of humanity and civilization.
You're bringing this all together and spreading it around the world. What do you hope people will take away as they learn from what you're sharing with them?
Yves Ubelmann: I think whatever digital world we can build, whatever synthetic landscape AI can generate, we all share the same ground. The same ground is... I mean, the reality this heritage is our roots, what we have around us, and the goal of my work is to preserve this common ground for all of us.
And I think trying to replicate this work into the digital today is the more efficient way to be able to preserve it for the future generation. So, I hope even if there is some cataclysm, some destruction, we will keep this memory and this is a memory that will also keep dialogue between all of us through different cultures, through different generation, because anyway, we share the same roots.
Brad Smith: It is extraordinary to me because I do think you're using technology to better connect people with our past and with each other. So thank you. Thank you for what you're doing.
Yves Ubelmann: Thank you, Brad. Thank you very much.
Brad Smith: You've been listening to Tools and Weapons with me, Brad Smith. If you enjoyed today's show, please follow us wherever you like to listen.
Our executive producers are Carol Ann Browne and Aaron Thiese. This episode of Tools and Weapons was produced by Corina Hernandez and Jordan Rothlein. This podcast is edited and mixed by Jennie Cataldo with production support by Sam Kirkpatrick at Run Studios. Original music by Angular Wave Research. Tools and Weapons is the production of Microsoft made in partnership with Listen.