The Software that’s Saving Corals
A powerful visual analytics platform is translating photographs of coral reefs into navigable 3D maps that allow researchers to delve into the depths of these critically important and delicate ecosystems. Developed by Vid Petrovic, a computer scientist and software engineer with the Cultural Heritage Engineering Initiative at the UC San Diego Qualcomm Institute, the platform, called Viscore, is being used by researchers at Scripps Institution of Oceanography to study coral reefs around the world.
A 100 Island Challenge team member from Scripps Institution of Oceanography captures imagery of a coral reef.
Transforming Reefs into Virtual Maps
The 100 Island Challenge brings together researchers from Scripps Institution of Oceanography and the Qualcomm Institute (QI) to study the structure and health of coral reefs, and understand how they respond to climate change and inform their management. The project uses Viscore to digitally recreate plots from 100 reefs across the tropical Pacific, Caribbean and Indian Oceans.
Historically, researchers have relied on static, 2D representations of their study sites, such as photographs and measurements taken by divers in the field, to examine coral reefs from the laboratory. A number of factors—location, weather, water temperature, depth and finite reserves of breathing gas—can impact the amount of data that divers collect. Now, with Viscore, marine scientists at Scripps Oceanography can spend unlimited “dive time” exploring coral reefs—virtually.
“This is the kind of data that scientists haven’t had access to before. So it’s really exciting to see what sort of a difference it makes,” said Petrovic.
A 3D model of a coral reef plot near Kaho’olawe, southwest of Maui, imaged in 2016. The box highlights a 10 x 10m study region, with dots marking sample points used for coral cover estimation.
The process of creating a coral reef’s “digital twin” begins with thousands of overlapping images captured by Scripps Oceanography divers. Through a technique called structure-from-motion, researchers transform these images into hundreds of million or even billions of 3D points. These dense point-clouds are the first step in making a reef’s “digital twin,” capturing its unique geometry and color in lifelike resolution. With more than one thousand coral reef plots—some dating back nearly a decade—surveyed over years, the 100 Island Challenge has created what Falko Kuester, director of QI’s Cultural Heritage Engineering Initiative (CHEI) and a professor with the Department of Structural Engineering, calls a “data tsunami.”
“While we often hear about big-data science, this truly is an example of ‘Ocean Exploration in a Sea of Data,’” said Kuester.
Vid Petrovic studies a coral reef plot in CHEI’s Wide Angle Virtual Environment (WAVE), a room-scale immersive virtual reality environment that allows researchers to explore study sites and travel across space and time.
This is where Viscore comes in, transforming billions of data points into vivid, lifelike models that can be experienced, explored and analyzed in UC San Diego’s walk-in virtual reality environments the WAVE and SunCAVE. The immersive virtual reality systems can instantaneously transport researchers through time and space, giving them the power to revisit their study sites from hundreds of miles away.
Petrovic likens the experience of swimming through a virtual reef to “taking a patch of rainforest and being able to see all the life that’s there.” Researchers can inspect these rich ecosystems in detail, identifying coral species and tracking shifts in populations. With each passing year, these records provide a historical archive of coral reefs, one that can help marine scientists determine how corals respond to short-term disturbances, like storms, or to longer-term shifts, such as warming ocean temperatures.
“Viscore facilitates the creation of ‘digital twins’ allowing us to measure, classify, and annotate, as well as quantify change over time,” said Dominique Rissolo, a QI research scientist involved in the project. “This has resulted in observations of ecological processes that would otherwise not be possible.”
(Experience the fascinating coral reef visualizations created with Viscore on the project’s YouTube channel.)
“The tools developed by our colleagues at QI have been invaluable for visualizing coral reefs,” said Brian Zgliczynski, project coordinator for the 100 Island Challenge at Scripps Oceanography. “For instance, we can observe how corals respond to disturbances including storms and variations in sea temperature. While these events can negatively impact the structure and function of coral reefs, we have documented a number of cases where corals can recover and bounce back. This is incredible and is changing our understanding of how coral reefs will respond to a changing world.”
Time series showing change in a coral reef from Palmyra Atoll in the Pacific Ocean.
A New Way to Look at the World
The 100 Island Challenge has fostered a thriving partnership between CHEI’s engineering team, led by Kuester, and the team of marine specialists working under marine ecologist Stuart Sandin at Scripps Oceanography. Each has learned from the other; Petrovic has refined his approach and adjusted Viscore’s algorithms based on feedback from his Scripps Oceanography colleagues, while his counterparts have made note of new measurements they’d like to collect and add to the archive. Petrovic, CHEI’s remote imaging expert Eric Lo and Scripps Oceanography doctoral student Hugh Runyan are also in the process of developing machine learning techniques to automate the otherwise painstaking work of identifying individual coral species.
Beyond the 100 Island Challenge, Viscore supports efforts to explore and document vulnerable historical and cultural sites across the globe, including heritage sites in Mexico, Puerto Rico and Italy, and the marine ecosystem off the coast of Bermuda. Viscore has transported researchers to shipwrecks, underwater cavesand ancient Maya ruins, has aided in digitally preserving indigenous archaeological sites and historically significant buildings, and has enabled the assessment of at-risk national and international infrastructure and life-lines in the face of extreme events.
The platform is featured in multimedia art installations, as well; one recent piece, “Hearing Seascapes,” blends visuals of coral reefs and an original score composed by Lei Liang, QI’s Research Artist in Residence.
“While serving as a fundamental research tool, Viscore is enabling scientifically anchored story-telling, allowing scientists and the ultimate stake holder, the public, to explore and experience the beauty of the environment that surrounds us, and with a bit of luck help create the empathy and stewards needed to preserve it,” said Kuester. “Just imagine what would be possible if we had the resources to create a global coral reef atlas, allowing anyone anywhere to join in the adventure of discovery, life-long-learning and stewardship of this precious resource.”
To donate to the 100 Island Challenge and support coral reef research, contact Adrienne Bolli at abolli@eng.ucsd.edu.
