Coal Age explores the future of mine mapping and management with drones
by carly leonida, european editor
The use of drones in mining has experienced a boom in recent years. Aside from being widely available and affordable to use, they offer a multitude of safety benefits, particularly in mapping applications.
Drones support an automated workflow and allow data to be collected quickly, more frequently, in many cases, with better coverage than through manual methods. Campaigns that traditionally required a full team for days or weeks can now be completed in hours or days with the help of drones.
3GSM is dedicated to the detailed analysis of rock and terrain using 3D images or models generated using drone-collected data, explained Markus Pötsch, customer support and sales manager. “The use of drones has led to the collection of more comprehensive data set,” he said. “Data from highwalls and bench faces can now be gathered at the same time as from the floor and top of benches. Drones have the ability to change their flight path, altitude and camera angles easily, which is difficult or even impossible with pure terrestrial imaging.”
The 3GSM team has noticed that mapping data gathered by drones usually shows a higher level of detail compared to data gathered terrestrially. The coverage is more complete, and data sets have a high level of redundancy, which is important for 3D model generation.
“The amount of data collected by drones for mining projects can be overwhelming,” Pötsch said. “It’s important that software products can handle this without compromising performance and accuracy. Mining operations, particularly those in remote areas, may suffer from suboptimal network connections, so it’s also important to have software running on local computers as well as in the cloud. The software should be able to process the data and display the results on a standard computer and it should be easy to learn and easy to use in order to provide real value to people and operations.”
“Another advantage is that drones can be deployed during normal operations (except for blasting) without having to stop any mining activities,” said Delta Drone International CEO Christopher Clark. “Terrestrial scanners require multiple setups, yielding digital terrain model data sets with large gaps making them inferior to drone-captured data. These are then not suitable for producing complete pit models on a daily basis.”
Oftentimes, mining operations need to stop while the surveyor gathers data, which means loss of production.
LiDAR Above and Below Ground
Drone-mounted light detection and ranging (LiDAR) technology has evolved significantly over the past five years, and COVID-19 further accelerated its adoption due to the fact that manned aircraft could not easily traverse borders for nearly two years. Above ground, drone-mounted LiDAR scanners can be used to quickly capture entire and accurate digital twins of stockpiles and infrastructure for volume calculations, condition monitoring, change detection, reverse engineering, and space management, among other things.
Clark explained: “Traditionally, drone LiDAR technology has been a clumsy collaboration between first generation sensors and bolted on laser scanners from various manufacturers. These components didn’t always integrate well and required manual intervention at times. In addition, the excessive physical weight of the sensor limited operations to large multi-rotor drones, which capped flight endurance to around 12 minutes.
“The current generation of LiDAR sensors are significantly lighter (around 750 g) and better integrated, which allows us to operate fixed-wing vertical take-off and landing (VTOL) drones. These models can cover over 350 ha in a single 90-minute flight.”
While surface mining operations have seen many benefits, the largest impact of drones over the past three to five years has been in GPS-denied environments, particularly underground. Drones provide a safe and efficient method to gather excavation data, including areas that have experienced a fall-of-ground.
“However, an autonomous drone equipped with LiDAR technology and a camera can enter excavations without endangering personnel as the drone operator resides in a safe position underground,” Clark said.
As drone technology becomes widely accepted, new technologies are continually coming to the fore. With each advancement comes better artificial intelligence (AI) capabilities, more stable flights, and smaller airframes that have integrated or modular cameras.
Drone Nerds specializes in helping enterprises, including those in mining, to find the best solution for their needs. Giovanni Ruiz, director and technical solutions engineer, explained.
“The most important benefit drones offer within mining applications is their ability to access tight or unstable areas without sending in ground crews. Drones can be customized with different payloads for different purposes, including optimized gas imaging (OGI) cameras, thermal cameras, or simultaneous localization and mapping (SLAM)-based LiDAR equipment for mapping.
Carlos Fraga, director of research and development, added: “UAVs that are equipped with the right tools and payloads will be able to ‘see’ and capture data much faster than a team performing the same activities on foot.”
Dr. Farid Kendoul, CTO and co-founder of Emesent, said, “Drone-mounted rapid scanning methods enable data collection to occur at regular intervals, improving recognition of convergence trends and closure rates and enabling full autonomy of convergence monitoring.”
SLAM uses range and sensor data to estimate the motion of the range sensor itself. Once this motion has been estimated, the range data can be projected into a common coordinate frame to produce a 3D map. The estimated motion can also be used to control the robotic platform carrying the sensors. In the case of Emesent’s Hovermap system, LiDAR and internal measuring unit data are used to create maps and control the drone’s flight.
Handling Big Data
Jamie van Schoor, CEO and founder of Dwyka Mining Services, added: “We view drones as a platform to provide repeatable remote sensing and visualization capabilities across mine sites. The blended sensors that are attached to drones are extending the information we used to see based solely on photogrammetry into LiDAR data.”
The complexity of mine sites means that there is no one-size-fits-all solution when it comes to mapping and visualization technologies, and there is usually a trade-off around the cost of technology, resourcing, plus the time it takes to gather data versus the robustness and reliability of the data required.
One of the challenges that van Schoor and many of his peers mentioned is the processing of data post-survey.
“Edge processing means that companies are now able to access their scan data much more quickly,” he said. “But a big challenge that many mines face is how to manage the growing data sets with existing ICT infrastructure, because the two aren’t necessarily keeping pace. Cloud processing and storage for very large point cloud data sets is the obvious answer, but it does bring other challenges around ensuring connectivity and security.”
Ben Vagg, global product manager for surveying at Datamine, said, “With a standard type of survey instrument, you might collect tens of thousands of data points. Whereas with drone-based instruments, we’re talking tens of millions, if not hundreds of millions of data points. So, there is a discussion to be had around how much data mines realistically need. Are they getting any more benefit from an analysis point of view, versus how long it takes to process that data?”
Cloud-based processing and decimation can be of help but again, they depend upon the mine’s connectivity as well as upload and download speeds. There is an opportunity for mines to look at smarter ways of handling very large datasets.
“In open-pit applications where there’s more photogrammetry used rather than LiDAR data capture, the turnaround time is significantly longer, but it does generate a visual representation of the mine,” Vagg said. “A mixture of photogrammetry and LiDAR capture for surface operations seems to be the most appropriate way to go today. Underground, doesn’t require as much colorization, so LiDAR is adequate.”
The Value of Autonomy
Different drones have different autonomous capabilities; some have none at all, while others offer a basic return-to-home feature in which the drone automatically returns to the point of origin if it loses connection with the remote controller.
“Obstacle sensing is another popular feature,” said Drone Nerds’ Fraga. “During a flight, the pilot may have limited control over the drone when flying through areas where a visual line of sight is limited. With obstacle avoidance, a drone is set to fly an autonomous mission and can recognize potential obstacles, thus adjusting speed, altitude or direction independently.”
Some autonomous robots or drones can be given high-level objectives, oftentimes called waypoints, from a human operator to plan a mission. In a mining environment this usually consists of “takeoff and fly 40 meters forward into an open cavity” then “fly up 60 meters” to map the rest of the cavity before finally returning home. Autonomy level 4A, as set out by Exyn in its 2021 whitepaper “Defining Levels of Aerial Autonomy,” simplifies this by allowing the operator to define an area of interest for the robot to explore. For example, a survey team can draw a 3D box around a cavity of unknown size and the robot will choose points of interest itself to capture a complete map of its environment. This will enable use cases such as search and rescue, rapid infrastructure inspection, and fast digital twin creation for construction environments.
The ExynAI software from Exyn Technologies offers this capability and is platform-agnostic, so it can be used to autonomously power a variety of airframes for different mining applications.
Emesent’s Hovermap system has an Autonomous Waypoints mode which enables it to autonomously fly beyond line of sight and communication range, even in GPS-denied environments using waypoints provided by the operator on a tablet and the Guided Exploration feature. Hovermap takes care of the rest, navigating to the waypoints, mapping the area, and keeping itself and the drone safe from obstacles as small as 2 mm.
Dr. Kendoul explained: “Hovermap also has a Pilot Assist mode that provides omnidirectional collision avoidance, line of sight GPS-denied flight capability, and stability control. Basically, it acts as a safety bubble for the drone.
“We’re also commercializing the extended autonomous capabilities of Hovermap that were developed as part of our participation in the DARPA Subterranean Challenge. This includes Hovermap-enabled drones autonomously exploring beyond line of sight and standard communication range, defining their own missions, and navigating into unknown underground environments with the objective of exploring and searching the environment.”
What Will the Future Look Like?
Today, we are just scratching the surface of how drones and robotics can be deployed in mining.
“There is a growing interest in more integrated and autonomous systems and operations,” Dr. Kendoul said. “Ultimately, the application of robotics, machine learning and AI has the potential to enable full workflow automation of a mine, joining up multiple autonomous processes so that the mine itself becomes the autonomous system, with key decisions made by an integrated AI layer optimizing the entire process.
“Multi-agent, fully autonomous drones and robots could form a pivotal part of the core infrastructure, acting as mobile sensors and constantly collecting and live streaming 4D data to a data management platform on the surface, with real-time automated decision making and actions enabled by machine learning and AI.
“Digital twins would run thousands of ‘what if’ simulations per second analyzing and validating data from multiple sources to identify insights, anomalies and optimizations. Those outputs would feedback 24/7 to autonomous robots that set to work to make the changes in the real world, with the whole cycle continuing in perpetual motion.”
Achieving complete workflow automation is a challenge, and a vital step in this journey is the ability to integrate data captured from every process, unmanned vehicle and data collection platform in the mine onto a single analytical platform.
This article was adapted from an article that was published in the July 2022 edition of Engineering & Mining Journal (E&MJ).