A New Era in Robotic

Infrastructure Inspections
Sophisticated methods are reducing
costs and increasing safety
A U GU S T 2 0 1 8
Introduction
T he process of making sure that highways, pipelines, refineries and other
infrastructure are sound is critically important—but it is not easy. It often
requires getting a good look at assets that are high up on towers, buried
in the ground or under water—or close to hazardous materials or in hostile
conditions. And to get a good look, typically, companies have had to send
people to those difficult-to-reach and dangerous locations using complicated
and expensive methods.
Increasingly, however, companies are looking to mobile robots, instead
of humans, to perform those infrastructure inspections. The idea of having
machines handle these difficult tasks is not new, but the use of robot inspec-
tors has been fairly limited. Now, however, that is changing. “We’re getting
to a level of sophistication with the technology, with the workflow, and with
the culture in organizations where we’re starting to see a steep ‘hockey stick’
growth curve in the industry,” says Earnest
Changes on many fronts have enabled
Earon, co-founder and CTO of PrecisionHawk,
new capabilities and new uses for
a Raleigh, N.C., provider of drone remote-sens-
inspection robots.
ing applications and data processing services.
“And people are starting to see results and a real bottom-line impact.”
Changes on many fronts have enabled new capabilities and new uses for
inspection robots. Mobile platforms have become more stable, reliable, and
robust. Onboard computers have become smaller and lighter. Sensors have
become more sophisticated. And robot components in general have become
less expensive.
“It’s not any single factor—it’s all of these things coming together,” says
Earon. “As that continues, we’ll see an explosion in adoption and the wide-
spread use of robots in inspection. A lot of capabilities that were possible a
few years ago are now becoming very accessible, as well. So we’re going to
see a real sea change in the industry over the next two to five years.”

August  2018 Mechanical  Engineering Magazine Special Report 2

New Payloads, New Uses
T oday, inspection robots offer various types of mobility. Drones, or un-
manned aerial vehicles (UAVs), can fly above and around assets. Crawl-
ers, which use various types of tracks and wheels, can move along horizontal
surfaces and through pipes—or, with magnetic tracks, up the sides of metal
structures. And submersibles can examine assets that are underwater—or
even immersed in other types of liquid.
To a great extent, the mobile-platform technology used in inspections is
fairly mature. That means the developers of this technology are now focused
largely on the payloads they put on those platforms and on exploring new use
cases for robotic inspectors.
“They are finding ways to use a variety of different sensors and com-
bine them into packages on submersibles, crawlers, or drones,” says John
Grimes, robotics product development manager at ASME. “And they have
a wide range of components to choose from.” These include visible light,

ADVANTAGES OF AUTONOMY
Offshore oil and gas companies inspect thousands of miles of un-
derwater pipelines. They have been doing those inspections using
ROVs tethered to large ships for several years. But truly autono-
mous robots will have many advantages over piloted drones. They
typically will have more on-board navigational and collision-avoid-
ance capability, and some may be able to perform real-time detec-
tion. Such robots would send an alarm to human inspectors with
a geotagged reference of where the leak or anomaly was detected
rather than sending back video over limited bandwidth.

August  2018 Mechanical  Engineering Magazine Special Report 3

infrared, and optical gas imaging cameras; thermal sensors; radar and LIDAR;
ultrasonic (UT) testers, and chemical detectors, among others. These can be
combined and exchanged to perform a variety of inspection tasks.
Inuktun InCommand Robotics, based in Pasadena, Tex., has been doing
just that, with a modular approach to creating inspection robots. The compa-
ny’s crawler was originally built to inspect pipes, but it can now be reconfig-
ured to perform other tasks, as well.
“We build off of the base platform and reconfigure things like cameras to
inspect other types of infrastructure,” says Wes Kirkland, the company’s vice
president. Now, different configurations of the Mobile robots can examine cables
same robot can be used to examine bridges, and pipes underwater, snake through
offshore oil platform risers and chemical tanks. small pipes and sewers, fly along
“With this multi-mission modular concept, you electricity transmission lines, and
have a robotic tool box, and one operator who look for small defects in wind-turbine
is familiar with the crawler can do multiple jobs blades high off the ground—among
without having to train on different platforms,” other things.
he says.
Mobile robots are now being used or piloted in a number of areas. They
can examine cables and pipes underwater, snake through small pipes and
sewers, climb and inspect suspension bridge cables, fly along electricity
transmission lines, check antenna angles on cell phone towers and look
for small defects in wind-turbine blades high off the ground—among other
things. These robots are typically controlled by humans. UAVs, for example,
are usually flown by operators using a radio-control joystick, while crawlers
often have a tether connecting the robot to an operator.
Robots can capture images and data that can be sent back to inspection
experts in a central location.
“The team in the field that’s operating the inspection drone is often not the
team that is making decisions,” says PrecisionHawk’s Earon. Instead, experts
working in a central office can monitor information from many drones, allow-
ing companies to leverage skilled engineering talent across field locations
while speeding up the inspection process. “Your subject matter experts at
the centralized location can view the information, make their assessment and
then send recommendations right back out,” Earon says.
Meanwhile, analytics is playing an increasingly key role in the field. For ex-
ample, people still need to put time and effort into combing through the large
number of images coming from robotic inspectors.
“Somebody has to go through those pictures to understand what’s hap-
pening,” says Earon. “But analytics tools can help you immediately zero in on
problem areas.”
“Robotics platforms are now well-developed. It’s how you use the collect-

August  2018 Mechanical  Engineering Magazine Special Report 4

ed data that’s really making a big difference these days,” says Alex Tepper,
founder and head of corporate and business development of Avitas Systems,
a GE Venture company that provides infrastructure inspections services.
Avitas Systems is focused on “sensor-based analytics” that feeds sensor
data into a cloud-based artificial intelligence (AI) system. “We take what the
robots are seeing or sensing and apply AI to that data to automatically detect
defects, like corrosion or cracks.”
Some companies have also been exploring the use of machine-learning ca-
pabilities. The Southwest Research Institute in San Antonio, Texas, has devel-
oped a Smart Leak Detection (SLED) system to find hydrocarbon leaks. The
system, which can be mounted on a UAV, fuses data from two off-the-shelf
cameras—thermal imaging and visible light—to identify leaked hydrocarbons
on the ground. To “train” the machine-learning algorithms, developers operat-
ed the system under a range of conditions over the course of a year.
“We simulated a lot of leaks in the summer, fall, winter, spring and on a
variety of surfaces—grass, gravel, dirt, concrete,” says Maria Araujo, manag-
er of R&D at the institute. “SLED is now able to not only detect hydrocarbon
leaks, but it can also differentiate between them. If you were flying over a
pipeline, for example, you would be able to identify crude oil or gasoline or

August  2018 Mechanical  Engineering Magazine Special Report 5

diesel on the surface.” SLED also minimizes false alarms, she says: “If you
were to throw both gasoline and water on grass, it can tell the difference.”
And it can detect very small spills, as well. With a typical pipeline, Araujo
says, traditional methods would miss leaks smaller than 1,000 barrels per
day—while SLED would be able to detect spills as small as 20 barrels.

August  2018 Mechanical  Engineering Magazine Special Report 6

Making the Business Case
R obotic inspections offer a number of advantages over traditional meth-
ods—starting with costs. Inspections that require skyhook trucks, dig-
ging up gas lines, or flying human inspectors over pipelines are expensive. So
too is the practice of taking assets offline for extended periods of time while
they are inspected.
Robotic inspections can reduce such costs significantly. A Michigan
Department of Transportation study estimated that a typical highway bridge-
deck inspection using people and a bucket truck takes about eight hours and
costs about $4,600. The same work done with a UAV equipped with infrared
and high-resolution cameras costs about $250, while also minimizing the
need for lane closures that create congestion and delays.
Robots also speed up the gathering of data, and ultimately, corrective
action. Take, for example, rail inspections: “If a manual inspector walking
the tracks sees a potential issue, it might take a week to get that information
processed so that they can send a team out—but you have trains running
every hour on that track,” says Tepper. “The faster you can get the results,
the better.” With sensors feeding track images back to experts in a central
location, the entire process is accelerated significantly.
Robots also offer increased safety. Traditional inspections often involve
people moving across scaffolding to look at factory equipment, using ropes

August  2018 Mechanical  Engineering Magazine Special Report 7

and harnesses to reach wind turbine blades, climbing built-in ladders to view
towers—in short, humans working in hazardous conditions.
“If you talk to people who have been involved in power line inspections for
very long, they’ll usually tell you they know somebody who has died doing it.
It’s a risky environment,” says PrecisionHawk’s Earon. “Using a drone could
keep humans from having to dangle out the windows of a helicopter 100 feet
above the ground to survey power lines.”
A key concern is the inspection of confined spaces, such as large chemical
tanks, sewers, underground cable galleries, and silos. Humans entering those
spaces can encounter risks ranging from dangerous gases and explosions to
slippery surfaces.
“Industry is trying to eliminate the need for confined-space entry by hu-
mans,’ says Inuktun’s Kirkland. “In petrochemical facilities specifically, most
injuries or lost-time accidents are associated with confined-space injury. So
instead of a person going inside a tank or a big pipeline, they can send the
robot down there with a UT probe.” That approach is not only safer—but it
can also eliminate the need to take the tank offline. “For internal tank in-
spection, there is a significant cost in having to drain the tank, clean it out
and make it safe for human entry,” he says. “A robot could potentially crawl
through the product in the tank, while the tank is in service.”

August  2018 Mechanical  Engineering Magazine Special Report 8

Working Smarter
S afety is a key benefit of robotic inspections, but robots now do much
more than relieve humans from the performance of dangerous tasks.
Indeed, it is becoming increasingly clear that there are some things that they
can do better than their human counterparts. For example, they are objective:
“With a drone, everything is recorded, time-stamped and geo-referenced, and
that information is there as part of a permanent record of that facility,” says
Earon. “There is no subjectivity in it, like you can have with different human
inspectors. The machines are very precise about it.”
Southwest Research Institute’s Araujo points
to the SLED system’s ability to identify pools“With a drone, everything is recorded,
of spilled hydrocarbons that human inspectors time-stamped and geo-referenced, and
would likely miss. In addition, her company that information is there as part of a per-
is now working with the U.S. Department of manent record of that facility.”
Energy on a methane-detection version of the
system that uses mid-wave infrared cameras.
“It’s the same concept as SLED, but now we’re trying to find a gas,” she
says. “Methane, because it is a gas, is not stationary, it varies with the wind,
and it’s a substance that people cannot see with the naked eye or smell it.”
Even when looking at specialized infrared camera images, she says, it can

ASME’S ROBOTICS FOR INSPECTION AND

MAINTENANCE FORUM
In June 2018, ASME convened industry leaders and decision makers
for the first Robotics for Inspection and Maintenance Forum (RfIM)
in Houston.
The two-day event consisted of presentations, panel discussions,
case studies, and live UAV demonstrations by representatives of
many of the leading companies in robotic inspection. Among the
themes explored during the event were:
• How autonomous technology solutions are changing business
models and enhancing business transformation;
• How to optimize asset management through data analytics, data
workflow, modeling, digital twin, and artificial intelligence;
• How to guide the value chain to meet utility and petrochemical
industry needs;
• How to drive technology adoption through knowledge dissemi-
nation, workforce development, standards development, and confor-
mity assessment solutions.

August  2018 Mechanical  Engineering Magazine Special Report 9

be difficult for humans to see the methane—but the system’s algorithms are
proving adept at spotting it.
Robotic inspections also open the door to doing things differently and
developing new ways of working. For example, as inspections become safer
and cheaper, companies can conduct them more frequently and even inspect
more types of assets—all of which could help them detect problems earlier
and resolve them before they become major issues.
Companies can also take advantage of the inherent repeatability of robots.
Robotic platforms can be navigated precisely to the same location time after
time, providing the same view of an asset over and over again. This essen-
tially eliminates the variables involved with human-based inspections, where
angles and assessments are likely to differ over time. “By using a robot to
inspect a pipeline several times, you can build
up a data set that could be used to show or “It’s a different mind set—it’s being
even quantify degradation over time,” says proactive instead of being reactive. And
Kirkland. that’s almost a new paradigm for the
With growing amounts of such consistent, industry.”
objective data, companies can make better
use of analytical tools to predict which assets are likely to fail under which
conditions, before an actual problem develops. This makes it possible to take
a risk-based approach to inspection and maintenance, rather than a time-
based approach based on a pre-set schedule—helping companies reduce
asset downtime and optimize the use of crews. “You want to be able to say, ‘I
need to go out to this part of our power line this month, because if we don’t,
we’re going to have a right-of-way incursion from a tree—and the next time
there’s a big storm, it’s going to take the line down,’ ” says PrecisionHawk’s
Earon. “That’s the kind of thing that people are looking at now, because the
analytical tools are there.”
Companies can also incorporate the wealth of data coming from inspection
robots into “digital twins”—virtual models of assets that can be used to plan
and manage maintenance and utilization.
“Drones can be in for the initial 3-D mapping used to create the digital
twin,” says ASME’s Grimes. “Then, over time you can keep adding inspection
data to the digital twin to let inspectors see what has changed from the last
time it was inspected.”
In this way, robotic inspection can provide a powerful means to keep digital
twins up to date and in sync with the actual physical asset.
The plethora of new approaches makes it difficult to determine broadly
what the ROI of the technology will be. “It’s not always easy to compare what
you can do now with what you used to do, because they can be so different,”
says Earon.

August  2018 Mechanical  Engineering Magazine Special Report 10

 For example, operators of solar farms have traditionally done little in the
way of preventive site inspection. Instead, they have waited until they see a
significant degradation of power output and then sent crews out to find the
problem. A UAV equipped with a thermal camera, on the other hand, can cov-
er “many megawatts worth of panels at a time, and produce a precise picture
of the temperature variation among all of those panels to identify problems
early on,” he says.
In one case, a PrecisionHawk drone found that a solar farm was experi-
encing a shortfall of 10 MW that the operator didn’t know about. “And they
weren’t going to know about it for perhaps another five years,” says Earon.
“So there was a pretty compelling business case for the inspection,”
“It’s a different mind set—it’s being proactive instead of being reactive. And
that’s almost a new paradigm for the industry.”

August  2018 Mechanical  Engineering Magazine Special Report 11

Extending the Reach of Robots
A s sophisticated as today’s inspection robots are, they are still in the
early stages of their evolution. Many companies are now focused on
expanding AI to enhance inspection capabilities. “For us, the most exciting
developments recently have been around AI, and applying it to what the
robots are sensing,” says Avitas Systems’ Tepper. “We believe that’s going
to be the future of industrial inspection—using AI to take the knowledge that
the inspectors have in their heads and start to codify that and apply it in the
robots’ work.”
Meanwhile, many observers expect to see more navigational autonomy
in the platforms themselves, which today are typically guided largely by
humans. UAVs, for example, are likely to have more navigational and col-
lision-avoidance capabilities on board. Or, says Kirkland, “you might have
three or four magnetic UT inspection robots along the outside of a tank, guid-
ing themselves as they swarm and crawl around the tank simultaneously.” His
company has developed a prototype of such a crawler, and he expects to see
more autonomy in robots in the near future. “Car companies and others are
already experimenting heavily with autonomous vehicles,” he says. “Those
algorithms, as they progress, will become more commercially available and
then more adaptable to a robotic platform.”
Some also expect to see greater autonomy Even with the growing sophistication
in terms of the onboard systems that detect and autonomy of robots, humans are still
problems. Southwest Research Institute is likely to be involved in inspections for the
already moving in that direction with the SLED foreseeable future—particularly when it
system, which embeds processors and algo- comes to UAVs.
rithms in the robotic platform itself, rather than
in a back-end system. “It’s able to do the hydrocarbon detection from the
camera images in less than two seconds—to say, ‘yes there’s a hydrocarbon
here and this is what it is,’” says Araujo. “It can do autonomous detection in
real time, and send an alarm with a geotag reference of where the leak was
found. It’s doing the computing live, at the edge.” Human inspectors can
then respond quickly by asking for more data or sending repair crews out.
That edge computing is especially important in remote locations, where there
may not be enough wireless bandwidth to stream live images back to human
inspectors or a centralized computing platform.
Even with the growing sophistication and autonomy of robots, humans are
still likely to be involved in inspections for the foreseeable future—particularly
when it comes to UAVs. Current FAA regulations prohibit the flying of these
vehicles past the point where the operator can see them, or beyond visual

August  2018 Mechanical  Engineering Magazine Special Report 12

line of sight (BVLOS). The FAA does grant some waivers that allow BVLOS
flight in specific cases, but not many. However, BVLOS operations have
tremendous potential for robotic inspection. For long distance, horizontally
oriented work—such as railways and electricity transmission line inspec-
tions—BVLOS is important to making robotic inspections cheaper and more
commercially attractive.
The BVLOS capabilities themselves are essentially in place. For its part,
PrecisionHawk has developed a drone outfitted with the needed technology:
It can detect other aircraft within a 10 kilometer radius, uses GPS technology
to constantly let operators know its location and trajectory, and has a hybrid
engine designed for long flights. “The challenge has been building a safety
case so that the regulators are comfortable—and the technology is now at a
point were we can demonstrate that,” says Earon.
The FAA is working to integrate drones into the airspace in the U.S., and
several companies are helping in that effort. This initiative is making progress,
says Avitas Systems’ Tepper, whose company is working with the FAA on
getting exemption for flying beyond visual line of sight (BVLOS). “We expect
regulations to turn relatively quickly—and we’re excited because that will
open up new use cases for robotic inspections,” he says.
Beyond that, however, regulators are not likely to approve fully autonomous
operation of UAVs any time soon because of safety and privacy concerns
says Earon. “I don’t think that in the near future we’re going to see drones
deciding to take off, fly away, do their work, and come back without any hu-
man ever knowing about it,” he says. A more likely model, he says, will be the
use of many drones that are overseen by a small number of human managers
who can intervene when necessary. “It’s a one-to-many scenario, rather than
the one-to-one scenario that we have now,” he says. “It’s a human ‘on the
loop’ rather than ‘in the loop’ model.”
More broadly, Earon continue, humans and robots will continue to have
complementary roles in inspections. “Machines are incredibly good at being
repeatable and being precise. Humans are very good at being subjective
and being interpretive, and they have a great deal of information and a priori
knowledge about these problem domains. They will continue to have a role
in things like deciding when you need to take action. So there’s going to be
human involvement in this for a very long time.”

August  2018 Mechanical  Engineering Magazine Special Report 13

 Mechanical Engineering Magazine Special Reports is a series of multimedia
projects intended to provide insights into evolving technology areas.
Each report includes a white paper, a feature article in Mechanical
Engineering magazine, and a multipart set of videos looking at a key
technology reshaping industry.

The Mechanical Engineering Magazine Special Reports series is found at

go.asme.org/MEmagazine-special-reports

ASME is a not-for-profit membership organization that enables collaboration,

knowledge sharing, career enrichment, and skills development across all en-
gineering disciplines, toward a goal of helping the global engineering commu-
nity develop solutions to benefit lives and livelihoods. Founded in 1880 by a
small group of leading industrialists, ASME has grown through the decades to
include more than 130,000 members in 151 countries, and its reach extends
to government, academia, and industry.

To learn more, go to ASME.org.

“Mechanical Engineering Magazine Special Reports: A New Era in Robotic

Infrastructure Inspections—Sophisticated methods are reducing costs and
increasing safety” is a publication of ASME and Mechanical Engineering mag-
azine. Copyright © 2018 ASME.

The American Society of Mechanical Engineers®

ASME®