SKANSKA:

Reducing risk and improving safety with Smartvid.io, Autodesk’s BIM 360
and machine learning

OVERVIEW / BACKGROUND
Skanska USA is one of the largest, most
financially sound construction and development
companies in the U.S., serving a broad range
of clients including those in transportation,
power, industrial, water/wastewater, healthcare,
education, sports, data centers, government,
aviation and commercial. In 2016, their work in
building construction, civil and power/industrial
construction, commercial development and
infrastructure development (public-private
partnerships) generated $6.8 billion – or
roughly one-third of Skanska’s global revenue.

SKANSKA’S FOCUS ON SAFETY

Skanska’s purpose is to build for a better society.
This core mission led to the commitment to
pursue an Injury-Free Environment®, which
was launched in 2003 as a way to build a broad
safety culture with practical applications. When
Richard Cavallaro, then President of Skanska
USA, was asked to weigh the obstacles facing
the $10 trillion global Architecture, Engineering
and Construction (AEC) industry, he replied
“safety is number one by far.” Given Skanska’s
company-wide commitment to safety, the
internal operations, safety and innovation teams 2112 Pennsylvania Avenue, Washington DC
are always looking to improve every stage of
the risk monitoring and prevention process.
INTRODUCING SMARTVID.IO from heavy equipment, and highlighting
proper housekeeping. Team members were
Skanska began piloting Smartvid.io, an industrial
photo and video analytics platform in 2016 to also instructed to say terms like “Best Practice”
manage media captured from diverse sources, or “Improvement Opportunity” so the video
such as its drone program or field progress could be searched by relevant audio tags.
walks. Smartvid.io provides a single location for
all photo and video content from current and IMPLEMENTATION–
past projects. Once in Smartvid.io, photos and
videos are assigned labels (called “SmartTags”) AUTODESK BIM 360
based on their visual content, such as a crane,
as well as any accompanying audio content like
AND NVIDIA
Skanska then began testing Smartvid.io’s
voiceovers or recordings from field personnel, by integration with Autodesk’s BIM 360, which
leveraging machine learning-powered technology. teams have used since 2008 for document
When Smartvid.io introduced SmartTags for management, quality control, and other field
detecting common safety risks like missing activities. Once integrated, jobsite photos
personal protective equipment (PPE), Skanska’s stored in the BIM 360 database - whether
innovation and safety teams began exploring added by project managers, engineers, or
how it could apply the technology to its safety subcontractors - automatically synced to
programs. Skanska Environmental, Health & Smartvid.io by project. Next, the SmartTagging
Safety Director Jason Timmerman explained: engine automatically tagged potential safety
hazards like PPE violations (SmartTags included
“We were looking to explore two aspects of our safety ‘Missing Hard Hat’ and ‘Missing Safety Vest’).
program. First, we wanted to validate that the SmartTag
technology could pick up on the safety hazards that
an experienced safety manager would typically find HOW IT WORKS
and help codify best practices and improvement (NVIDIA-powered machine learning)
opportunities. Second, we wondered if we could leverage
the tagged video to pass along safety knowledge to all The Smartvid.io Smart Tagging engine, nicknamed
personnel on a job, to help them see examples of the “VINNIE”, leverages machine learning (also called “Deep
risks specific to each phase of their own project. By learning”) technology powered by NVIDIA. NVIDIA’s
having an “extra pair of eyes that doesn’t sleep” looking Quadro M6000 and Quadro GP100 boards power the
at our photo data, we could better identify both positive deep learning training and model production hardware
examples and areas for improvement that we might used by Smartvid.io to build their vision and speech
otherwise miss and grow our safety library. We’re also models tailored to the needs of construction. When
exploring how to tie Smartvid.io technology into our Skanska uploads photos and videos to VINNIE, NVIDIA’s
proprietary EHS planning platform called PlanIt, in order GPUs power the Amazon cloud services doing the real
to mitigate risk prior to worksite tasks taking place. time analysis. Smartvid.io was selected as one of the top
two “most disruptive AI companies in 2017” by NVIDIA
Skanska tested Smartvid.io’s SmartTag from a pool of over 600 candidates.
functionality to help with an emerging best
practice: capturing narrated videos of project
walks at key project phases, highlighting
proper implementation of safety practices
and potential risks. Using GoPro cameras and/
or the Smartvid.io mobile app via an iPhone,
site safety managers took videos narrating
observed examples of safety best practices
and potential hazards. Common examples
included Skanska’s “ladders last” policy, ensuring
clear entry and egress paths, observing crane Image tagged as workers having “No high visibility PPE”
paths, personnel maintaining safe distance (Image courtesy of ENR)
 “On today’s construction site, nearly every person has
a smartphone they use to capture photos and video,
RESULTS
in addition to the many other jobsite monitoring Using Smartvid.io to enable video
systems such as fixed cameras and drones. Additionally, inspections and analysis of jobsite photos
systems like BIM 360 are collecting massive amounts captured in BIM 360 has shown excellent
of photo data from issues, checklists, and other field potential for Skanska’s risk monitoring and
processes. It is simply not possible to manually review prevention efforts. Gains included:
all of the photos in this deluge of data, let alone
extract insights or trends over time. Smartvid.io’s • Existing photos in BIM 360 were aggregated
photo and video machine learning analytics makes it and indexed automatically, eliminating time
possible to turn all of those jobsite observations into spent on manual organization and retrieval
actionable data for the project team, to ultimately
deliver high quality and safer projects,” said Peter • Improved jobsite risk analysis by
Billante, Autodesk’s Senior Manager for BIM 360. automatically tagging safety best
practices and areas for improvement
• Managers could use search and collaboration
features to find photos, full videos and
video snippets by relevant tags, and
share dynamic links to access them with
both internal and external colleagues,
• The “video safety job walk” method was found
to be promising for communication of phase-
specific project risks and for highlighting
proper approaches to them. Safety directors
used audio tags to pull site-specific visual
examples of risk and best practice, filtered by
type and phase of project, for safety trainings.
Image tagged as workers having “No gloves” Skanska has also created a dynamic library of
(Image courtesy of ENR) all these safety and best practice examples to
easily retrieve and share for future training use.
NEXT STEPS
In the next phase, Skanska plans to sync more BIM 360 projects to Smartvid.io and increase the
frequency of captured videos in the safety inspection process in order to get more examples of proven
practices and improvement opportunities. This will help teams get broader safety coverage of its projects
and increase the number of site-specific video examples available for training use.

This pilot has also been a vital step towards a company goal: moving away from project-specific and
application-specific data silos, towards a repository of Skanska-wide data across businesses. Skanska has
thousands of projects worldwide, all generating large volumes of photo and video content; how might
they be used to look backwards and mine years of data for business insights to inform the future of
safety, quality, and more?

ANTHONY COLONNA, Senior Vice President,

Innovative Construction Solutions, Skanska USA
“Machine-learning and artificial intelligence are exciting tools for an industry
that collects a vast array of complex, unstructured data such as images,
video and point clouds. We’re on our way to unlocking the potential of that
data; from mining insights of historical data to filtering and analyzing data
in real-time in order to improve decision-making on the jobsite.”

350 Fifth Avenue, 32nd Floor 101 MAIN STREET, 14TH FLOOR
New York, NY 10118 CAMBRIDGE, MA 02142
https://www.skanska.com http://www.smartvid.io
Skanska is one of the largest, most financially sound Smartvid.io is a software company headquartered in
construction and development companies in the U.S., Cambridge, MA which provides machine learning tools
serving a broad range of clients. for the management and analysis of industrial media.