Automated Progress Monitoring Using Unordered Daily

Automated Progress Monitoring Using Unordered Daily
Construction Photographs and IFC-Based

Building Information Models
Mani Golparvar-Fard, A.M.ASCE 1; Feniosky Peña-Mora, M.ASCE 2; and Silvio Savarese 3
Downloaded from ascelibrary.org by Universidad de los Andes on 09/12/21. Copyright ASCE. For personal use only; all rights reserved.
Abstract: Accurate and efficient tracking, analysis and visualization of as-built (actual) status of buildings under construction are critical
components of a successful project monitoring. Such information directly supports control decision-making and if automated, can signifi-
cantly impact management of a project. This paper presents a new automated approach for recognition of physical progress based on two
emerging sources of information: (1) unordered daily construction photo collections, which are currently collected at almost no cost on all
construction sites; and (2) building information models (BIMs), which are increasingly turning into binding components of architecture/
engineering/construction contracts. First, given a set of unordered and uncalibrated site photographs, an approach based on structure-from-
motion, multiview stereo, and voxel coloring and labeling algorithms is presented that calibrates cameras, photorealistically reconstructs a
dense as-built point cloud model in four dimensions (three dimensions + time), and traverses and labels the scene for occupancy. This strategy
explicitly accounts for occlusions and allows input images to be taken far apart and widely distributed around the environment. An Industry
Foundation Class–based (IFC-based) BIM is subsequently fused into the as-built scene by a robust registration step and is traversed and
labeled for expected progress visibility. Next, a machine-learning scheme built upon a Bayesian probabilistic model is proposed that auto-
matically detects physical progress in the presence of occlusions and demonstrates that physical progress monitoring at schedule activity level
could be fully automated. Finally, the system enables the expected and reconstructed elements to be explored with an interactive, image-
based, three-dimensional (3D) viewer where deviations are automatically color-coded over the IFC-based BIM. To that extent, the underlying
hypotheses and algorithms for generating integrated four-dimensional (4D) as-built and as-planned models plus automated progress mon-
itoring are presented. Experimental results are reported for challenging image data sets collected under different lighting conditions and
severe occlusions from two ongoing building construction projects. This marks the presented model as being the first probabilistic model
for automated progress tracking and visualization of deviations that incorporates both as-planned models and unordered daily photographs in
a principled way. DOI: 10.1061/(ASCE)CP.1943-5487.0000205. © 2014 American Society of Civil Engineers.
Author keywords: Automation; Computer vision; Industry foundation classes (IFC); 3D reconstruction; Model-based recognition;
Multiple view geometry; Photogrammetry; Structure-from-motion.
Introduction collection, processing, and representation are time-consuming and

labor-intensive (Bosché 2010; Golparvar-Fard et al. 2009a; Kiziltas
Accurate and efficient tracking of the as-built (actual) status of et al. 2008; Navon and Sacks 2007). These methods require manual
buildings under construction has been repeatedly reported as a criti- data collection and extensive as-planned and as-built data extrac-
cal factor for success of project control (e.g., Bosché 2010; Bosché tion from construction drawings, schedules, and daily construction
et al. 2009; Zhang et al. 2009; Navon 2007). Such information di- reports produced by superintendents, subcontractors, and trade
rectly supports progress monitoring and control and, if automated, foremen. The quality of the daily progress reports also largely de-
can significantly affect the management of a project. Despite the pends on the data collected by field personnel, which tend to be
importance of progress monitoring, current methods for site data based on field personnel’s interpretation of what needs to be mea-
sured, the way it needs to be measured, and the way it needs to be
1
presented; therefore, the reports may not reveal the actual impact of
Assistant Professor, Charles E. Via Dept. of Civil and Environmental
site circumstances on the construction project (Golparvar-Fard et al.
Engineering, Virginia Tech, 200 Patton Hall, Blacksburg, VA 24061 (cor-
responding author). E-mail: golparvar@vt.edu
2009a; Navon and Sacks 2007). For example, in a daily construc-
2
Dean, Fu Foundation School of Engineering and Applied Science tion report submitted by a drywall contractor, it may be reported
and Morris A. and Alma Schapiro Professor of Civil Engineering and that framing was conducted without specifying the amount of
Engineering Mechanics, Earth and Environmental Engineering, and Com- resources used, the exact location of the work performed, or the
puter Science, Columbia Univ., 510 S.W. Mudd Building, 500 W. 120th St., progress made. Even if progress is measured, it may be conducted
New York, NY 10027. E-mail: feniosky@columbia.edu in a nonsystematic way and metrics may tend to be subjective.
3
Assistant Professor, Electrical and Computer Engineering, Univ. of For example, a concrete subcontractor reports that 60% of the roof
Michigan, 1301 Beal Ave., Room 4120, Ann Arbor, MI 48109-2122. work is complete. Does this mean 60% of the planned area/volume
E-mail: silvio@eecs.umich.edu
of concrete is placed? Or has 60% of the planned labor hours been
Note. This manuscript was submitted on December 21, 2011; approved
on February 9, 2012; published online on February 11, 2012. Discussion spent? Or perhaps 60% of the actual requirement is complete? If the
period open until August 25, 2014; separate discussions must be submitted item being referenced is a small work unit, it may not matter that
for individual papers. This paper is part of the Journal of Computing in much. However, if the references are to the whole task, the as-
Civil Engineering, © ASCE, ISSN 0887-3801/04014025(19)/$25.00. sumption of input/output proportionality could be very misleading
© ASCE 04014025-1 J. Comput. Civ. Eng.
J. Comput. Civ. Eng., 2015, 29(1): 04014025

(Meredith and Mantel 2003). Finally, progress-monitoring reports (e.g., a façade blocking the observation of elements in the interior)
are visually complex. Typically, decision making for corrective or occlusions caused by temporary structures (e.g., scaffolding or
control actions and the revision of work schedules takes place in temporary tenting); and (2) dynamic occlusions, which are rapid
contractor coordination meetings. A wide range of individuals with movements of construction machinery and workers during the time
different areas of expertise and interests often attend these meet- photographs are taken. Developing computer vision techniques
ings. In these face-to-face interactions, progress information needs that can effectively work with such imagery to monitor building
to be easily and quickly communicated among the participants. element changes has been a major challenge.
However, none of the existing reporting methods (e.g., progress In this paper, these challenges are addressed and based on
S curves, schedule bar charts) easily and effectively presents multi- a priori information (4D BIM), and a new approach to monitoring
variable information (e.g., schedule and performance), nor do they as-built elements using unordered photographs is presented. First
intuitively reflect information pertaining to the spatial aspects of using structure-from-motion (SfM) techniques, an as-built point
progress and their associated complexities (Poku and Arditi 2006; cloud model is generated and photographs are automatically regis-
Koo and Fischer 2000). Existing representations cause a significant tered. Subsequently, the as-built point cloud model is registered
amount of information to be inefficiently presented in coordination over the as-planned model and improved by multiview stereo
meetings. As a result, extra time needs to be spent on explaining (MVS). At this stage a new voxel coloring algorithm is used to
the context in which problems occurred rather than understand- generate a volumetric reconstruction of the site, labeling different
ing the causes of the problems, evaluating alternatives to solve the areas according to consistent visual observations while fully ac-
problems, and discussing corrective actions. Therefore, there is a counting for occlusions. The same labeling process is conducted on
need for effective monitoring that allows data to be collected easily the as-planned model to identify occupied and visible areas for
and at almost no cost, processing the information automatically, progress monitoring. Finally, a Bayesian probabilistic model is
and reporting back in a format that can be used by all project introduced to automatically recognize progress deviations by com-
participants. paring measurements of progress with dynamic thresholds learned
Nowadays, cheap and high-resolution digital cameras, low- through a support vector machine (SVM) classifier. This model
cost memory, and increasing bandwidth capacity have enabled quantifies progress automatically and accounts for occlusions and
capturing and sharing of construction photographs on a truly recognizes whether reconstructed building elements are missing
massive scale. For example, on a 20; 000 m2 (200,000 S.F.) build- because of occlusions or because of changes. This makes the pre-
ing project in Champaign, IL, the construction management team sented model the first probabilistic model for automated progress
collects an average of 250 photos per day. Such a large and diverse tracking and visualization of deviations to incorporate both as-
set of imagery, along with the photo contractors and their subcon- planned models and unordered daily photographs in a principled
tractors take (approximately 25 photos/day for each work package) way. The presented model can use existing information without
as well as the photos owners take (approximately 25–50 photos/ imposing an additional burden of explicit data collection on proj-
day), enables the as-built scene to be fully observed from almost ect management teams. In this paper, this model is validated by
every conceivable viewing position and angle during construction tracking and visualizing progress on two building projects. In the
of the project. The availability of such rich imagery—which following sections, previous works on automated progress monitor-
captures dynamic construction scenes at minimal cost—may ing are first reviewed, and subsequently the automated detection
enable geometrical reconstruction and visualization of as-built model is presented in detail.
models at high resolution, which can have broader impacts for the
architecture/engineering/construction (AEC) community.
In the meantime, Building Information Models (BIMs) are also
Previous Work
increasingly turning into binding components of AEC contracts. In the last decade, the capabilities of site data collection technol-
For example, as of July 2009, Wisconsin has established itself ogies have significantly increased. These technologies include
as the first state requiring BIM models for public projects (Yoders barcode and radio-frequency identification (RFID) tags (Kiziltas
2009). In a recent survey McGraw-Hill Construction (2009) re- et al. 2008; Navon and Sacks 2007; Ergen et al. 2007; Jaselskis
ported that 49% of AEC companies were already using BIM and El-Misalami 2003; Echeverry and Beltran 1997), laser scanners
(a growth of 75% from 2007). While the application of BIMs is (Bosché 2010; Bosché et al. 2009; El-Omari and Moselhi 2008;
increasing, Gilligan and Kunz (2007) report that significant atten- Su et al. 2006; Akinci et al. 2006), and wearable computers
tion is devoted to project design and system clash detection. If (Reinhardt et al. 2000; Peterson and Fischer 2009). In this section,
linked with project schedules, BIMs can form detailed chronologi- recent works on terrestrial laser scanning and photography (con-
cal models that allow four-dimensional (4D) [three-dimensional ventional photogrammetry and vision-based) techniques for prog-
(3D) + time] clash detection and schedule quality control to be ress monitoring are specifically reviewed because recent works on
conducted (Gilligan and Kunz 2007; Koo and Fischer 2000). these two types of technologies could potentially automate all steps
Furthermore, they can serve as a powerful baseline for progress of collecting, analyzing, and representing progress and its devia-
tracking and in the visualization of discrepancies (Golparvar-Fard tions from a construction plan.
et al. 2011). Application of these models during the construction
phase can be increased if further potential added values from inte-
grating BIMs with as-built models are investigated. Laser Scanning–Based Systems
Nonetheless, linking unordered photo collections with as- A popular trend in automating progress monitoring is to acquire
planned models for the purpose of monitoring construction multiple depth maps with a laser range scanner, register these point
progress is challenging. First, such imagery is usually unordered clouds using an iterative closest point (ICP) algorithm (Besl and
and uncalibrated, with widely unpredictable and uncontrolled McKay 1992), merge them into a single 3D model (Bosché 2010;
lighting conditions. Second, visibility order and occlusions need to Huber and Hebert 2003; Levoy et al. 2000), and, finally, have it
be considered for successful alignment and measurements. In par- compared with the as-planned model (Bosché 2010; Bosché et al.
ticular, one needs to account for two types of occlusions: (1) static 2009; Gordon et al. 2003; Huertas and Nevatia 2000). Recent
occlusions, which are self-occlusions caused by progress itself examples on the application of laser scanners to construction data
J. Comput. Civ. Eng., 2015, 29(1): 04014025

collection and analysis include construction quality control (Akinci Different methods are presented that can manually, semiautomati-
et al. 2006; Shih and Wang 2004), condition assessment (Gordon cally, and automatically interpret these photographs (e.g., Cordova
et al. 2003), health monitoring (Park et al. 2007), and component et al. 2009; Ibrahim and Kaka 2008; Soibelman et al. 2008;
tracking (Bosché 2010; Bosché et al. 2009; Teizer et al. 2005). Ordonez et al. 2008; Shih et al. 2006; Trupp et al. 2004; Abeid
High-end laser scanners can acquire 3D data with high accuracy et al. 2003; Abeid and Arditi 2002; Nuntasunti and Bernold 2002;
(at approximately 3 mm at 50 m spot size scan resolution and single Everett et al. 1998; Abudayyeh 1997). As digital photography
point position accuracy of approximately 12 mm at 100 m, as advances, the prices for cameras have fallen dramatically (about
Reported by Bosché 2010), yet their cost is in the $100,000 range a few hundred dollars), yet they are able to capture several high-
(Bosché 2010; Furukawa and Ponce 2006). Despite significant resolution (10 Mpixel) images or medium-resolution (2 Mpixel)
research on automated spatial data collection using laser scanners videos at 20 m=s (60 ft=s). Of course, prices on laser scanners will
in various research fields, spatial and temporal resolutions are still drop, yet they are unlikely to catch up with those on cameras in the
limited (Furukawa and Ponce 2006). Other limitations include the near future since their manufactures do not respond to the competi-
discontinuity of spatial information (which requires frequent and tive mass market of digital cameras (Furukawa and Ponce 2006).
sometimes manual registrations), the mixed-pixel phenomenon For these reasons many recent computer vision works are focusing
(Kiziltas et al. 2008), the need for regular sensor calibrations, and on the application of imagery to 3D reconstruction (Furukawa and
slow warm-up time. For example, moving objects in the line of Ponce 2010; Agarwal et al. 2009; Snavely et al. 2008). In what
sight of a scanner do not allow the point cloud model of a given follows, those image-based techniques that have been proposed
component to be captured. In addition, moving machinery and perfor the monitoring of construction projects are reviewed.
sonnel can create noise in a point cloud model and consequently One early technique was the use of conventional photogramme-
cause the expenditure of additional effort on the part of the user to try. Photogrammetric techniques use conventional high-resolution
manually improve the point cloud in a postprocessing stage. Even if cameras and provide high-accuracy site models, comparable to that
the scanner is transferred to a new location, the new scanned point of the best laser scanners, but their spatial resolution is even more
cloud model would still require registration. Like other sensing de- limited, mostly about a few hundred scattered points (Furukawa
vices that depend on the line of sight, as the distance between the and Ponce 2006) and they may require markers (Uffenkamp 1993).
laser scanner and the building components increases, the level of Their analog instruments have many limitations that do not affect
detail that can be captured is reduced. Since these devices are not their application in the production of maps and plans (Moore 1992)
easily portable, they cannot efficiently be used for scanning indoor yet restrict their application to nontopographic subjects. Examples
environments. For these technical reasons, the value of applying of such issues are restrictions in camera rotation, range of focal
laser scanners has not yet been significantly observed in the length, and analysis of orientation data. Their application due to
AEC industry. such technical issues is almost out of date (Moore 1992) and, es-
In a recent study, and based on observations from the majority of pecially given the repetitive nature of progress monitoring, makes
cases where the application of laser scanners has been reported, them unattractive. Over the past few years, with advancements in
Bosché (2010) relates this issue to the low level of automation, SfM techniques, some of those techniques have been revived and
the inherent ambiguity in object detection with laser scanning point are being used to automatically generate the structure and capture
clouds—“no two objects should have the same representation” the motion of cameras (see subsequent discussion).
(Arman and Aggarwal 1993)—and poor efficiency—“in terms of More recently, several vision-based systems have been pro-
the number of elements that can be investigated in a day.” Yet, posed for tracking progress (Ibrahim et al. 2009; Zhang et al. 2009;
Bosché (2010) proposed the first quasi-fully automated systems for Golparvar-Fard et al. 2009b; Lukins and Trucco 2007; Podbreznik
tracking progress using a 3D computer-aided design (CAD) model and Rebolj 2007; Golparvar-Fard and Peña-Mora 2007; Alves and
of a steel structure along with site scans. In Bosché et al.’s (2009) Bartolo 2006; Jung et al. 2004; Wu and Kim 2004). The general
early algorithms, a CAD element is converted to a point cloud process for tracking in these systems is to first capture time-lapsed
model representation. Subsequently, using a point-to-point com- photographs from a fixed location, register digital site images to 3D
parison, the range of as-built and as-planned models is evaluated, CAD models/BIMs in a common coordinate system using camera
and if this range is less than some threshold, then the CAD element pose estimation techniques (e.g., Golparvar-Fard et al. 2009; Song
is recognized. Such an approach may not be robust to different 2007; Golparvar-Fard and Peña-Mora 2007), and then have the site
angular scan resolutions and depends on scanner–building compo- 2D photograph image processed and compared with the as-planned
nent distances. It also depends on the accuracy of registration (in model (Golparvar-Fard and Peña-Mora 2007). Specifically, in
their experiments, 50 mm). In a recent work (Bosché 2010), regis- Golparvar-Fard et al. (2009b) and Golparvar-Fard and Peña-Mora
tration accuracy is improved, and instead of point-to-point compari- (2007) each BIM component is projected on the time-lapsed
son, a surface is recognized for each element, and then this surface images, and the specific areas that represent that component are
is compared with a minimum recognizable surface. The results of further processed to identify whether or not those areas have similar
the researchers’ recognition performance are improved, yet the pre- expected appearances (e.g., concrete versus form). Ibrahim et al.
cision of such a proposed method has not been fully verified, and (2009), Zhang et al. (2009), and Lukins and Trucco (2007) propose
the approach may still be susceptible to partial occlusions (as re- similar semiautomated techniques with differences in the recogni-
ported by the authors themselves). More research must be done on tion step. In these works, precalibrated photographs are compared
the efficient application of laser scanners for automated progress with previously taken photographs. This is done by searching spe-
tracking. cific regions of interest and recognizing progress as regions of
images that undergo significant changes from the previous image
[by computing changes in pixels and by using an Adaboost detector
Image-Based Systems
(Freund and Schapire 1999)]. Then, theoretically the changes are
Over the past decade, advancements in digital photography and compared to a 3D model to calculate the percentage completion of
techniques that process such visual data have led to a signifi- each building component. The approach seems promising given the
cant amount of research reported on the application of site photo- nature of time-lapsed images and has the most automation reported
graphs for various construction management tasks and techniques. so far, yet it has several limitations intrinsic to the application of
J. Comput. Civ. Eng., 2015, 29(1): 04014025

time-lapsed images: (1) since the camera is fixed, small registration In some of these techniques, such as Zebedin et al. (2008), aerial
errors will significantly affect registration and minimize the allo- images are used for reconstructing building models. In others, such
cated image area for each element, making the task of recognition as Agarwal et al. (2009), an entire city is reconstructed from un-
much more challenging (Golparvar-Fard et al. 2009b); (2) the sen- ordered photographs collected from the Internet, or, as in Cornelis
sitivity of the region of interest and detectors to different lighting et al. (2008) and Pollefeys et al. (2008), building façades are recon-
conditions, particularly in the presence of severe shadow lines, af- structed from car-mounted videos, and in Sinha et al. (2008), pho-
fects image processing; (3) progress can be monitored only on the torealistic architectural surfaces are interactively reconstructed. In
closest structural frame of the building to the camera; (4) dynamic Golparvar-Fard et al. (2009a), once the underlying structure and the
occlusions (movement of machinery and personnel), which are motion of the camera are captured, the reconstructed point cloud
common on construction sites, may result in the false detection model is registered over an IFC-based BIM and deviations between
of regions of interest, ultimately resulting in false object detection as-planned and as-built models are visualized both in 3D as well as
[extreme clutter is reported by Zhang et al. (2009) and Golparvar- 2D augmented reality views. Golparvar-Fard et al. (2009a) further
Fard et al. (2009a)]; and, finally, (5) static occlusions (progress it- describe how, given a set of daily site images, a sparse representa-
self) make it difficult to analyze elements farther from the camera tion of a site can be generated. Observed and perceived applica-
(Golparvar-Fard et al. 2009b). In Ibrahim et al. (2009), Zhang et al. tions of the D4 AR system are discussed, and potential automated
(2009), Golparvar-Fard et al. (2009b), Leung et al. (2008), the ap- progress-monitoring algorithms are also road-mapped. However,
plication of a network of cameras is suggested, yet installation costs no implementation of the automated detection of progress devia-
and security and privacy issues versus the value obtained make the tions was proposed. In this paper, a new approach for volumetric
use of such a network less attractive. Other shortcomings are related and dense reconstruction of an as-built scene is introduced. The
to the proposed vision-based approach: (1) a considerable amount presented work in image-based reconstruction is closest to that of
of preparatory work occurs both around and within the construction Furukawa et al. (2009), in which a stereo algorithm is designed for
area of the final component, yet actual progress can occur very building interiors that consist largely of piecewise planar surfaces.
quickly (i.e., an entire prefabricated column lowered into place); However, in comparison to Furukawa et al. (2009), the images can
and (2) variations in the shape of the structure may not actually be widely distributed and have less overlap because the focus is to
occur very frequently; rather, it is often the effects of exterior modi- use the images that are already available on the construction sites.
fication that give visible indications of change. Relating such Furthermore, no prior assumption on piecewise planar surfaces
changes in particular types of events to the completion of the com- (such as Manhattan world) is made because during the construction
ponent is challenging. Zhang et al. (2009) also argue that a work phase of a project, surfaces are not yet complete, and therefore such
breakdown structure (WBS) in their as-planned models signifi- assumptions may lead to an improper automated tracking of prog-
cantly affects results and makes their approach less practical ress. Finally, and most importantly, the quality of reconstruction is
because the approach requires AEC professionals to manually de- not the focus; rather, the focus is on detecting changes in elements
compose the as-planned model to the appropriate level of detail. In given partial occlusions. In what follows, the underlying hypoth-
Golparvar-Fard et al. (2009b), the application of a large number of eses for automated physical progress tracking are outlined.
casually collected site images was proposed, which is one of the
underlying components for this work.
Recently, Quiñones-Rozo et al. (2008), Dai and Lu (2008), and Underlying Hypotheses for Automated Physical
Kim and Kano (2008) reported on the application of close-range Progress Monitoring
digital photogrammetry and imaging techniques to track construc-
tion activities or model machinery. Their works (Quiñones-Rozo In this work, the detection of progress deviations is based on
et al. 2008; Dai and Lu 2008) still require manual detection and a priori information (4D BIM), as well as daily construction photo-
manual matching of feature points (Dai and Lu 2008), placement graphs. Suppose one is interested in monitoring the progress of
of special targets (Quiñones-Rozo et al. 2008), or manual applica- FPRS basement concrete columns (FRPS ¼ form=rebar=pour=
tion of surveying equipment. A substantial amount of human strip) activity. In the proposed approach, the WBS for the 4D model
intervention makes such applications time-consuming and less at- is governed by the level of detail presented in the schedule, i.e., if
tractive for repetitive progress-monitoring tasks. The image- FPRS of all the basement concrete columns is linked to this activity,
processing technique reported in Quiñones-Rozo et al. (2008) still all those elements will become the baseline for tracking progress
requires that one have a clear view of the site to detect excavation and progress for those will be reported in a mutually independent
work. In addition, the pattern detection and comparison in these fashion. In other words, it is assumed that the construction
works are highly sensitive to lighting conditions and may require operation sequence (i.e., workflow) within any given activity is un-
that images be taken under similar lighting conditions. known. Secondly, progress is defined as the observation on the day
on which the element is expected to be placed, and operational de-
tails (e.g., forming stage of columns) are not considered (the for-
Unordered Daily Construction Photography mulation presented in this paper accounts for operational details).
To automate progress-monitoring data collection, processing, and Currently, a superintendent or a field engineer walks around the site
visualization, Golparvar-Fard et al. (2009a) proposed the use of all day every day to observe progress from all possible viewpoints.
unordered daily site photographs and IFC-based BIMs. The focus Subsequently, these observations are compared with paper-based
is on large sites and capturing the entirety of an as-built model from plan information (e.g., construction drawings and schedules), and
images that are casually taken by project engineers in and around deviations are reported back to the project manager. It is assumed
the construction site. The proposed algorithm builds upon a set of that at several stages of these observations, site images are captured
SfM algorithms where the objective is to reconstruct the scene to visually document the work progress. Since these images are
without any strong geometric or semantic priors (Agarwal et al. collected from different viewpoints and under various lighting
2009; Furukawa et al. 2009; Pollefeys et al. 2008; Schindler et al. conditions, they challenge any vision-based system by generating
2008; Sinha et al. 2008; Snavely et al. 2008; Zebedin et al. 2008; swift intensity changes within a short distance of the image and
Cornelis et al. 2008; Snavely et al. 2006; Debevec et al. 1996). by generating two types of occlusions: static occlusions, which are
J. Comput. Civ. Eng., 2015, 29(1): 04014025

automatically extracted from the EXIF tag of JPEG images (avail-
able in all cameras). Finally, for registration of as-planned and
as-built models, it is assumed that at least three distinct control
points are available so that the as-planned model can be superim-
posed on the as-built sparse point cloud model. Finally, it is pre-
sumed that there will be a registration error and that that error needs
to be considered in the formation of the monitoring module.
Overview of D4 AR Progress Visualization and

Automated Monitoring
As shown in Fig. 2, the present work is based on the joint appli-

cation of daily construction photographs, an IFC-based BIM as

well as a construction schedule, to generate D4 AR models and
automatically measure progress deviations. First, the schedule is
Fig. 1. Progress monitoring and challenges; student dining hall fused into the IFC-based BIM by manually linking the elements to
construction project, Champaign, IL (Aug. 27, 2008) (image courtesy activities and creating a 4D baseline model for progress monitoring.
Univ. of Illinois Dept. of Civil and Environmental Engineering) Next, using SfM techniques, an underlying 3D geometry for the
as-built scene is generated that sets a baseline for visual navigation
through registered imagery in the scene. This is completed by cal-
culating camera location, orientation, field of view, and sparse 3D
self-occlusions caused by progress itself (e.g., a façade blocking Cartesian coordinate information of the as-built model (Golparvar-
the observation of progress in the interior) or occlusions caused by Fard et al. 2009a).
temporary structures (e.g., scaffolding or temporary tenting), and Subsequently, the 3D IFC-based BIM is superimposed on the
dynamic occlusions, which are rapid movements that occur when integrated as-built point cloud and camera model (camera 3D po-
photographs are taken (e.g., movement of construction machinery sitions and viewing directions). The Euclidean point cloud model,
and crew). Fig. 1 highlights the technical challenges of a vision- along with the camera parameters, is fed into the MVS algorithm
based progress-monitoring system. (Furukawa and Ponce 2010) to improve the density of the recon-
As far as the as-planned model is concerned, two assumptions struction. The results are placed into an as-built voxel-coloring and
are made. The first assumption is that an IFC-based BIM is gen- labeling algorithm developed in this research to get a dense recon-
erated based on the most updated construction drawings. An archi- struction of the as-built site and label scene for as-built occupancy.
tect’s supplemental instructions (ASIs), requests for information Using a similarly structured voxel-coloring algorithm, the as-
(RFIs), requests for proposal (RFPs), or change orders are reflected planned scene is also labeled for occupancy and visibility. These
in the revised plan model. The second assumptions is that the most two labeled as-built and as-planned spaces are fed into a Bayesian
updated project schedule is used to generate the underlying 4D model and used to assess progress through a SVM classifier.
model. For the as-built model, it is further assumed that all the pho- Finally, the detected as-built elements, camera parameters, and 4D
tographs are collected on one particular day or within a short period BIM are fed into the D4 AR viewer to visualize the as-built and
of time (e.g., a couple of days) where no significant progress is as-planned models and progress deviations in an integrated fashion.
made in construction. In the proposed approach there is no need In the following sections, the SfM and other steps designed for
to infer temporal order from images. Rather, such information is progress tracking are presented.
Fig. 2. Overview of data and processes in proposed tracking, analysis, and visualization system
J. Comput. Civ. Eng., 2015, 29(1): 04014025

Reconstructing an Underlying As-Built the same camera viewpoint, and Fig. 3(e) shows the image textured
Representation using Structure-from-Motion on the camera viewing plane.
Recently, Golparvar-Fard et al. (2009a) sparsely reconstructed

and visualized an as-built scene from unordered daily photographs. Aligning the As-Built and As-Planned Models
The work was based on a SfM technique similar to that of Snavely
et al. (2008) to automatically reconstruct an as-built point cloud To align the as-built point cloud model with the as-planned model,
model from a set of images (no manual intervention at any stage). transformation between these two Cartesian coordinate systems
This module consists of the following steps: (1) analyze images and needs to be found. For alignment of an as-built point cloud model
extract scale invariant feature transform (SIFT) feature points that is reconstructed from photos collected at a time (t), an as-
(Lowe 2004) from images; (2) match image features across the im- planned model that shows progress up to time (t0 ) (t0 ≤ t) is used.
age set (Hartley and Zisserman 2004); (3) find an initial solution for The alignment transformation can be formed as a rigid-body mo-
the 3D locations of these features points, calibrate the cameras for tion and hence can be decomposed into rotation and translation.
an initial image pair, and reconstruct the rest of the observed scene; However, in SfM, the scale may be unknown. In addition, the point
in addition, estimate the motion of the cameras based on a bundle cloud model produces a significantly large number of points that
adjustment algorithm (Nistér 2004; Triggs et al. 1999); finally, do not belong to the building model itself (e.g., generated from the
(4) register the point cloud models that are generated for each façade of surrounding buildings, machinery, or even people and
day to build a 4D as-built model. To show how these steps are per- plants on or around the site). Further, the vertices extracted from
formed, two sets of 112 and 160 images are chosen that were taken the as-planned model are also very sparse and thus may not yield
on Aug. 20 and Aug. 27, 2008 during construction of the Ikenberry a good representation of the expected progress. Therefore, in the
residence hall (RH) in Champaign, IL. In both cases, the field en- proposed approach, users are allowed to select a set of correspond-
gineer casually walked along the sidewalk of the project and took ing control points from the as-built point cloud or the registered
images within a span of a few minutes. Figs. 3(a and b) represent imagery and have those associated with the as-planned model.
the sparsely reconstructed scene from the same image subset and These points could be surveying control points or a set of points
show five registered cameras in the D4 AR environment. Once a that represent the geospatial location of the site. In the case studies
camera is visited, the camera frustum is texture-mapped with a full presented in this paper, these points are mostly chosen from corners
resolution of the image so users can interactively zoom in and vis- of the foundation walls and columns because their detection and
ually acquire information on progress, quality, safety, and produc- correspondence is visually easier.
tivity, as well as workspace logistics. Figs. 3(a and b) visualize the The unknown uniform scaling adds one more degree of freedom
as-built point cloud model from synthetic views, Fig. 3(c) shows (DOF) to the original transformation problem (overall 7 DOF).
the location of a camera frustum, Fig. 3(d) shows the site through Therefore, three points known in both coordinate systems will
Fig. 3. (a) Synthetic bird’s-eye view of as-built point cloud model reconstructed; (b) five camera rendered frusta representing location/orientation
of superintendent when site photographs were taken; (c) one camera frustum is rendered and its location/orientation is visualized; (d) as-built point
cloud observed through camera frustum [same camera as in panel (c)]; (e) camera frustum textured visualizing photograph registered over 3D
point cloud
J. Comput. Civ. Eng., 2015, 29(1): 04014025

be theoretically sufficient to permit determination of these seven focal length is accurately calculated through the bundle adjustment
unknowns. However, in practice, these measurements are not exact, stage [readers are encouraged to consult Golparvar-Fard et al.
and if more than three points are used, greater accuracy can be (2011, 2009a) for more details]. In the proposed approach and the
sought. By adding additional points, instead of directly solving developed system, the user only needs to initially register the as-
the transformation that exactly maps the measured coordinates planned and as-built models. Thenceforth, any new point cloud
of points from one system to the other, the sum of squares of model only needs to be registered with the underlying point cloud
residual errors is minimized. Let there be n points from as-planned models. To automatically register several point clouds, an iterative
and as-built models for registration. The points in these coordinate closest point (ICP) algorithm (Besl and McKay 1992) that can also
systems can be denoted by {rb;i } and {rp;i }, respectively, where i solve for scale (Du et al. 2007) is developed. For initialization of the
is the number of corresponding points, which ranges from 1 to n, ICP algorithm, SIFT feature keypoints from the previous as-built
rb;i and rp;i are the Cartesian coordinates of matched as-built and reconstruction are used.
as-planned model i, with i ∈ ½1; n. The transformation has the This method automatically finds a set of SIFT points from each
following form [Eq. (1)]: point cloud model and automatically aligns the new point cloud
model to the former one, in turn having the new point cloud model
rp ¼ sRðrb Þ þ T ð1Þ
registered with the as-planned model. This generates 4D as-built
point cloud models wherein users can navigate the as-built scene
where s = uniform scale factor; T = translational offset; and Rðrb Þ =
both spatially and chronologically. The 4D as-built point cloud
rotated version of planned model. Minimization of the sum of the
squared errors is formulated as models registered with the 4D as-planned models enable the
expected and the actual project schedules to be compared as well.
X
n X
n Fig. 4 shows eight snapshots from the two case studies: RH (res-
kei k2 ¼ krp;i − sRðrb;i Þ − Tk2 ð2Þ idence hall) and SD (student dining) construction projects. In each
1 1 row, two separately reconstructed point cloud models are shown,
while in the third image, the two point cloud models are registered
To solve for this transformation, Horn’s (1987) method is used,
and simultaneously visualized. Finally, in Figs. 4(d and h), the
which gives a closed-form solution to the least-squares problem of
registration of an IFC-based BIM with a point cloud model in
absolute orientation. The error (Δe) can be measured in millimeters
Figs. 4(b and e) is visualized. In Figs. 4(a and b), the 3D point
using the following formula [Eq. (3)]:
clouds are reconstructed using 112 and 160 photographs collected
Δemm ¼ w̄CCD;mm × Δepixels =w̄pixels ð3Þ from outside of the RH basement along the sidewalk, and in
Figs. 4(e and f) the 3D point clouds are reconstructed using 288
where Δepixels = error in pixels; w̄pixels = image width in pixels; and and 118 photographs collected from inside and around the SD base-
w̄CCD;mm = charge-coupled device (CCD) width of the camera in ment. Tables 1 and 2 report high accuracies for both point cloud/
millimeters. Similar to Golparvar-Fard et al. (2009a), images used point cloud and point cloud/BIM registrations. In our approach, the
in the as-built reconstruction module do not necessarily need to be registration accuracy is insensitive to how the control points are
shot with the same camera. Rather, for each image (or camera), the selected. Since usually more than the minimum number of control
focal length and the CCD width are initialized for the SfM process points (three) are selected, such interactive selection errors are
through extraction of the EXIF tag of JPEG images. Later on, the minimized.
Fig. 4. Point cloud/point cloud and point cloud/BIM registrations: (a) point cloud model reconstructed from 112 images from RH project (Aug. 20,
2008); (b) point cloud model reconstructed from 160 images from RH project (Aug. 27, 2008); (c) violet point cloud is (a) and orange point cloud
is (b); (d) registration of BIM with point cloud in (b); (e) point cloud reconstructed from 288 images from SD project (Jul. 7, 2008); (f) point cloud
model reconstructed from 118 images from SD project (Jul. 24, 2008); (g) the dark gray point cloud is (e) and the light gray point cloud is (f);
(h) registration of BIM with point cloud in (e)
J. Comput. Civ. Eng., 2015, 29(1): 04014025

Table 1. Registration Errors Measured on Reconstructions Shown in Fig. 4 for RH Project
Case study Test case number BIM + point cloud (4-a) BIM + point cloud (4-b) Point clouds (a) and (b)
RH project: Image size 2,144 × 1,424 1,715 × 1,139 —
RH 2 and 3 Number of feature points 62,323 43,400 —
Number of corresponding points 7 7 Randomly chosen by ICP
Δemm (mm) 0.20 0.65 0.43
Notes: BIM = building information model; RH = residence hall.
Table 2. Registration Errors Measured on Reconstructions Shown in Fig. 4 for SD Project

Case study Test case number BIM + point cloud (4-e) BIM + point cloud (4-f) Point clouds (e) and (f)
2,144 × 1,424 2,573 × 1,709 —
SD project: Image size

SD 1 and 2 Number of feature points 61,638 31,661 —
Number of corresponding points 9 9 Randomly chosen by ICP
Δemm (mm) 0.73 0.69 0.70
Notes: SD = student dining; BIM = building information model.
Automated Progress Monitoring Problem Setup As a result of this constraint, voxels are traversed in an increasing
and Notation distance from the set of cameras. First, the voxels in the layer
immediately adjacent to the camera are visited; then all voxels that
To detect progress, the integrated as-built and as-planned scene (Ω) are in the next layer immediately adjacent to the first layer are
is discretized into a finite set of opaque voxels (volume element in visited. Using this approach, when a voxel is visited, all other
space) along the dominant Euclidean axes. This discretization is in voxels that can occlude the current one from the set of cameras
the form nx δ x × ny δy × nz δz , where each voxel (ν) occupies a finite are already visited. This strategy proactively considers static and
homogenous volume of the scene (δ x δ y δ z ) and has a consistent dynamic occlusions from the set of cameras and labels and colors
visual appearance. This approach enables and supports reasoning each voxel based on a consistent visibility from projections of a
progress in small volumes of space. In the proposed model, voxels voxel on all images.
are assumed to be equilateral; therefore, the resolution of the voxel Before traversing the scene, the integrated as-built and as-
grid is determined by δ. Given an image Πi , proji ðνÞ is used to planned scene is transformed into a new coordinate system wherein
denote the reprojection of the voxel over the image i. The eight the axes are aligned with the dominant axes of the as-planned site.
corners defining each voxel are projected into the image 2D plane. This will minimize the search space since reasoning for expected
Next, the bounding values for each axis (u, v) are kept to form the progress only needs to be done in areas in which progress is
2D reprojection bounding box. The following formulas are used to expected to be observed in the site coordinate system. To consider
represent this transformation: ahead-of-schedule activities, this area will only contain those BIM
2 3 components that are expected to be observed from the detailed
2 3 x
3-week look-ahead schedule, i.e., (1) the work breakdown structure
u 6 7
6 7 6y7 is more detailed and (2) the elements that are not yet constructed
∀ k ∈ f1; 2; : : : ; 8g → 4 v 5 ¼ K i ½Ri jT i 6
6z7
7 ð4Þ
4 5 have a smaller chance of self-occluding the as-planned model.
1 k Thus, the scene is traversed from the closest voxel to the convex
1 k hull of the cameras (rough approximation of the scene boundaries)
in a 2D plane normal to the convex hull and eventually in a front-to-
proji ðνÞ ¼ ½minðuk ; vk Þ:: maxðuk ; vk Þ ð5Þ
back order (see Fig. 5: Axis 1 to 3 directions). In the case where the
where k = index of the voxel corners, and K i , Ri , and T i = camera goes all around the building, the voxels are analyzed at an
intrinsic camera parameters, rotation, and translation of camera i increasing distance from the set of cameras. All the voxels that are
in the scene. in the layer immediately adjacent to the camera convex hull are
visited. Subsequently, all the voxels in the next layer immediately
adjacent to the first layer are analyzed. This process is repeated
Voxel Traversing and Labeling until all voxels are visited. Similarly, in the case where the cameras
are all looking outward, first the voxels that are in the layer immedi-
The next step is to traverse the integrated as-built and as-planned ately adjacent to the camera convex hull are visited. Subsequently,
scene. In this process, each voxel is assigned two sets of labels voxels in the succeeding layers adjacent to the first layer are ana-
(as-built and as-planned) as well as a color. Within this step, the ex- lyzed layer by layer. As the algorithm marches through the voxels,
pected and actual progress of each voxel is sensed. It is critical to the visibility constraint is verified. The labeling process is as fol-
traverse the voxels in a certain order; otherwise, the reconstruction lows. For every voxel (υi;j;k ) in the scene, two sets of labels lðυi;j;k Þ
results will not be unique. To address this issue, an ordinal visibility are defined: (1) as-built and (2) as-planned labels. For each image,
constraint similar to that of Seitz and Dyer (1999) is introduced, a marking board is also generated wherein each pixel is initially
allowing certain invariant voxels to be found whose colorings are marked by a 0, and when a pixel satisfies the consistent visibility
uniquely defined. Rather than only using this constraint to address constraint, the pixel label is changed to 1.
the uniqueness of the solution, the proposed approach finds the As-built labeling: for the as-built model, the algorithm first
voxels that are occupied by as-built and as-planned components checks whether a voxel already contains reconstructed SIFT or
and are visible from the set of cameras (i.e., observable progress). MVS points. In this case, that voxel is labeled Occupied (Ob )
J. Comput. Civ. Eng., 2015, 29(1): 04014025

ðn − 1ÞSD2
λv ¼ ≤ thresh ð7Þ
σ20
where SD = standard deviation of color values; σ20 = accuracy

of irradiance measurement (sensor color measurement error); and
n = number of images that observe the voxel. If λv is less than a
maximum allowable correlation error (thresh), then that voxel is
labeled visually consistent (Ob ); then the labeled voxel is projected
on all images that observe it and the pixels associated with the voxel
on the marking board are labeled accordingly. In the conducted
experiments there is a minimum allowable number of reprojected
pixels for each voxel from all images that have a null value
(n > 20 pixels). If the consistency control on the pixel radiances
is not satisfied, the voxel is labeled Empty (Eb ); finally, if the

condition on the minimum allowable number of pixels is not
satisfied, i.e., the voxel is occluded from all views, then the voxel
Fig. 5. Representation of as-built site and camera configurations; is denoted as Blocked (Bb ). In the experiments conducted for
reprojections of voxel are shown on camera Frusta 1 and 2; marking this paper, the thresh was set to 1. This threshold is computed
for Camera 1 also shown on left-hand side; in this case, the voxel is through a tradeoff on completeness versus accuracy for as-built
detected as occupied; therefore, all pixels belonging to the reprojection reconstruction. The strategy of marking voxel projections by gen-
of the voxel on all images are marked 1 erating a marking board for each image accounts for both complete
and partial occlusions.
This process has three significant outputs: (1) labeling all
voxels in the as-built model as [Ob jEb jBb ], enabling reasoning
and is subsequently projected on all images that observe that voxel about the presence of full and partial occlusions (both static
[Eqs. (4) and (5)], and if the reprojection (i.e., the boundaries of the and dynamic); (2) creating a dense point cloud reconstruction
voxel projection) falls within the image boundaries, then the pixels through joint visualization of the sparse point cloud and the cent-
inside the projected voxel will all be marked 1 [Eq. (6)]: roid of all occupied voxels; and (3) creating as-built range images
based on observations. Fig. 6(a) shows a plan view of voxel la-
−w=2 w=2 beling, whereas in 6(b) the reprojected voxel shown in Fig. 6(a)
∀ i ∈ fC1 ; : : : ; Ck g; ifprojðvk Þ ∈ −
−h=2 h=2 is marked on the image as proj1 ðυÞ. In Fig. 6(c), the unchanged
i
versus progress observation concept is visualized. Fig. 7 sum-
→ ∀ m; n ∈ proji ðvk Þ; Marki ðm; nÞ ¼ 1 ð6Þ marizes the as-built occupancy/visibility labeling and marking
algorithm.
If a voxel does not contain reconstructed SIFT or MVS points As-planned labeling: the as-planned model by itself accounts
(which is more often the case), the visual consistency is controlled. for static occlusions. Nonetheless, to consider dynamic occlusions,
In the absence of noise or quantization effects, an occupied as-built the nonoverlapping areas of the as-built scene (e.g., reconstruction
voxel that is at least partly visible from an image should have an of excavators, temporary structures) are placed over the corre-
equal radiance on all projected pixels. In the presence of these sponding as-planned voxels. The as-planned scene is traversed
effects, the correlation of pixel radiances is evaluated to quantify in a similar fashion to that of the as-built. Hence, if a voxel has at
voxel consistency: least one of its corners inside an IFC element, that voxel is labeled
Fig. 6. (a) Plan view of discretization of scene to voxels along dominant axes; each voxel with respect to shown camera configuration is either
occupied (Op), blocked (Bb), or empty (Eb); (b) Image 1 (Π1) from camera configuration in (a), where projl ðυÞ shows projection of voxel (υ) from
(a) over Π1, which is marked (color coded differently from unmarked voxel reprojections); (c) progress versus unchanged observations
J. Comput. Civ. Eng., 2015, 29(1): 04014025

Fig. 7. As-built voxel labeling and image marking; if a voxel contains

at least one feature point or has a consistent visual appearance, it will be
labeled as occupied (20 is the minimum number of unmarked pixels Fig. 8. As-planned voxel labeling and image marking; if a voxel is
expected per projection of each voxel) filled by an IFC element, it will be labeled occupied; if it is observable
from at least one camera, it is marked visible (20 is the minimum num-
ber of unmarked pixels expected per projection of each voxel)
Occupied [Op ]. Subsequently, the voxel is projected on all images

that observe it and the corresponding pixels on a similarly gener- can be formulated. In the proposed progress-detection model, prog-
ated as-planned marking-board are marked with a 1. In this case, ress (observation per expected as-planned element i) is formulated
the depth value of each voxel is kept as another marking layer as a binary value ðEi Þ∶Ei ¼ 1 if progress is detected and Ei ¼ 0 if
for the image. Based on the location and depth of all IFC elements, not. First, for the purpose of generating a proper baseline for prog-
this new strategy automatically generates as-planned depth maps ress monitoring, and given the underlying level of detail in the
and segments images into visible IFC elements. In the case of non- 4D BIM, the underlying IFC-based 4D BIM is broken down into
overlapping as-planned and as-built areas, the consistency from the several independent elements.
as-built marking layer is controlled. Those voxels that pass the con- For example, in the case of FPRS basement concrete columns
sistent visibility check are projected on all images and their corre- activity, the progress-detection algorithms need to check the
sponding pixels are marked. This further considers occlusions since observation of all expected columns attached to this activity (all
if a voxel projection contains a minimum number of unmarked Elements i as Ei ). Assume that each element Ei attached to this
pixels, then the voxel can be labeled as Visible [V p ]. As mentioned activity consists of n voxels. A set of probability events is intro-
previously, in the proposed model, all labels are independent of one duced: within a given volume in the scene (ωi ), let η be the event
another and marked with binary values (either 1 or 0). In addition to of observing an as-built element (any tangible physical element), θP
labeling voxels, image pixels are also marked, so that if a pixel is the event of observing an as-planned element, and θT the event that
an as-planned element is expected to be observed from the 4D as-
observed, the pixel is labeled with a 1, and if it is not observed,
planned model. The probability of observing progress for element
it remains marked as 0 (see left side of Fig. 5). Such labeling en-
Ei associated with a given schedule activity (duration ¼ d days) is
ables reliable and consistent reasoning about progress in full and
defined as a conditional probability of the form
partial visible areas. Fig. 8 summarizes the as-planned occupancy/
visibility labeling and marking algorithm.
PðθiT jηi ÞPðηi Þ
Pðηi jθiT Þ ¼ ð8Þ
PðθiT Þ
Probabilistic Model for Progress Detection and
Discriminative Learning where PðθiT jηi Þ = probability of observing the expected as-planned
element given some evidence of occupancy; Pðηi Þ = probability of
Once partial visibility and occlusion, as well as the as-built and as- observing the as-built element (a function of confidence in coloring
planned occupancies, are labeled across a scene, progress detection and labeling of the voxel, as well as those occupied voxels that
J. Comput. Civ. Eng., 2015, 29(1): 04014025

belong to the as-built element—the current system assumes this is
equal to 1); and PðθiT Þ = probability of observing the expected
progress for element i. For each element under an ideal situation,
where there is no occlusion and no partial visibility, each of these
probabilistic values can be directly measured; however, since oc-
clusions predominate, these probabilities need to be measured
within the expected volume of observable progress. In this case,
each of these probabilities can be computed as follows:
P
Ob
For the as–built∶PðθiT jηi Þ ¼ P P ð9Þ
Ob þ Eb θi
P
P
Vp
For the as–planned∶ PðθiP Þ ¼ P ð10Þ

O p θi
P
Fig. 9. Tracking physical progress for activity in work schedule

t
PðθiT Þ ¼ V ð11Þ
d
where PðθiP Þ = expectation of observable progress for element i using a linear SVM (Fan et al. 2008; Chang and Lin 2001). Once
(percentage of visibility from camera set); d = total duration of the classifier is learned, given a new observation [that is, a meas-
construction activity; t ¼ t th day within this duration (d); and urement of progress Pðηi jθiT Þ] along with the measured features
V = volume of expected as-built element. In other words, Eq. (9) [θip (t), t=d, T i , Ψðt), δ, and thresh], the progress binary value can
represents the percentage of voxels occupying the expected visible be determined by feeding the observation into the classifier and
voxels for element i; Eq. (10) measures the percentage of those retaining the output.
voxels that are expected to be occupied and are visible; finally, The progress deviations at the individual element level can be
Eq. (11) represents the volumetric percentage of progress for those expanded to represent deviations at the schedule activity level.
cases in which monitoring is conducted during the time the element Progress for a given schedule activity that is linked to n mutually
is expected to be completed and is not finished. independent elements in the IFC-based BIM can be formulated as
To classify progress Ei and under the condition of partial vis-
ibility of a given as-planned element, Pðηi jθiT Þ needs to be com- Q ¼ Pf½ηi ; i ¼ 1 : : : njθip ; i ¼ 1 : : : ng ð12bÞ
pared with a threshold Γi . Choosing an optimal value for the
threshold for each element is problematic. For example, given a where Pf½ηi ; i ¼ 1:::njθip ; i ¼ 1:::ng is the probability of observ-
10% probability for observing progress PðθiP Þ and 25% for observing progress for a schedule activity, given its mutually independent
ing as-planned elements given evidence of occupancy PðθiT jηi Þ, sequence conditions (e.g., construction of column–slab; column–
measurement of Pðηi jθiT Þ may be susceptible to noise and inaccur- column and column–wall are considered mutually independent).
acy in reconstruction. Therefore, such a measurement might not be In this case, progress is formulated as
reported as detected progress. This selection of threshold is particu- P i
nE × Vp
i
larly difficult because (1) to achieve a desired accuracy for different Q¼ P i ð13Þ
element types with different materials, different thresholds need to n Vp
be used; (2) a progress monitoring task with partial visibility is sub-
jective by nature and needs an expert’s opinion as to whether or not where V ip is the volume that is expected to be observed and occu-
it has taken place. Thus a machine learning methodology is used to pied for each element associated with the construction schedule
estimate such dynamic thresholds in a principled way. The thresh- activity. Fig. 9 summarizes the progress-detection process for each
old (Γi ) can be expressed as construction schedule activity.
Γi ¼ fðθip ðtÞ; pðηi jθiT Þ; t=d; T i ; ΨðtÞ; δ; thresh; εReg ; εRec Þ ð12aÞ
Experiments and Results
where t = construction activity duration from t ¼ 0 to d; T i =
element type (e.g., column, beam, foundation); ΨðtÞ = visual ap- To verify the robustness of the proposed reconstruction pipeline
pearance of element i (e.g., concrete, formwork, steel); δ = voxel and validate the automated progress-detection module, three dif-
resolution; thresh = voxel consistency threshold; and εReg and ferent experiments are conducted. The image data sets for these
εRec = accuracy in registration of as-planned model over point experiments were collected under different viewpoint and lighting
cloud model and the accuracy of underlying reconstruction pipe- conditions. These data sets are two photo collections of 112 and
line, respectively. For the sake of simplicity at this stage, as shown 160 images from the RH project and a 288-image data set from
in Tables 1 and 2, it is assumed that there are minimal errors in the SD project. In both RH project data sets, a significant amount
(1) as-built and as-planned registration and (2) the underlying mod- of occlusion is observed since the images were not taken from in-
ule for as-built reconstruction. The threshold Γi can be learned by side the basement area. Rather, they were all taken along a sidewalk
posing the problem as a linear classification problem, that is, of the project [see locations of the camera frusta in Fig. 3(b)]. The
by learning the hyperplane that separates the two classes in a spatial resolutions of these images were synthetically reduced to
multidimensional feature space. The feature space is defined by approximately 2 Mpixels to test the robustness of the proposed ap-
Pðηi jθiT Þ, θp ðtÞ, t=d, T i , ΨðtÞ, δ, and thresh. The two classes are proach to the quality of images. The voxel resolution was initially
progress = 1 and no-progress = 0. The optimal hyperplane that set to 1=5 ft (approximately 0.06 m). The IFC-based 4D BIMs
separates the two classes can be learned in a supervised fashion for RH and SD projects have relevant schedule activities that are
J. Comput. Civ. Eng., 2015, 29(1): 04014025

Fig. 10. (a–d) Dense as-built reconstruction for RH data set presented in Fig. 4(b)
connected to 152 and 321 elements, respectively (Fig. 4 for segmented and visualized. This robustly takes occlusions into
the relevant part of the RH project schedule). Figs. 10(a–d) and account because all the elements that are located closer in the
Figs. 11(a–d) illustrate the results of dense reconstruction for the line of sight to the camera will be detected first (ordinal visibility
case presented in Fig. 4(b) (RH 160) as well as the SD project. constraint introduced in this research). This will further enable a
All the snapshots in this case are taken from synthetic views in a texture-recognition algorithm to be created to detect Pðηi Þ and
3D virtual environment (none of these views exists in an image data account for progress details accordingly. For example, consider a
set; each is a result of synthetic 3D visualization). concrete foundation wall that is to be further prime-coated and in-
Fig. 12 illustrates the distinct contribution of the MVS as well sulated. Since the system is based on an IFC as-planned platform
as voxel coloring/labeling algorithms on the density of the as- and is linked to the schedule, expected progress information can be
built point cloud models. Compared to MVS, the effect of voxel queried from the 4D BIM and, given the time the image is captured
coloring/labeling on increasing the density of the point cloud is (extracted from the EXIF tag of the JPG image), the visual appear-
marginal (approximately 5–20%), though it enables labeling of the ance of the surface will be known.
scene and all voxels both for expected and actual occupancy and
visibility.
Fig. 13 shows the results of traversing, labeling, and reproject- Discussion on Automated Detection Accuracy
ing detected areas of as-built and as-planned environments. For
the same image plane shown in Fig. 13(a), range images for both In the conducted experiments, the performance of the progress-
as-planned and as-built environments are generated. Based on the detection model is analyzed using a number of common object-
depth from the camera plane, Fig. 13(b) illustrates the reprojection recognition metrics. In particular, the following metrics are used:
of voxels occupied by the IFC elements. To visualize the depth, (1) recall: the fraction of truly recognized IFC-model elements
a color-coding scheme is represented where depth is visualized (TP = true positive) relevant to the total number of model elements
in relationship to the furthest elements from the camera plane that are used for the detection model (TP + FN = true positive +
(in this case, the rear foundation wall). In Fig. 13(c), the consis- false negative). This parameter will show the sensitivity of the
tently observed as-built voxels are reprojected back. Combining detection model; (2) precision: the fraction of relevant IFC-model
Figs. 13(b and c) allows specific areas within each image where elements relevant to the total number of model elements that
IFC elements are supposed to be observed to be automatically are recognized (TP + FP = true positive + false positive). In the
Fig. 11. (a–d) Dense reconstruction of SD data set
J. Comput. Civ. Eng., 2015, 29(1): 04014025

Fig. 12. (a and c) Sparse structure-from-motion point cloud models; (b and d) dense point cloud that is the result of structure-from-motion, multiview
stereo, and voxel coloring/labeling
proposed approach, the SVM kernel machine classifies progress the accuracy of the SVM linear classifier. The performance of the
with a binary value (progress/no progress). classifier is further tested and validated on RH 160 and SD 288
In the experiments presented here, the SVM model is trained image data sets. The results of average accuracy for the experimen-
over the RH 112 image data set, and the hyperplane that separates tal data sets are presented in Table 4.
the progress/no-progress categories is automatically learned. The The correlation between the progress that is expected to be
results from the experiments show that if visibility (the observable detected [PðθiT jηi Þ] and the expected observable regions [Pðθip Þ]
part) of an as-planned element is less than 20% and the volumetric is also studied. Fig. 14(a) shows the results of the experiment on
reconstruction is only able to reconstruct 50% of that observable the RH 112 training data set. As the figure indicates, the majority of
part, progress will not be detected. The performance of the training false detections happen for less than 20% of the observable prog-
is cross checked by asking two field engineers and a superintendent ress Pðθip Þ. This further illustrates that in the presence of severe
to label the classification results. The accuracy of training was occlusion and poor reconstruction, no decision on progress should
experienced to be 87.50%. Table 3 shows an example of how be made. To further investigate the sensitivity of the detection
the SVM classifies the state of progress for two classes of concrete model to the presence of occlusions, the relationship between the
columns and foundation walls. In this example, the detection fea- accuracy to the percentage of visibility is studied. As observed in
ture vector values are shown. In the proposed approach, as more Fig. 14(b), there is no linear relationship between the percentage
experiments are conducted, the outcomes can be added to increase of occlusion and the accuracy of automated progress detection.
Fig. 13. (a) Image taken on RH project dated Aug. 27, 2008; (b) range image generated for expected IFC elements; grayscale shows ratio of depth
along camera line of sight compared to rear foundation wall; (c) expected as-built progress voxels detected and projected back on image plane
J. Comput. Civ. Eng., 2015, 29(1): 04014025

Table 3. Supervised SVM Learning of Detection Threshold for T i = Rather, the relationship between observed PðθiT jηi Þ and the ex-
(i ¼ 0 Column; i ¼ 1 Wall) and ΨðtÞ = Concrete pected observable Pðθip Þ controls how the progress and no-progress
Γ θp ðtÞ pðηjθÞT t=d T δ th categories are classified. Based on several experiments with dif-
ferent levels of occlusion, the standard deviations of the detection
−1 0.16 0.16 1.00 0 0.20 1
−1 0.24 0.84 1.00 1 0.20 1 accuracies are calculated and visualized in this figure.
þ1 0.32 0.75 1.00 0 0.20 1 To examine the precision and sensitivity of the detection model,
þ1 0.35 0.84 1.00 1 0.20 1 precision-recall and true-positive/false-positive graphs are further
−1 0.36 0.24 1.00 0 0.20 1 analyzed. Fig. 15 illustrates the results over the experimental data
þ1 0.36 0.71 1.00 1 0.20 1 sets. These graphs are only drawn for the elements that were
þ1 0.37 0.80 1.00 0 0.20 1 expected to be detected and not for those elements that are fully
þ1 0.41 0.79 1.00 1 0.20 1 occluded. Despite the presence of a a significant amount of occlu-
−1 0.43 0.21 1.00 0 0.20 1
sion in the training data set, the results seems promising. These
þ1 0.46 0.89 1.00 1 0.20 1
þ1 0.49 0.88 1.00 0 0.20 1
results show that the proposed method is insensitive to the forma-
þ1 0.51 0.85 1.00 1 0.20 1 tion of the SVM hyperplane.

−1 0.52 0.25 1.00 0 0.20 1 Finally, to represent progress and color-coded changed and
−1 0.57 0.43 1.00 1 0.20 1 unchanged elements with red and green, the same D4 AR modeling
þ1 0.63 0.75 1.00 0 0.20 1 platform as reported in Golparvar-Fard et al. (2009a) and the color-
þ1 0.71 0.89 1.00 1 0.20 1 coding scheme as presented in Golparvar-Fard et al. (2009b) are
used. Fig. 16 shows the results of visualizing the outcome of the
progress detection model. In these cases, the behind-schedule or
Table 4. Average Accuracy of SVM Binary Detection for Training and
on-schedule IFC elements are color-coded with red and green,
Testing Data Sets respectively. For those elements for which progress is not reported,
the color remains gray. Such a color-coding scheme makes it easy
Number of Number of Detection to observe the accuracy of progress detection while, if needed,
Experiment Data set images IFC elements accuracy (%)
allowing corrections to be made on a case-by-case basis.
Training RH 2 112 152 87.50 Fig. 17 illustrates a few examples of the detection’s false pos-
Testing RH 1 160 152 82.89 itives and missed positives (false negative in proper statistical
Testing SD 288 321 91.05
terms). As observed, since the detection model does not contain
Fig. 14. (a) Ratio of expected progress, PðθTijηiÞ, to expected observable regions, PðθpiÞ, for progress-detection results from RH 1 experiment;
(b) ratio of accuracy of detection to percentage of visibility (1 – occlusion)
Fig. 15. (a) Precision-recall graph; (b) true positive/false positive graph for progress-detection model
J. Comput. Civ. Eng., 2015, 29(1): 04014025

Fig. 16. (a) Visualized progress for RH project over D4AR D4 AR environment; (b) semitransparent view of RH progress from camera viewpoint;
(c) RH progress detection results over IFC-based BIM; (d) visualized progress for SD project over D4 AR environment; (e) semitransparent view of
SD progress from camera viewpoint; (f) SD progress detection results over IFC-based BIM
Fig. 17. (a and b) False positive: formwork should not be detected as evidence of progress; (c and d) missed positive (false negative): wall should be
detected for progress, though it is severely occluded
operational details (e.g., forming stages), the formwork is falsely Contribution

detected as a finished concrete element. In Fig. 17(c), the high-
lighted wall should have been detected, but, due to occlusions, The first contribution of this paper is a significantly improved
the element is not properly reconstructed and consequently not algorithm for dense reconstruction and robust registration for 4D
detected. as-built point cloud models from daily site photo collections. Com-
According to Eq. (13) and based on the detected progress per pared to Golparvar-Fard et al. (2009a), a more robust vision-based
element Pðηi jθiT Þ, progress can be reported at the schedule-activity method comprised of SfM, MVS, and voxel coloring algorithms is
level. Fig. 18 presents a part of the RH project schedule and illus- presented and used for reconstruction. Furthermore, registration is
trates what activities are tracked for progress. Since some elements enhanced allowing 4D as-built point cloud models to be automati-
may not be visible at all, a metric for visibility per schedule activ- cally generated through superimposition of various point cloud
ity is also reported, which gives an indication of reliability of prog- models, in turn semiautomatically registering them over a 4D IFC-
ress values. As observed, given the accuracy of the detection based BIM. The superimposition of the BIM over the point cloud
engine at this stage, progress can still be reported at the schedule- requires the user to select matching feature points, and thus this
activity level. Since the exact timing of each operational stage step only needs to be done once at the beginning of a project. The
(e.g., forming/pouring) is not known, progress cannot be reported observations and conducted experiments show that the resulting
at any finer level of detail. Yet with Eq. (8) it can be expected that D4 AR visualization has the following capabilities:
when the material/texture recognition is incorporated and Pðηi Þ is • Data collection: the approach is fully dependent on daily photo
measured, without a need for a detailed WBS, progress information collections and has no costs or need for any manual intervention
can be reported at finer levels of detail [in all of the experiments beyond uploading images to the system and works even with
conducted for this paper, Pðηi Þ ¼ 1]. low-quality images taken from a cell phone. Ideally, a large
J. Comput. Civ. Eng., 2015, 29(1): 04014025

Fig. 18. Progress reported on RH construction schedule (progress monitoring date: Aug. 27, 2008)
number of images need to be captured to guarantee successful (e.g., differentiation of concrete from formwork), the proposed
automated progress tracking. Bayesian model explicitly accounts for this, which in turn facil-
• As-built modeling automation and visualization: this process is itates the extension of the proposed algorithms.
fully automated, i.e., once images are deposited in the system, • Progress monitoring accuracy: the metrics shown in the experi-
features are fully automatically identified and matched to visua- ments seem to be satisfactory given the formation of this
lize the underlying as-built point cloud model. The camera progress-tracking model for the first time. More importantly,
configurations are automatically identified as well. the visibility metric represented at the schedule-activity level
• Occlusion handling: since unordered daily photographs are highlights those activities and elements that need to be revisited.
usually taken with the smallest amount of occlusions, their Based on this confidence factor, users can either take more
application is very desirable for automated as-built modeling. images for an accurate assessment of progress or manually
The underlying SfM automatically removes noise and other revisit those elements.
inconsistent representations, so there is no need for postproces- • Robustness to occlusions: compared to Bosché (2010) and
sing of the point cloud model (as is the case in laser-scanning Zhang et al. (2009), in the proposed statistical model for auto-
point cloud models). In addition, the contextual semantic information of monitoring, the threshold for detection of different
mation associated with moving objects in the scene are not fully types of elements under partial visibility is dynamically learned.
Yet the metrics for comparison of different automated monitor-
removed from the point cloud models; rather, they are dynami-
ing models need to be set using similar construction case
cally captured in registered images.
studies.
• As-built processing efficiency: each point cloud model can be
• Computational efficiency: the progress deviations are cur-
generated in a few hours (computational cost at this stage). Once
rently computed over a few hours. Since the frequency of
the underlying model is generated, adding new photographs to
progress monitoring in most cases is not expected to be more
the system is processed in a matter of seconds.
than one observation per day, this computation time can be
• Augmented reality registration: registration of the 4D IFC-based
acceptable.
BIM is still semiautomated because it requires a set of initial
control points to be manually selected for automated matching.
This needs to be done only once in the initial stages of the Conclusions and Summary
project. Registration of point cloud models over one another is
done automatically using iterative closest point (ICP) algorithms An automated approach to tracking, analysis, and visualization of
by selecting a subset of points that has a consistent visual ap- progress using daily site photographs and 4D IFC-based BIMs is
pearance in point cloud models (e.g., an existing structure that is presented. In the proposed approach, images can be of low quality
reconstructed in consecutive point cloud models). yet robustly generate dense as-built point cloud models. Sub-
The foremost contribution of this paper is the automated prog- sequently, the underlying point cloud model is registered with other
ress monitoring model and the SVM machine learning approach. point cloud models as well as the as-planned model, generating
In the presence of a large number of photographs, it is shown that an integrated 4D as-built and as-planned model for progress visu-
the automated progress monitoring module can result in high pre- alization. The as-built and as-planned voxel coloring and labeling
cision. The approach generates range images for each photograph algorithm demonstrates high accuracy in the labeling of a construc-
and segments each image based on observed progress and dynamic tion scene for occupancy and visibility. The SVM kernel machine
occlusions (a robust 2D segmentation of observed objects over site shows promising results in detecting progress. Overall, the pre-
images). It is shown through multiple experiments that the pro- sented results mark the approach presented in this paper as the first
posed automated detection has the following characteristics: of its kind to take full advantage of already available daily site
• Progress monitoring automation: except for the initial registra- photographs and IFC-based 4D BIMs for automated progress
tion step of the BIM and point cloud models, the monitoring of tracking and analysis. Application of the D4 AR modeling system
physical progress is fully automated. Although the operational is perceived to achieve minimization of the time required for
progress details cannot currently be automatically identified as-built data collection and as-planned data extraction, removal of
J. Comput. Civ. Eng., 2015, 29(1): 04014025

the subjectivity of progress detection through automated systematic Agarwal, S., Snavely, N., Simon, I., Seitz, S. M., and Szeliski, R. (2009).
detection, and, finally, interactive visualization to minimize the “Building Rome in a day.” Proc., Int. Conf. on Computer Vision, IEEE,
time required for discussions on coordining progress, possibly Kyoto, Japan.
leading to better decision making for project control. More experi- Akinci, B., Boukamp, F., Gordon, C., Huber, D., Lyons, C., and Park, K.
ments on the underlying techniques and further research on three (2006). “A formalism for utilization of sensor systems and integrated
project models for active construction quality control.” J. Autom.
fronts need to be conducted:
Constr., 15(2), 124–138.
1. 4D volumetric reconstruction: the proposed reconstruction pi-
Alves, N., and Bartolo, P. (2006). “Integrated computational tools for
peline shows great accuracy but has not yet been fully verified. virtual and physical automatic construction.” J. Autom. Constr., 15(3),
More experiments need to be conducted for the reconstruction 257–271.
of indoor environments and steel frameworks because indoor Arman, F., and Aggarwal, J. K. (1993). “Model-based object recognition in
areas in particular are textureless, which potentially makes dense-range images: A review.” Comput. Surv., 25(1), 5–43.
application of SIFT feature points (used in the developed Besl, P., and McKay, N. (1992). “A method for registration of 3-d shapes.”
system) more challenging. The photorealistic reconstruction IEEE Trans. Pattern Anal. Mach. Intell., 14(2), 239–256.
pipeline also needs to be tested on modeling the mechanical/ Bosché, F. (2010). “Automated recognition of 3D CAD model objects
electrical/plumbing components of buildings given the reflec- in laser scans and calculation of as-built dimensions for dimensional
tivity of the surface of some of these elements and the minimal compliance control in construction.” Adv. Eng. Inf., 24(1), 107–118.
volume they occupy. The scalability of the algorithm (tradeoff Bosché, F., Haas, C. T., and Akinci, B. (2009). “Automated recognition of
between thresh/voxel size) and the accuracy of the suggested 3D CAD objects in site laser scans for project 3d status visualization
as-built pipeline compared to the number of photographs need and performance control.” J. Comput. Civ. Eng., 10.1061/(ASCE)0887
to be further investigated. -3801(2009)23:6(311), 311–318.
2. Progress-monitoring detection: the model needs to be further Chang, C.-C., and Lin, C.-J. (2001). “LIBSVM: A library for support
enhanced by incorporating surface-recognition techniques to vector machines.” 〈http://www.csie.ntu.edu.tw/cjlin/libsvm〉 (May 1,
2010).
detect progress according to operational details. By properly
Cordova, F., Sideris, D., Brilakis, I., and Angelides, D. (2009). “Validation
forming Pðηi Þ [Fig. 15(c)], reporting progress at finer levels of
of vision tracking at Egnatia Odos Motorway.” Proc., ASCE Construc-
detail compared to the underlying WBS of the IFC model is
tion Research Congress, ASCE, Reston, VA.
possible. The stochastic components of error in registration Cornelis, N., Leibe, B., Cornelis, K., and Gool, L. V. (2008). “3D urban
and reconstruction need to be further examined. Finally, voxel scene modeling integrating recognition and reconstruction.” Int. J.
traversing needs to be tested under all camera configurations. Comput. Vision, 78(2–3), 121–141.
3. Progress sequence knowledge: the presented model does not Dai, F., and Lu, M. (2008). “Photo-based 3D modeling of construction
currently comprise any formalized schedule sequence ratio- resources for visualization of operations Simulation: Case of modeling
nale for construction monitoring. A formalized monitoring a precast façade.” Proc., 2008 Winter Simulation Conf., Informs
sequence knowledge based on the existing literature (e.g., Koo Simulation Society, Washington, DC, 2439–2446.
et al. 2007; Echeverry et al. 1991) will be developed. Instead Debevec, P. E., Taylor, C. J., and Malik, J. (1996). “Modeling and rendering
of assuming mutual independence among elements for a given architecture from photographs: A hybrid geometry and image-based
construction schedule, the progress-detection model will be approach.” Proc., 23rd Annual Conf. on Computer Graphics and Inter-
extended to incorporate such a sequencing rationale. Such an active Techniques, ACM, New York, 11–20.
extension to this work can further support the measurement of Du, S., Zheng, N., Ying, S., You, Q., and Wu, Y. (2007). “An extension of
earned progress. the ICP algorithm considering scale factor.” Vol. 5, IEEE Int. Conf. on
Image Processing, IEEE, San Antonio, TX, 193–196.
Echeverry, D., and Beltran, A. (1997). “Bar-code control of construction
field personnel and construction materials.” Proc., 4th Congress in
Acknowledgments Computing in Civil Engineering, Philadelphia, PA, ASCE, Reston,
VA, 341–347.
Authors like to thank Turner Construction and University of
Echeverry, D., Ibbs, C. W., and Kim, S. (1991). “Sequence knowledge for
Illinois–Housing and Facilities and Services for their contributions construction scheduling.” J. Constr. Eng. Manage., 10.1061/(ASCE)
to this research project. Special thanks to Mr. Greg Cuttell, Turner 0733-9364(1991)117:1(118), 118–130.
Construction’s project manager and Mr. Robert Bursack and El-Omari, S., and Moselhi, O. (2008). “Integrating 3D laser scanning and
Mr. Nick Canellis, Turner Construction’s project executives. photogrammetry for progress measurement of construction work.”
This work was funded by the National Science Foundation grant J. Autom. Constr., 18(1), 1–9.
CMMI-0800500. Any opinions, findings, and conclusions or Ergen, E., Akinci, B., East, B., and Kirby, J. (2007). “Tracking components
recommendations expressed in this paper are those of the authors and maintenance history within a facility utilizing radio frequency iden-
and do not reflect the views of the National Science Foundation or tification technology.” J. Comput. Civ. Eng., 10.1061/(ASCE)0887
-3801(2007)21:1(11), 11–20.
the companies and individuals mentioned above.
Everett, J., Halkali, H., and Schlaff, T. (1998). “Time-lapse video applica-
tions for construction project management.” J. Constr. Eng. Manage.,
10.1061/(ASCE)0733-9364(1998)124:3(204), 204–209.
References Fan, R. E., Chang, K. W., Hsieh, C. J., Wang, X. R., and Lin, C. J. (2008).
“LIBLINEAR: A library for large linear classification.” J. Mach. Learn.
Abeid, J. N., and Arditi, D. (2002). “Time-lapse digital photography Res., 9, 1871–1874.
applied to project management.” J. Constr. Eng. Manage., 10.1061/ Freund, Y., and Schapire, R. E. (1999). “A short introduction to boosting.”
(ASCE)0733-9364(2002)128:6(530), 530–535. J. Jpn. Soc. Artif. Intell., 14(5), 771–780.
Abeid, J., Allouche, E., Arditi, D., and Hayman, M. (2003). “PHOTO- Furukawa, Y., and Ponce, J. (2006). “High-fidelity image based modeling.”
NET II: a computer-based monitoring system applied to project Technical Rep. 2006-02, Univ. of Illinois, Urbana,IL.
management.” J. Autom. Constr., 12(5), 603–616. Furukawa, Y., and Ponce, J. (2010). “Accurate, dense, and robust multi-
Abudayyeh, O. Y. (1997). “Audio/visual information in construction view stereopsis.” IEEE Trans. Pattern Anal. Mach. Intell., 32(8),
project control.” Adv. Eng. Software, 28(2), 97–101. 1362–1376.
J. Comput. Civ. Eng., 2015, 29(1): 04014025

Furukawa, Y., Curless, B., Seitz, S. M., and Szeliski, R. (2009). “Recon- Lowe, D. (2004). “Distinctive image features from scale-invariant key-
structing building interiors from images.” Proc., Int. Conf. on Computer points.” Int. J. Comput. Vision, 60(2), 91–110.
Vision, IEEE, Kyoto, Japan. Lukins, T., and Trucco, E. (2007). “Towards automated visual assessment
Gilligan, B., and Kunz, J. (2007). “VDC use in 2007: Significant value, of progress in construction Projects.” Proc., British Machine Vision
dramatic growth, and apparent business opportunity.” CIFE Technical Conf., The British Machine Vision Association and Society for Pattern
Rep. #TR171, Stanford University, Stanford, CA. Recognition, Warwick, U.K.
Golparvar-Fard, M., Peña-Mora, F., and Savarese, S. (2011). “Integrated McGraw-Hill Construction. (2009). “The business value of BIM: Getting
sequential as-built and as-planned representation with D4AR tools in BIM to the bottom line.”〈http://www.bim.construction.com/research/
support of decision-making tasks in the AEC/FM industry.” J. Constr. pdfs/2009_BIM_SmartMarket_Report.pdf〉 (Oct. 7, 2009).
Eng. Manage., 137(12), 1099–1116. Meredith, J., and Mantel, S. (2003). Project management: A managerial
Golparvar-Fard, M., Peña-Mora, F., and Savarese, S. (2009a). “D4AR- A approach, 5th Ed., Wiley, New York.
4-dimensional augmented reality model for automating construction Moore, J. F. A. (1992). Monitoring building structures, Nostrand Reinhold,
progress data collection, processing and communication.” J. Inf. Tech- New York.
nol. Constr., 14, 129–153. Navon, R. (2007). “Research in automated measurement of project perfor-
Golparvar-Fard, M., Peña-Mora, F., Arboleda, C. A., and Lee, S. H. mance indicators.” J. Autom. Constr., 16(2), 176–188.
(2009b). “Visualization of construction progress monitoring with 4D Navon, R., and Sacks, R. (2007). “Assessing research in automated
simulation model overlaid on time-lapsed photographs.” J. Comput. project performance control (APPC).” J. Autom. Constr., 16(4),
Civ. Eng., 10.1061/(ASCE)0887-3801(2009)23:6(391), 391–404. 474–484.
Golparvar-Fard, M., and Peña-Mora, F. (2007). “Development of visuali- Nistér, D. (2004). “An efficient solution to the five-point relative
zation techniques for construction progress monitoring.” Proc., ASCE pose problem.” IEEE Trans. Pattern Anal. Mach. Intell., 26(6),
Int. Workshop on Computing in Civil Eng, ASCE, Reston,VA. 756–770.
Gordon, C., Boukamp, F., Huber, D., Latimer, E., Park, K., and Akinci, B. Nuntasunti, S., and Bernold, L. (2002). “Beyond WEBCAM: A site-
(2003). “Combining reality capture technologies for construction defect Web-Site for building construction.” Proc., Int. Symp. on Automation
detection: A case study.” Proc., 9th EuropIA Int. Conf. (EIA9), Istanbul, and Robotics in Constr., IAARC, Bratislava, Slovakia.
Turkey, 99–108. Ordonez, C., Arias, P., Herraez, J., Rodriguez, J., and Martin, M. (2008).
Hartley, R., and Zisserman, A. (2004). Multiple view geometry, Cambridge “Two photogrammetric methods for measuring flat elements in build-
University Press, Cambridge, U.K.
ings under construction.” J. Autom. Constr., 17(5), 517–525.
Horn, B. (1987). “Closed-form solution of absolute orientation using unit
Park, H. S., Lee, H. M., Adeli, H., and Lee, I. (2007). “A new approach for
quaternions.” J. Optic. Soc. A, 4(4), 629–642.
health monitoring of structures: terrestrial laser scanning.” Comput.
Huber, D., and Hebert, M. (2003). “3D modeling using a statistical sensor-
Aided Civ. Infrastruct. Eng., 22(1), 19–30.
model and stochastic search.” Proc., Computer Vision and Pattern
Peterson, F., and Fischer, M. (2009). “Project monitoring methods explor-
Recognition. 2003 IEEE Computer Society Conference on, Vol. 1,
atory case analysis: industry responses.” ASCE Int. Workshop on
IEEE, Madison, WI, 858–865.
Computing in Civ. Eng., ASCE, Reston, VA.
Huertas, A., and Nevatia, R. (2000). “Detecting changes in aerial views of
Podbreznik, P., and Rebolj, D. (2007). “Real-time activity tracking system–
man-made structures.” Image Vision Comput., 18(8), 583–596.
The development process.” 24th W78 Conf.: Bringing ITC Knowledge
Ibrahim, Y. M., and Kaka, A. P. (2008). “Review of photographic/imaging
to Work, Maribor, Slovenia.
applications in construction.” J. Built Human Environ. Rev., 1(2008),
Poku, S., and Arditi, D. (2006). “Construction scheduling and progress
99–117.
control using geographical information systems.” J. Comput. Civ. Eng.,
Ibrahim, Y. M., Lukins, T. C., Zhang, X., Trucco, E., and Kaka, A. P.
10.1061/(ASCE)0887-3801(2006)20:5(351), 351–360.
(2009). “Towards automated progress assessment of workpackage
components in construction projects using computer vision.” Adv. Pollefeys, M., et al. (2008). “Detailed real-time urban 3D reconstruction
Eng. Inf., 23(1), 93–103. from video.” Int. J. Comput. Vision, 78(2–3), 143–167.
Jaselskis, E., and El-Misalami, T. (2003). “Implementing radio frequency Quiñones-Rozo, C., Hashash, Y., and Liu, L. (2008). “Digital image
identification in the construction process.” J. Constr. Eng. Manage., reasoning for tracking excavation activities.” J. Autom. Constr., 17(5),
10.1061/(ASCE)0733-9364(2003)129:6(680), 680–688. 608–622.
Jung, Y., Chin, S., and Cho, C. (2004). “Automated progress measurement Reinhardt, J., Garrett, J., and Scherer, J. (2000). “The preliminary design of
framework using standard work packages.” Proc., 4th Int. Conf. on a wearable computer for supporting construction progress monitoring.”
Construction Project Management, ICCPM, Marina, Singapore, Internationales, Kolloquium über die Anwendung der Informatik
472–436. und der Mathematik in Architektur und Bauwesen, Univ. of Weimar,
Kim, H., and Kano, N. (2008). “Comparison of construction photograph Weimar, Germany.
and VR image in construction progress.” J. Autom. Constr., 17(2), Schindler, G., Krishnamurthy, P., Lublinerman, R., Liu, Y., and Dellaert, F.
137–143. (2008). “Detecting and matching repeated patterns for automatic geo-
Kiziltas, S., Akinci, B., Ergen, E., and Tang, P. (2008). “Technological tagging in urban environments.” Proc., Computer Vision and Pattern
assessment and process implications of field data capture technologies Recognition, IEEE, Anchorage, AK.
for construction and facility/infrastructure management.” J. Inf. Tech- Seitz, S. M., and Dyer, C. R. (1999). “Photorealistic scene reconstruction
nol. Constr., 13, 134–154. by voxel coloring.” Int. J. Comput. Vision, 35(2), 151–173.
Koo, B., and Fischer, M. (2000). “Feasibility study of 4D in commercial Shih, N., and Wang, P. (2004). “Point cloud-based comparison between
construction.” J. Constr. Eng. Manage., 10.1061/(ASCE)0733-9364 construction schedule and as-built progress: Long-range three-
(2000)126:4(251), 251–260. dimensional laser scanner’s approach.” J. Archit. Eng., 10.1061/
Koo, B., Fischer, M., and Kunz, J. (2007). “Formalization of construction (ASCE)1076-0431(2004)10:3(98), 98–102.
sequencing rationale and classification mechanism to support rapid Shih, N., Lai, J., and Tsai, Y. L. (2006). “The application of a panorama
generation of sequencing alternatives.” J. Comput. Civ. Eng., 10.1061/ image database management system (pidms) for information integration
(ASCE)0887-3801(2007)21:6(423), 423–433. on construction sites.” ITcon, 11, 641–654.
Leung, S., Mak, S., and Lee, B. (2008). “Using a real-time integrated com- Sinha, S., Steedly, D., Szeliski, R., Agrawala, M., and Pollefeys, M. (2008).
munication system to monitor the progress and quality of construction “Interactive 3D architectural modeling from unordered photo collec-
works.” J. Autom. Constr., 17(6), 749–757. tions.” Proc., SIGGRAPH Asia 2008, ACM, New York, 1–10.
Levoy, M., et al. (2000). “The digital Michelangelo project: 3D scanning Snavely, N., Seitz, S. M., and Szeliski, R. (2006). “Photo tourism: Explor-
of large statues.” Proc., ACM SIGGRAPH 2000, ACM, New York, ing photo collections in 3D.” ACM Trans. Graph. (SIGGRAPH Proc.),
131–144. 25(3), 835–846.
J. Comput. Civ. Eng., 2015, 29(1): 04014025

Snavely, N., Seitz, S. M., and Szeliski, R. (2008). “Modeling the Trupp, T., Marulanda, C., Hashash, Y., Liu, L., and Ghaboussi, J.
world from internet photo collections.” Int. J. Comput. Vision, 80(2), (2004). “Novel technologies for tracking construction progress of deep
189–210. excavations.” Proc., Geotechnical Engineering for Transportation
Soibelman, L., Wu, J., Caldas, C., Brilakis, I., and Lin, K. Y. (2008). Projects, ASCE, Reston, VA, 2254–2262.
“Management and analysis of unstructured construction data types.” Uffenkamp, V. (1993). “State of the art of high precision industrial photo-
J. Adv. Eng. Inf., 22(1), 15–27. grammetry.” Proc., 3rd Int. Workshop on Accelerator Alignment,
Song, L. (2007). “Progress measurement using CAD-based vision system.” Springer, Berlin, Heidelberg.
Proc., 2007 Construction Research Congress, ASCE, Reston, VA. Wu, Y., and Kim, H. (2004). “Digital imaging in assessment of construction
Su, Y., Hashash, Y., and Liu, L. Y. (2006). “Integration of construction
project progress.” Proc., 21st Int. Symp. on Automation and Robotics in
as-built data via laser scanning with geotechnical monitoring of urban
Construction, IAARC, Bratislava, Slovakia.
excavation.” J. Constr. Eng. Manage., 10.1061/(ASCE)0733-9364
(2006)132:12(1234), 1234–1241. Yoders, J. (2009). “Wisconsin becomes the first state to require BIM
Teizer, J., Kim, C., Haas, C., Liapi, K., and Caldas, C. (2005). “Framework on large, public projects.” 〈http://www.bdcnetwork.com/wisconsin
for real-time three-dimensional modelling of infrastructure.” Geology -becomes-first-state-require-bim-public-projects〉 (Jul. 20, 2009).
and properties of earth materials 2005, Transportation Research Board Zhang, X., et al. (2009). “Automating progress measurement of construc-
National Research Council, Washington, DC, 177–186. tion projects.” J. Autom. Constr., 18(3), 294–301.
Triggs, B., McLauchlan, P., Hartley, R., and Fitzgibbon, A. (1999). “Bundle Zebedin, L., Bauer, J., Karner, K., and Bischof, H. (2008). “Fusion of
adjustment – A modern synthesis.” Int. Workshop on Vision Algorithms, feature- and area-based information for urban buildings modeling from
Springer, Berlin, Heidelberg, 153–177. aerial imagery.” Comput. Vision–ECCV, 5305, 873–886.
J. Comput. Civ. Eng., 2015, 29(1): 04014025

Automated Progress Monitoring Using Unordered Daily

Uploaded by

Document Informationclick to expand document information

Document Informationclick to expand document information

Copyright:

Available Formats

Automated Progress Monitoring Using Unordered Daily

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Automated Progress Monitoring Using Unordered Daily

Uploaded by

Copyright:

Available Formats

Automated Progress Monitoring Using Unordered Daily

Construction Photographs and IFC-Based

Introduction collection, processing, and representation are time-consuming and

© ASCE 04014025-1 J. Comput. Civ. Eng.

J. Comput. Civ. Eng., 2015, 29(1): 04014025

© ASCE 04014025-2 J. Comput. Civ. Eng.

J. Comput. Civ. Eng., 2015, 29(1): 04014025

© ASCE 04014025-3 J. Comput. Civ. Eng.

J. Comput. Civ. Eng., 2015, 29(1): 04014025

© ASCE 04014025-4 J. Comput. Civ. Eng.

J. Comput. Civ. Eng., 2015, 29(1): 04014025

Overview of D4 AR Progress Visualization and

As shown in Fig. 2, the present work is based on the joint appli-

cation of daily construction photographs, an IFC-based BIM as

© ASCE 04014025-5 J. Comput. Civ. Eng.

J. Comput. Civ. Eng., 2015, 29(1): 04014025

Recently, Golparvar-Fard et al. (2009a) sparsely reconstructed

© ASCE 04014025-6 J. Comput. Civ. Eng.

J. Comput. Civ. Eng., 2015, 29(1): 04014025

© ASCE 04014025-7 J. Comput. Civ. Eng.

J. Comput. Civ. Eng., 2015, 29(1): 04014025

Table 2. Registration Errors Measured on Reconstructions Shown in Fig. 4 for SD Project

SD project: Image size

© ASCE 04014025-8 J. Comput. Civ. Eng.

J. Comput. Civ. Eng., 2015, 29(1): 04014025

where SD = standard deviation of color values; σ20 = accuracy

is not satisfied, the voxel is labeled Empty (Eb ); finally, if the

© ASCE 04014025-9 J. Comput. Civ. Eng.

J. Comput. Civ. Eng., 2015, 29(1): 04014025

Fig. 7. As-built voxel labeling and image marking; if a voxel contains

Occupied [Op ]. Subsequently, the voxel is projected on all images

© ASCE 04014025-10 J. Comput. Civ. Eng.

J. Comput. Civ. Eng., 2015, 29(1): 04014025

For the as–planned∶ PðθiP Þ ¼ P ð10Þ

  Fig. 9. Tracking physical progress for activity in work schedule

© ASCE 04014025-11 J. Comput. Civ. Eng.

J. Comput. Civ. Eng., 2015, 29(1): 04014025

Fig. 11. (a–d) Dense reconstruction of SD data set

© ASCE 04014025-12 J. Comput. Civ. Eng.

J. Comput. Civ. Eng., 2015, 29(1): 04014025

© ASCE 04014025-13 J. Comput. Civ. Eng.

J. Comput. Civ. Eng., 2015, 29(1): 04014025

þ1 0.51 0.85 1.00 1 0.20 1 tion of the SVM hyperplane.

© ASCE 04014025-14 J. Comput. Civ. Eng.

J. Comput. Civ. Eng., 2015, 29(1): 04014025

operational details (e.g., forming stages), the formwork is falsely Contribution

© ASCE 04014025-15 J. Comput. Civ. Eng.

J. Comput. Civ. Eng., 2015, 29(1): 04014025

© ASCE 04014025-16 J. Comput. Civ. Eng.

J. Comput. Civ. Eng., 2015, 29(1): 04014025

© ASCE 04014025-17 J. Comput. Civ. Eng.

J. Comput. Civ. Eng., 2015, 29(1): 04014025

© ASCE 04014025-18 J. Comput. Civ. Eng.

J. Comput. Civ. Eng., 2015, 29(1): 04014025

© ASCE 04014025-19 J. Comput. Civ. Eng.

J. Comput. Civ. Eng., 2015, 29(1): 04014025

You might also like

Fig. 9. Tracking physical progress for activity in work schedule