Professional Documents
Culture Documents
2009 - Visualization of Construction Progress Monitoring With 4D Simulation Model Overlaid On Time-Lapsed Photographs - Journal of Compu
2009 - Visualization of Construction Progress Monitoring With 4D Simulation Model Overlaid On Time-Lapsed Photographs - Journal of Compu
2009 - Visualization of Construction Progress Monitoring With 4D Simulation Model Overlaid On Time-Lapsed Photographs - Journal of Compu
Abstract: The ability to effectively communicate progress information and represent as-built and as-planned progress discrepancies are
identified as key components for successful project management that allow corrective decisions to be made in a timely manner. However,
Downloaded from ascelibrary.org by University of Manitoba on 01/25/22. Copyright ASCE. For personal use only; all rights reserved.
current formats of reporting 共e.g., textual progress reports, progress curves, and photographs兲 may not properly and quickly communicate
project progress. Current monitoring methods also require manual data collection and extensive data extraction from different construction
documents, which distract managers from the important task of decision making. Therefore, to facilitate progress monitoring, this paper
proposes visualization of performance metrics that aims to represent progress deviations through superimposition of four-dimensional
共4D兲 as-planned model over time-lapsed photographs in single and comprehensive visual imagery. As a part of the developed system,
registration of the 4D model with photographs, augmenting photographs, and occlusion removal for progress images are presented. While
contextual information is preserved, the as-built photographs are enhanced and augmented with 4D as-planned model in which the
performance metrics are visualized. The augmented photographs provide a consistent platform for representing as-planned, as-built, and
progress discrepancies information and facilitate communication and reporting processes.
DOI: 10.1061/共ASCE兲0887-3801共2009兲23:6共391兲
CE Database subject headings: Monitoring; Imaging techniques; Computer aided simulation; Information technology 共IT兲; Con-
struction management; Registration.
both as-planned and as-built information to be integrated and vi- resources, procurement and quality.
sualized to provide a holistic view of all processes during con- 2. To accurately convert as-built progress data from construc-
struction progress. Therefore in this paper, a visualization tion operation into information. The system should be realis-
tic and should recognize means of processing the
technique for progress monitoring is presented that visualizes
information, the skills available, and the value of information
progress deviations through superimposition of four-dimensional
compared with the cost of obtaining it.
共4D兲 as-planned model on real time-lapsed photographs in single
3. To identify and assess the critical information from a given
and comprehensive visual imagery.
progress situation.
Visualization of the as-planned progress in 4D environment
4. To report the information to managers in time and in a form
enables project participants and clients—regardless of their level which can best be interpreted by management, and at an
of construction knowledge and expertise—to understand spatial appropriate level of detail for the individuals who will be
constraints and explore design and construction alternatives be- using it so that corrective action could be taken on the
fore construction starts. It provides a consistent visual platform progress situation that generated the data in the first place.
and a common language of as-planned construction that could be 5. To record the control action taken; to represent the as-built
extended to the monitoring phase 共Song et al. 2005兲. performance of the project.
In addition, visualization of as-built progress in photographs Without data collection, thorough comparison of the planned
not only has the advantage of being understandable to those who and as-built performance and a proper communication and re-
are not well versed in studying written material or numerical data cording, there may be no basis for proper project control and
analysis or even those who question verbal or written reports decision making. This requires progress data to be analyzed and
共Oglesby et al. 1989兲, but also allows large amount of data to be maintained in a desired level of detail 共i.e., according to decision
understood and absorbed quickly. Compared with progress report- maker’s needs兲 so that understanding the progress would be easier
ing techniques which generate words and numbers, techniques 共Jung and Kang 2007兲; however the process of monitoring is
such as time-lapse photography and videotaping provide a rich faced with a series of challenges such as
data set that can be a good source for as-built data collection and 1. Current progress monitoring is time consuming and labor
act as good communication tools for progress monitoring among intensive. Projects are not constantly monitored making it
the project participants. very difficult to take corrective actions in a timely basis.
Considering these advantages, the presented visualization Current methods require manual data collection and also ex-
model integrates the 4D model and time-lapsed photographs tensive data extraction from construction drawings, sched-
within an augmented reality 共AR兲 environment where progress ules, and budget information produced by project teams in
discrepancies are identified and visualized. To present this model, which none is independent 共Navon 2007兲. Field staffs collect
this paper begins with introducing the challenges of progress progress data from the construction site, analyze, and deliver
monitoring followed by the proposed methodology that visualizes them to project managers in a format specific to their areas of
discrepancies between the as-planned and as-built progress. In the expertise, e.g., construction drawings, spreadsheets, bar
charts, critical path method 共CPM兲, or progress site photo-
subsequent sections, three important steps for the augmented pho-
graphs or videos. Such discrete and exhaustive reports could
tograph are explained in details: 共1兲 4D simulation as the under-
be produced but do not explicitly convey level of perfor-
lying context for representing visualized metrics; 共2兲 Time-lapse
mance, problems, and their causes and impacts on construc-
photography and real-time filming for as-built progress data col-
tion situation 共Song et al. 2005兲. Consequently, project
lection; and 共3兲 visualization techniques to represent performance managers need to devote significant amount of time and ef-
metrics on augmented photograph. As a part of the system, reg- fort to sort out, prioritize, and interpret these data.
istration of the 4D model with photographs, augmenting photo- 2. Quality of manually collected and extracted progress data are
graphs and occlusion removal for progress images of the building low. Manual collection of progress information—usually ac-
structure and façade are presented. While contextual information quired by field staff—is dependent on the status seen on site
in photographs is preserved, the real-world image is enhanced and and the information collected, which in turn makes it subjec-
augmented with 4D simulation model where performance metrics tive and may not reveal the impact of site circumstances on
are visualized. The augmented photographs provide a consistent construction 共personal communication with field staffs on
platform for representing as-planned, as-built, and progress dis- five different projects 共9/2006–6/2007兲; Navon and Sacks
crepancies information and facilitate communication and report- 2007兲. This may affect the quality of the collected data and
ing processes. makes it error prone since the ability of anticipating possible
referenced is a small work unit, it may not have a significant 4D simulations to augment as-built progress photographs with the
difference; however, in case where the references are to the purpose of project monitoring and control. In this context, Song et
whole task or project, assumption of input/output proportion- al. 共2005兲 introduced project dashboard as a three-dimensional
ality could be very misleading 共Meredith and Mantel 2003兲. 共3D兲 model visual representation to show a holistic picture of a
Thus, the most commonly used methods to monitor progress project by applying the multiple project data sets to the geometric
are: 共a兲 Monitoring physical progress in percentile: used in attributes 共e.g., shape, faces, and edges兲 of the building product
most construction fields that heavily relies on experience and model through color-tone variations and motion. It was suggested
knowledge of the project management personnel. This metric that consistent application of colors would allow project perfor-
is used subjectively and is inefficient at presenting progress mance metrics to be represented easily. This would also purge
due to its abstract nature representation of physical progress visual complexities which could be caused by complexities asso-
共Song et al. 2005兲; 共b兲 Budget based monitoring: based on ciated with large and sophisticated building product models.
percentage of the budget paid to contractors according to the Nonetheless in their presented system, as-built progress was not
schedule-based inspections. This method of monitoring cre- visualized using photographs or any other means different from
ates time lag between progress estimations and schedule up- that of as-planned model. Rather, geometric attributes of the
dates; besides, judgments are usually subjective and building product model were used to communicate progress.
misleading especially if a field manager makes any erroneous Based on the same concept of consistent visual representation,
decision 共Shih and Wang 2004兲. Without a specific compara- Lee and Peña-Mora 共2006兲 superimposed planned product models
tive analysis on construction plan, resources, and cost data, with photographs and initiated a new paradigm on visualization of
wrong assumption and inaccurate measurement on the construction progress monitoring where deviations between
progress status could be made. Mistakes such as over paying planned and as-built performance models were conceptually rep-
and overlook of expected delay might appear. resented in an AR environment. AR is an environment wherein
4. Progress monitoring reports are visually complex. Kerzner virtual and real world are combined to enhance user’s experience
共2005兲 argued that 30 to 40 different data representations are of the virtual world through contextual information 共Wang and
currently being used in construction industry. These graphi- Dunston 2005; Azuma 1997兲. It gives the user the ability of ob-
cal representations can serve several functions such as show- serving the background environment and superimposes virtual
ing data, analysis methodology, and communication means model over the real-world background. Considering the benefits
共Oglesby et al. 1989兲. These methods require drawing, of visualization techniques and AR environment for visualization
sketches 共to show layout and physical details兲, and graphs and automation of progress monitoring specifically assisting
and charts 共which present numerical data and the results ob- project managers, a new approach is presented which integrates
tained by observation兲 to represent schedule, cost, and per- three different modules.
formance. The choice among them is dependent on the 1. 4D Simulation as the as-planned progress information,
intended audience. For example, upper level management 2. Time-lapse photography and videotaping as the as-built
may be interested in costs and integration of activities with progress data collection, and
very little detail; hence summary-type charts normally suffice 3. Visualizing progress through augmenting the as-built photo-
for this purpose. Daily practitioners, on the other hand, may graph with the as-planned data
require as much detail as possible in daily schedules. In ad-
dition, understanding the situation only based on the sched-
4D Simulation as the As-Planned Progress Data
ules may be difficult as they lack information relating to
spatial context and complexities of project components 共Koo 4D simulations have been developed for the main purposes of
and Fischer 2000兲. None of the existing reporting methods detecting spatial and temporal conflicts, understand construction
effectively present multivariable information 共i.e., schedule, logistics, coordinate the construction with subcontractors and
cost, and performance兲 in a holistic manner nor do they re- trades, and demonstrate the planned progress to the owners
flect the spatial and visual aspects of as-planned and as-built 共Tabesh and Staub-French 2006; Kamat and Martinez 2003; Kam
construction and their associated complexities simulta- and Fischer 2002兲. This kind of a time-based monitoring focuses
neously. Consequently it affects the ability of communicating on a preconstruction study that will allow for better management
effective progress information which is a definite prerequisite of a site afterward 共Haymaker and Fischer 2001兲. So far, 4D tools
for successful project management. have not been used for project progress management through as-
Based on the deficiencies mentioned and considering chal- built data collection during the construction phase 共Chin et al.
lenges with current reporting formats and communications, the 2005兲. Currently 4D applications are able to detect some sched-
visualization of progress monitoring is presented. In the following uling errors in the construction and enable project participants and
Fig. 1. Augmented progress images: 4D simulation superimposed on time-lapsed photographs of a building construction project clockwise from
10/03/2006 to 12/02/2006 共Photograph subject: College of Business Instructional Facility, UIUC. Source: Facilities and Services, UIUC.兲
clients, regardless of their level of construction knowledge, to rain and snow weather conditions respectively and shows how
understand the spatial constraints and explore design and con- weather conditions can affect the quality of time-lapsed photo-
struction alternatives before construction starts. However, a pre- graphs in a construction site.
construction simulation cannot necessarily take into account Shadow is another problem which is caused by adjacent build-
every incident that might occur to the parts of a building under ings or elements 共temporary or permanent兲 and affects visual
construction but could be used as a base for the as-planned infor- quality of photographs. Figs. 4共a–c兲 show three photographs that
mation. 3D and 4D models can provide realistic visual expres- are selected out of a working day photo log. These photographs
sion, a consistent visual platform, and a common language of show how shadow affects the visibility of the work site and how
as-planned information for all parties involved within a project these effects may reduce visibility of the elements on a photo-
共Song et al. 2005兲. Specifically, an industry foundation classes graph. However since a full set of photographs are collected dur-
共IFC兲-based 4D system that contains all the information of the ing working hours, different zones and locations on the jobsite
as-planned parts and their relationships, not only can serve as a could be studied at different times.
complete as-planned database but also serves as the underlying Since progress is usually measured within weeks or even days
structure for monitoring progress where deviation between the 共not hours兲, this allows the selection of more visible photographs
as-planned and as-built progress could be visualized. Fig. 1 dem- for the purpose of analysis and representation. Furthermore, ap-
onstrates snapshots of a 4D simulation superimposed on time- plication of robust image analysis and pattern recognition tech-
lapsed photographs.
niques such as Gaussian filters 共Forsyth and Ponce 2003兲 allows geted in time-phased planning value increment, it can constitute a
the photographs to be enhanced which in turn reduces and/or performance measurement baseline 共Abba 1997兲 which is useful
neutralizes these effects. for comparison. EVA provides information in terms of as-built
Overall, compared with techniques that generate only words conditions, potential issues, prior concerns, and future scenarios
and numbers, photography has the advantage of being easily un- in one construct. Along with time photography as an automated
derstandable and believable to those who are not well versed in data collection method, 4D model and cost database, for every
studying material or numerical data analysis or who question ver- decision making, EVA performance metrics needs to be calculated
bal and written reports 共Oglesby et al. 1989兲. This technique is and visualized. The scheme of using this methodology is further
particularly useful when the activities depicted are not going as explained in the subsequent section.
smoothly as they might, since it is difficult for anyone to argue
successfully that the photographs do not portray the as-built work
situation.
Progress Monitoring Visualization System Scheme
Visualizing Construction Progress Visualization process consists of a series of modules which results
Considering the existing generic progress monitoring methods, in color coded time-lapsed AR imageries. Fig. 5 summarizes the
earned value analysis 共EVA兲 can provide a monitoring basis and information action-representation-environment perspectives for
allows the future performance to be forecasted. Although EVA has the proposed system. As represented in Fig. 5, raw data are col-
some limitations as referred in Kim and Ballard 共2000兲, but since lected from two different sources: the as-planned and the as-built
in EVA, all the construction work is planned, scheduled, and bud- performance environments. The collected information represents
Fig. 3. Different weather conditions during a construction project: 共a兲 fog; 共b兲 rainy; and 共c兲 snow 共Photograph subject: Institute of Genomics
Biology, UIUC. Source: Information, Technology and Communication Services, College of ACES, UIUC.兲
Fig. 4. Effect of shadow on a single working day 共Photograph subject: Institute of Genomics Biology, UIUC. Source: Information, Technology
and Communication Services, College of ACES, UIUC.兲
product models, i.e., IFC 3D as-planned model and site photo- namics of the construction operations and the site, such as tem-
graphs 共Fig. 5, 1-A and 1-C兲, process models, i.e., working sched- porary structures and site layout. In this case, a more detailed
ule and operation process 共Fig. 5, 3-A and 3-C兲 and cost modules, progress measurement is possible. However, visualizing detailed
i.e., estimated and performed costs 共Fig. 5, 4-A and 4-C兲. construction activities such as electrical rough-ins may not be
Collected information from these two environments is merged possible and therefore at this stage, the proposed model only in-
to produce a 4D as-planned simulation and time-lapsed photo- corporates construction schedules with a visible physical progress
graphs 共Fig. 5, 2-A and 2-C, respectively兲. For any given time, level in terms of the building structure and façade. Based on the
the as-planned model is superimposed on the as-built perfor- guideline set during the planning phase and generation of the
mance model 共i.e., site photograph兲 共Fig. 5, 2-B兲. This process as-planned model, interpretation and assessment of the data
involves proper registration of the 3D virtual world and photo- would be performed in the subsequent steps.
graph coordinates. The superimposed imagery would allow dis- At the following step, a series of visualization techniques is
crepancy to be either manually or automatically detected and applied to visualize EVA metrics 共Fig. 5, 5-B兲; i.e., according to
quantified 共Fig. 5, 3-B and 4-B兲. At this stage, cost values are the status of the progress, the as-planned model would be color
extracted from estimated and actual construction cost modules coded and superimposed on top of the photograph. Once the sta-
and are integrated to the system 共Fig. 5, 4-A and 4-C兲. This would tus of the progress is represented on the superimposed image, any
allow cost information required for EVA to be derived. This in- occlusion and blockage caused by the superimposition should be
formation is appended to the known as-planned and as-built in- removed. Therefore depth and perspective integrity of the virtual
formation and allows the budget spent to be properly assessed and and actual environments is maintained 共Fig. 5, 6-C兲. The final
the cost discrepancies to be understood. imageries are represented for decision making and are kept as a
The next step is to monitor progress against the performance record for progress monitoring photo log. Fig. 6 shows a color-
measurement baseline, or the planned value. The physical earned coded superimposed image where the progress status is visual-
value performed is then related to the actual costs spent to accom- ized. In this figure, on-schedule entities are represented in light-
plish the physical work performed, providing a measure of the green entities, ahead of schedule entities in dark green, and
project’s cost performance. To establish the guideline for the pro- behind-schedule entities in red color.
posed system, the IFC as-planned model and the work break This reporting process is repeated for every coordination cycle
down structure in the schedule are considered as the basis of where control actions are taken and the construction schedule is
monitoring 共Fig 5, 2-A兲. The level of details is based on the revised and updated by project participants. For example, if
product and process work breakdown structures and cost estimat- architectural/engineering/construction teams have coordination
ing scheme in the as-planned model. For example if a general meeting every other week, this report would provide progress
contractor schedule is provided, only major activities associated information from the last meeting to the current meeting consid-
with building elements are visualized in the 4D model and moni- ering the same time period for future activities that may have
toring system only involves that level of details in the schedule as been performed in the project. In this section, an overview of the
the baseline for measurements. If a detailed daily schedule is data collection, comparison baseline, and assessment processes
available, the 4D as-planned model may further visualize the dy- have been discussed. In the sections that follow, each of the steps
ahead of schedule, and red entities are behind schedule 共Photograph subject: College of Business Instructional Facility, UIUC. Source: Facilities
and Services, UIUC.兲
共i.e., registration, assessment, representation, and recording兲 is Then after knowing the camera is fixed, correspondence be-
discussed in detail. tween the 3D model and photograph needs to be set. The photo-
graph and camera coordinate systems are related by a set of
physical parameters, such as the focal length of the camera lens,
Geometric Camera Calibration
the size of the pixels, the position of the principle point 共of the
The first task in generating superimposed images is to relate the lens兲, and the position and orientation of the real world 共Forsyth
3D virtual model to the two-dimensional photograph. This means and Ponce 2003兲. In order to register the camera, these intrinsic
any point in the 3D model such as P共x , y兲 needs to be precisely and extrinsic parameters of the camera should be defined. Intrin-
related to image coordinates p共u , v兲. In the case of time-lapsed sic parameters relate the camera coordinate system to the ideal-
photographs, the camera location is fixed on the construction site, ized coordinate system 共i.e., effective focal length, aspect ratio,
therefore, ideally, if camera is registered once, the correspon- image center coordinates, and radial distortion coefficient兲 and
dences between the photographs and the virtual model would be extrinsic parameters relate camera’s coordinate system to a fixed
set for all subsequent images; i.e., with the same correspondence camera coordinate system and specify its position and orientation
relationship, all the images of the 4D environment could be su- in space 共i.e., rotation matrix and translation vector兲 共Forsyth and
perimposed on the photographs. However, photographic cameras, Ponce 2003兲. Estimating these parameters of a camera is called
webcams, and/or video recorders similar to any surveying camera geometric camera calibration 共Forsyth and Ponce 2003兲. If we
are subject to displacement and vibration caused by gravity and consider p = 共u , v , 1兲 and P = 共x , y , z , 1兲 to be the homogenous co-
lateral forces such as wind. Fig. 7 shows the camera registration ordinates of the points in the photograph and the virtual world,
error. A deviation in camera angle can make a major error in the relationship between these points in general terms can be
registration. As seen, perspective views A and B are seen from the represented as
same location with different view angles.
Due to these potential registration errors, the camera needs to 1
be regularly adjusted. In this scenario, it is assumed that the lo- p = MP 共1兲
z
cation of the camera will be regularly fixed at all times and the
same correspondence relationship between the 3D virtual world where M 3⫻4 = projection matrix and z = depth of the point.
and the photograph coordinates could be applied to all time- To solve Eq. 共1兲 for M, a set of features 共i.e., points兲 with
lapsed photographs. known positions in the photograph and the 3D environment are
required. In order to achieve good and predictable results for M,
there is a certain amount of preparation that needs to be done
manually. This preparation consists of two main areas: 共1兲 Iden-
tifying “matchable” features in the photograph and 共2兲 associating
3D features 共from the virtual model兲 to the features chosen in the
photograph.
It is extremely important for setting correspondences and solv-
ing the equation to choose accurate features 共points兲 within pho-
tograph and 3D model. However, since photographs are formed in
pixels, finding the accurate positions for features could be a very
challenging task. For example, let’s assume that the selected fea-
ture to be the top corner of two converging concrete walls. In a
low quality photograph, this point could be located in-between
pixels and it affects the quality of the preferred location for reg-
Fig. 7. Camera registration error: a deviation in the camera angle istration purposes which affect preciseness and quality of regis-
within the distance of the camera to the site could generate a major tration. An alternative to increase the accuracy for solving this
error in registration. Perspective views A and B show the result equation is to extract more features from the photograph. There-
of deviation in the photograph taken by the camera from the same fore, by selecting more features to establish the correspondence,
location. the error in a single feature selection would not significantly af-
Fig. 10. From top to bottom: 共a兲 the site photograph taken on December 2, 2006, 1:13:27 p.m.; 共b兲 snapshot of the 4D model at the same time
as the photograph; 共c兲 superimposed image; 共d兲 schedule deviation detected and color coded according to the schedule in 共e兲; and 共f兲 color-coded
superimposed 3D model on the site photograph 共Photograph subject: College of Business Instructional Facility, UIUC. Source: Facilities and
Services, UIUC.兲
shot of the 4D as-planned model taken on the same time are different colors 共light green for on schedule and red for behind
represented in Figs. 10共a and b兲, respectively. Based on the visual schedule兲 are assigned to each of the components depending on
comparison between the photograph and the as-planned model, its progress status.
discrepancies are identified. These discrepancies have been manu- It is clear that as more components are constructed or install
ally analyzed and the physical components of the basement level on the site, the number of 3D model components in the 4D envi-
that are behind or on-schedule are identified. The schedule devia- ronment increases and as a result visual representation could po-
tion is quantified by the management team based on the construc- tentially become more complex. Given the identified deviations, a
tion schedule and based on the EVA analysis performed. Then, consistent visual scheme is required to simplify and facilitate its
Fig. 11. Critical information sets for project managers during construction phase and the color spectrum
time-lapsed photographs, i.e., the shape or location of these fea- Dynamic Feature Extraction
tures, do not change rapidly. Once the superimposition of the Along with aforementioned static features, dynamic features also
colored as-planned model on the site photograph is performed, exist within photographs such as construction machinery, tempo-
some of these features are overlaid with the color coded 3D rary structures, and work crew. These features also need to be
model, while in camera’s line of sight, these features are located extracted to be overlaid back on the photograph to preserve depth
in front of the as-planned 3D model and should not be occluded and perspective within the superimposed photograph. The same
by the model. Hence these features need to be extracted from the feature detection technique i.e., SUSAN are applied to dynamic
original site photograph and overlaid on the superimposed image. feature extraction. As seen in Fig. 13 the truck in front of the
Since these static features do not frequently change throughout foundation wall is also recovered. This method required manual
the period of construction, once they are recovered, the same supervision and could be time consuming while other techniques
process could be applied to the rest of the time-lapsed photo- such as identification of moving objects between consecutive im-
graphs. In computer vision, there are many different edge detec- ages or using different points of view that do not have the ma-
tion techniques that could help in finding these features within chinery crossing their field of view could also be considered for
photograph. These methods include but are not limited to Canny, future implementations to reduce such overheads. 共The writers
Sobel, Robert, Laplacian of Gaussian, and SUSAN 共smallest uni- would like to acknowledge that this idea was suggested by one of
value segment assimilating nucleus兲 edge detection techniques the anonymous reviewers of the paper.兲
共Shin et al. 2001; Smith and Brady 1997兲. All these methods
could be used to detect corners of the features and localize them.
Visualized Progress Report
Among these methods, SUSAN edge detector had been applied in
this research due to its good detection, localization, response, and Fig. 14 illustrates a visualized report of progress monitoring. In
speed to be usable for image processing systems compared to the this figure, the photographs and 4D snapshots are presented and
rest of the mentioned edge detection techniques 共Shin et al. 2001; based on the work schedule and the comparison performed, de-
Smith and Brady 1997兲. In this method, nonlinear filtering is used viations are identified and are color coded. The deviations are also
to define image parts that are closely related to each individual quantified based on the number of days according to the schedule
pixel. These pixels are associated with their local regions within and are reported. Finally, based on the actual cost occurred and
the photograph that have similar brightness to that pixel. Feature the planned costs, cost performance index 共CPI兲 and schedule
detectors are based on minimization of local regions and noise performance index 共SPI兲 are calculated and presented 关Fig. 14共f兲兴.
reduction. The detail of this method is not scope of this paper and These forms of reporting can facilitate the coordination process
could be found in Smith and Brady 共1997兲; rather the applicabil- by reducing the time to inform the participants as to what the
ity of the method to extract static features is described in the AR situation is. Once the superimposed photographs are ready, the
model. Fig. 13 shows the result of applying the SUSAN edge report table could be generated. This sort of representation does
detection in recovering the excavation line, light poles, and some not require the observer to have any expertise or knowledge about
machinery located on the construction site. The recovery of the construction operations.
information below excavation line in the photograph has been Based on the positive feedback received from the profession-
done in a supervised manner, i.e., the required recovery section is als and executives of the five construction case studies in this
manually selected and it is automatically applied to subsequent research as well as other executives from leading construction
time-lapsed photographs. companies, writers believe the visualization will facilitate
Another possible way to overcome static occlusion problem is progress monitoring process.
to model the static occluding objects in 3D model and have them
hide the geometry of the augmented images just as any real object
hides the background in photographs. 共The writers would like to Conclusions
acknowledge that this idea was suggested by one of the anony-
mous reviewers of the paper.兲 However this method may increase This preliminary method has shown that AR environment can
the level of details required for 3D modeling and would not be successfully represent progress monitoring information in forms
suitable for cases where the occluding objects, themselves needs of as-planned, as-built information along with their comparison in
to be detected and/or tracked. a holistic manner. The superimposed images retain all the con-
Fig. 14. Visualized monitoring report: 共a兲 as-built photographs; 共b兲 4D snapshots; 共c兲 color coded virtual components; 共d兲 quantification of the
deviation; 共e兲 augmented photographs; and 共f兲 measured EVA performance metrics 共CPI and SPI兲
struction site information while the planned information along AR environment that automatically and distinctively recognize
with the status of progress is enriching the contextual information visual construction content within site photographs, compare with
within these photographs. The registration method gives the op- the as-planned 4D model and visualize the status of progress
portunity for image processing to be applied to specific regions using visualization and project management techniques. Consid-
within the photograph to assess the status of the progress based on ering application of time-lapsed photographs for visualization of
material and shape recognition techniques. Color-coding meta- as-built data collection, two major challenges are identified.
phors give the end users of the superimposed photograph the
opportunity of grasping progress status based only on a single
representation form and could facilitate the communication of Challenges with Time-Lapsed Photographs for
progress status within a coordination meeting, allowing more Visualization and Assessment of the As-Built Data
time to be spent on control decision making. Moreover prelimi-
nary results of applying feature detection technique preserves Occlusion/Proximity Problems for Data Collection
depth and perspective within the superimposed photograph allow- Type of structures 共e.g., steel, concrete, and composite兲, camera
ing a more realistic picture of the progress status to be repre- location for taking time-lapsed photographs 共e.g., ground level
sented. The overall methodology and reporting addresses the versus upper levels, proximity of components to the camera兲,
issues related to data collection and reporting steps of a robust horizontal and vertical obstacles 共e.g., static objects on site or
progress monitoring. blockage of the view of one element by the others兲, and outdoor
versus indoor monitoring are all among the challenges of visual-
izing as-built data. Fig. 15 shows two different scenarios on hori-
Future Work zontal and vertical occlusions and the challenges of visualizing
progress only on a single view. As seen in Figs. 15共a and b兲, a
This work is part of a larger project that aims to automatically column is occluded since there is another column which is block-
generate superimposed photographs for progress monitoring. ing camera’s line of sight toward to the specific column under
Overall, the aim is to develop methods and processes within an study. This situation has been solved by moving the camera to a
Fig. 15. 共a兲 Plan view of two column grids while one of the columns has occluded camera’s point of view on the other column; 共b兲 photograph
of the occluded column; 共c兲 position of the camera was changed and the column is not occluded; 共d兲 photograph of nonoccluded view; and 共e兲
camera pose problem in vertical situations, from ground level the highlighted beam is not visible
new location. The vertical occlusion case is also shown in Fig. with such level of detail depends on the robustness of image
15共e兲 where the slab has blocked the view toward the beam. There processing techniques in overcoming data collection problems
is a need for finding the optimum locations of a network of cam- and considering the occlusions.
eras both for outdoor and indoor progress monitoring to make Based on these challenges, the future work and automation of
sure all the elements could be monitored and data required for the visualization falls in four categories: 共1兲 exploring more visu-
as-built progress is collected. In addition, authors suggest using alization techniques and perform testability and applicability of
an unordered set of registered photographs that are taken from these techniques in communication of progress monitoring infor-
various viewpoints to tackle the occlusion issue, and/or using a mation on concurrent representation of performance metrics and
remote helicopter to capture photographs in order to avoid angle work sequence visualization; 共2兲 exploring time-lapsed photo-
and line of sight issues for higher elevations. graphing for data collection, optimum locations for exterior and
interior data collection considering all the challenges discussed;
Automatic Photograph Analysis for Progress Monitoring 共3兲 addressing measurement, quantification, and assessment of
Considering all the challenges with shadow and illuminations, a progress status using effective image processing and computer
series of image processing and pattern recognition techniques is vision concepts along with using edge detection techniques from
required to make sure progress of any type of element regardless one side and on the other hand, formalizing a database for the
of the material used or the texture of the surface could be detected system based on IFC; and 共4兲 applying the technique on various
under nonsevere noise in the photograph. construction projects. Ultimately, an all-inclusive methodology
will be developed that not only visualized progress status, but
Challenges with As-Planned Data automatically collects data, analyses the photographs, compares
them to as-planned model database and visualizes progress status.
Although the baseline for progress monitoring would be the as-
planned model, the correspondence between the schedule infor-
mation and a product model on one side and the level of details
within the 4D model on the other side, creates two major chal-
lenges for its application as a baseline for monitoring: 共1兲 Activi- Acknowledgments
ties with no correspondence in the 4D model: The 4D as-planned
model does not represent all the information within the schedule. The writers acknowledge the financial support for this research
It may not show activities within schedule that do not have cor- received from the National Science Foundation Awards Grant
responding components. For example there is no way to commu- Nos. CMS-0324501 and CMS-0427089. The contribution to this
nicate quality control activities within the current 4D models; and paper by Kipling Mecum, director of Emergency Planning, Clif-
共2兲 Level of details in the 4D model: The major challenge with ford Carey, project manager of University of Illinois Service and
the as-planned data are mostly related to the level of detail for Facilities, University of Illinois College of Business, Post Ge-
progress monitoring. Most of the 4D models can be viewed only nomics Institute, Brian Stauffer, photographer, Steve Hass, assis-
in one level of detail. For example, it many only communicate tant director of office for Information Management, as well as
schedule information within the general contractor’s interest and Aravind Sridharan, Sangwon Han, Edgar Matta, Sebastian Derza,
it may not include all the shop drawing details, however for Joe Garb, Joe Blecha, JR Patton and Lesley Santa at University of
progress monitoring these details could affect the decision made Illinois, Urbana-Champaign is also acknowledged. Any opinions,
on the progress observed. For instance, considering a steel struc- findings, and conclusions or recommendations expressed in this
ture, the level of details within joints between columns and beams publication are those of the authors and do not necessarily reflect
could be modeled but it may not be possible to perform the com- the views of the National Science Foundation or any of the afore-
parison in that level of details. The applicability of a 4D model mentioned people.
construction site image retrieval.” J. Comput. Civ. Eng., 19共4兲, 341– project performance control 共APPC兲.” Autom. Constr., 16共4兲, 474–
355. 484.
Chin, S., Yoon, S., Kim, Y., Ryu, J., Choi, C., and Cho, C. 共2005兲. “Real Oglesby, C., Parker, H., and Howell, G. 共1989兲. Productivity improve-
time 4D CAD+ RFID for project progress management.” Proc., Con-
ment in construction, McGraw-Hill, New York.
struction Research Congress 2005, ASCE, Reston, VA, 168–172. Peña-Mora, F., Han, S., Lee, S., and Park, M. 共2008兲. “Strategic-
Forsyth, D., and Ponce, J. 共2003兲. Computer Vision: A Modern Approach, operational construction management: Hybrid system dynamics and
Prentice-Hall, N.J.
discrete event approach.” J. Constr. Eng. Manage., 134共9兲, 701–710.
Golparvar-Fard, M., Sridharan, A., Lee, S., and Peña-Mora, F. 共2007兲.
Poku, S., and Arditi, D. 共2006兲. “Construction scheduling and progress
“Visual representation of construction progress monitoring metrics on
control using geographical information systems.” J. Comput. Civ.
time-lapse photographs.” Proc. Construction Management and Eco-
Eng., 20共5兲, 351–360.
nomics 25th Anniversary Conf., Univ. of Reading, U.K., 1–9. Shih, N., and Wang, P. 共2004兲. “Point-cloud-based comparison between
Golparvar-Fard, M., Staub-French, S., Po, B., and Tory, M. 共2006兲. “Re- construction schedule and as-built progress: Long-range three-
quirements for a mobile interactive workspace to support design de- dimensional laser scanner’s approach.” J. Archit. Eng., 10共3兲, 98–
velopment and coordination.” Proc., XI Joint Int. Conf. on Computing 102.
and Decision Making in Civil and Building Engineering, ASCE, Re- Shin, M., Goldgof, D., and Bowyer, K. 共2001兲. “Comparison of edge
ston, Va., 3587–3596. detector performance through use in an object recognition task.” Com-
Haymaker, J., and Fischer, M. 共2001兲. “Challenges and benefits of 4D put. Vis. Image Underst., 84共1兲, 160–178.
modeling on the Walt Disney Concert Hall project.” Rep. No. 064, Slaughter, S. 共1998兲. “Models of construction innovation.” J. Constr.
Center of Integrated Facility Engineering, Stanford Univ., Stanford, Eng. Manage., 124共3兲, 226–231.
Calif. Smith, S., and Brady, J. 共1997兲. “SUSAN—A new approach to low level
Jung, Y., and Kang, S. 共2007兲. “Knowledge-based standard progress mea-
image processing.” Int. J. Comput. Vis., 23, 45–78.
surement for integrated cost and schedule performance control.” J.
Song, K., Pollalis, S., and Peña-Mora, F. 共2005兲. “Project dashboard:
Constr. Eng. Manage., 133共1兲, 10–21.
Concurrent visual representation method of project metrics on 3D
Kam, C., and Fischer, M. 共2002兲. “PM4D final report.” Technical Rep.
No. 143, Center of Integrated Facility Engineering, Stanford Univ., building models.” Proc., Int. Conf. on Computing in Civil Engineer-
Stanford, Calif. ing, ASCE, Reston, Va., 1–10.
Kamat, V., and Martinez, J. 共2002兲. “Comparison of simulation-driven Songer, A., and Heys, B. 共2003兲 “A framework for multi-dimensional
construction operations visualization and 4D CAD.” Proc., 2002 Win- visualization of project control data.” Proc., Construction Research
ter Simulation Conf., IEEE, Piscataway, N.J., 1765–1770. Congress 2003, ASCE, Reston, Va., 121–130.
Kerzner, H. 共2005兲. Project management: A systems approach to plan- Tabesh, R., and Staub-French, S. 共2006兲. “Modeling and coordinating
ning, scheduling, and controlling, 7th Ed., Wiley, New York. building systems in three dimensions: A case study.” Can. J. Civ.
Kim, Y. W., and Ballard, G. 共2000兲. “Is the earned-value method an Eng., 33共12兲, 1490–1504.
enemy of work flow?” Proc., 8th Annual Conf. of the Int. Group for Wang, X., and Dunston, P. 共2005兲 “Heavy equipment operator training
Lean Construction, IGLC-6, Lean Construction Institute, Brighton, via virtual modeling technologies.” Proc., Construction Research
U.K. Congress 2005, ASCE, Reston, Va., 618–622.