Version of Record: https://www.sciencedirect.

com/science/article/pii/S0926580517300274
Manuscript_dfd7781d36cea98b8fccdd94c9060926

1 Linking BIM and Design of Experiments to Balance Architectural and

2 Technical Design Factors for Energy Performance
3
4
5 Authors
6 Arno Schluetera, Philipp Geyerb
7
8 a
Arno Schlueter, Architecture and Building Systems, Institute of Technology in Architecture (ITA), ETH
9 Zürich1
10 b
Philipp Geyer, Architectural Engineering Division, Faculty of Engineering Science, KU Leuven
11

12 Abstract
13 To transform the existing energy systems towards renewable energy sources, buildings need to use less
14 energy, use energy more efficiently and harness local renewable energy sources. For the design of
15 energy-efficient buildings, building energy simulation of varying sophistication is commonly employed.
16 Types of simulations range from simple, static calculations to sophisticated dynamic simulation.
17 Especially for building retrofit many assumptions on construction, material etc. have to be taken, which
18 increases the uncertainty of simulation results. In conjunction with simulation, methods of Building
19 Performance Optimization are increasingly employed. They are able to identify best performing designs
20 however do not provide insights on the mechanisms and interdependencies of the different design factors,
21 which are most valuable to make informed design decisions. We present a methodology that aims to
22 provide a better understanding and create knowledge about the influence and interactions of different
23 architectural and technical design factors on building energy performance of a specific design task. For
24 this purpose, we introduce Design of Experiments (DoE) in an integrated design workflow using the
25 Design Performance Viewer (DPV) toolset, combining Building Information Modeling (BIM),
26 distributed dynamic simulation and statistical analysis of the extensive simulation results. The
27 experiments created using the methodology allow to identify the strength of effects and interactions of
28 different design factors on selected performance indicators. We apply the methodology on an office
29 retrofit case, introducing a factor scatterplot for result visualization, development and comparison of
30 retrofit strategies. We further evaluate its potential to identify high performing strategies while balancing
31 architectural and technical factors and their impact on energy performance.
32
33 Keywords: Design of Experiments (DoE), Building Information Modeling (BIM), dynamic simulation,
34 distributed simulation, building retrofit, retrofit design strategies
35

1Corresponding author: Arno Schlueter, email: schlueter@arch.ethz.ch, Tel.: +41 44 633 93 91, Stefano-Franscini
Platz 1, CH 8093 Zürich

1
© 2017 published by Elsevier. This manuscript is made available under the Elsevier user license
https://www.elsevier.com/open-access/userlicense/1.0/
36 1 Introduction
37 Worldwide, buildings consume 32% of the global final energy and 25% greenhouse gas
38 emissions (GHG) [1]. To transform the existing energy systems towards renewable energy
39 sources and thus to meet the ambitious emission goals set by many countries and organizations
40 such as the European Union [2], future buildings will need to use less energy, use energy more
41 efficiently and harness local renewable energy sources. Compared to the existing building stock,
42 the annual rate of new and retrofitted buildings in Europe, however, is low. In Germany and
43 Switzerland, for example, retrofit rates stagnate at a low level of around 1-2% [3]. Studies claim
44 that this is due to insufficient information, low cost-effectiveness of measures and regulatory
45 constraints that make it challenging to find the appropriate solution [4].
46 To identify effective retrofit measures, building energy simulation is employed, sometimes
47 during design, most often however afterwards in order to comply to energy codes. Such
48 simulations range from norm-based steady-state calculations to sophisticated dynamic building
49 energy simulations. Due to missing information about the building itself, they are often based
50 on a variety of assumptions, simplifications and educated guesses. Whereas simulation data can
51 easily be produced, knowledge and understanding about the effects and interactions of different
52 parameters is often lacking. The objective of this work is therefore to develop a methodology
53 and computational toolset to identify the influence and interactions of architectural and
54 technical design factors on building energy performance and thus to derive strategic knowledge
55 for the designer rather than just numerical results.
56 For this purpose, we link Building Information Modeling (BIM), which is used to create and
57 store the necessary data and information, and Design of Experiments (DoE), an established and
58 successful methodology used in engineering disciplines and industry. DoE is a method that uses
59 simulation experiments and applies statistical analysis to obtain information on effects and
60 interactions between different factors, aiming at the least number of experiments necessary. To
61 embed DoE into the design process, we employ the Design Performance Viewer (DPV), a
62 toolset that allows establishing a workflow from a Building Information Model (BIM) to
63 parametric simulation, results collection and statistical analysis.
64 The paper is structured as follows: First, we provide an overview on existing approaches of
65 parametric design and environmental simulation for sustainable building design. Next, we
66 introduce DoE as a method and its previous application for building performance. In Section 3
67 we outline the integration of DoE into the Design Performance Viewer toolset to establish a
68 DoE workflow based on a BIM. This workflow is exemplified in Section 4 using an office
69 retrofit case study, identifying a range high-performing yet unique retrofit strategies. Section 5
70 summarizes and discusses the methodology whereas Section 6 concludes and provides an
71 outlook on future developments.

72 2 Background
73 The field of parametric building modeling and simulation and related methods of data analysis
74 and optimization constitutes the relevant background of this paper. Additionally, we briefly
75 review the method of Design of Experiments and its application to buildings.

76 2.1 Building Energy Performance Optimization

77 A large body of research exists that couples parametric building modeling, environmental
78 simulation and optimization. The study of Gero et al. [5] and the Building Design Advisor [6]
79 represent early, integrated design approaches combining multiple analysis and visualization
80 tools. Caldas and Norford [7] utilize genetic algorithms to search for optimized environmental
81 design solutions, focusing on façade configurations. Janssen [8] explores balancing heat losses
82 of the envelope and potential heat gains through openings using evolutionary approaches.
83 Turrin et al. [9] use genetic algorithms for the design of passive solar roofs. Heiselberg et al.

2
84 [10] apply sensitivity analysis to identify the most important design parameters of a sustainable
85 building design. Related to cooling in office buildings, Breesch and Janssens [11] utilize
86 uncertainty and sensitivity analysis to predict the performance of free cooling using building
87 energy simulation. More recently, multi-objective optimization techniques increasingly assist in
88 performance-driven building design already at the conceptual stage [12], [13], addressing
89 energy, exergy, lifecycle cost and other domains. A study by Attia et.al., which also provides an
90 extensive review on building performance optimization, highlights a consensus on the general
91 usefulness of building performance optimization tools to achieve energy efficient buildings
92 [14]. The study also mentions shortcomings as perceived by experts in the field such as
93 uncertainty of simulation model input, low trust in results and low interoperability for exchange
94 between different applications.
95 In practical application, a range of available tools link parametric building modeling to
96 environmental analysis, such as Autodesk Vasari or Design Builder, which uses EnergyPlus as
97 simulation engine. Additionally, a range of plugins for similar purposes exist, such as DIVA,
98 Honeybee or Ladybug for Rhinoceros. Both tools and plugins mainly focus on building
99 geometry, materials and resulting heating/cooling load calculations, which are delivered as
100 numerical results.
101 Rather than to base design decisions on numerical simulation and optimization results only, we
102 advocate to use a performance-driven design workflow including DoE as a method to obtain an
103 understanding of the nature and impact of design parameters, their interdependencies and
104 trends, and thus to be able to build knowledge within a specific design context. Such a workflow
105 can be automated; the interpretation however requires the judgement of the expert. The focus on
106 knowledge rather than data constitutes the main difference to the aforementioned optimization
107 approaches which provide results optimized for certain criteria, however little insight or
108 understanding about the mechanisms behind.

109 2.2 Design of Experiments

110 Design of Experiments (DoE) has been established and successfully applied in various fields
111 and industries, such as product design and development [15], chemical [16] and software
112 engineering [17]. A number of comprehensive textbooks exist [18]–[20]; therefore the general
113 methodology is outlined here only in brief. As part of the application, we exemplify each
114 methodological step for the case study.
115 The general concept of DoE is to create a series of real or simulated experiments on a system or
116 system model under observation. In each experiment, one or multiple of its design parameters
117 are changed and the impact on the system or model behavior is evaluated. Which parameters are
118 changed and how they are changed is defined using an experiment plan. The aim is to obtain as
119 much information as possible using the least amount of experiment runs, as experiments are
120 costly in terms of physical setups or computational effort. The behavior of the system based on
121 the parameter changes is observed using a set of outputs. In the context of DoE, the outputs can
122 be referred to as the ‘performance indicators’, the design parameters as the ‘factors’ and their
123 value settings as factor ‘levels’.
124 To analyze the impact of each factor on the system and its interactions with other factors, every
125 factor combination would need to be tested and therefore would require an experiment. Due to
126 the large amount of possible combinations this is an effort often infeasible in terms of time and
127 costs. DoE offers a range of methods - referred to as experiment plans or design tables - to
128 reduce the amount of necessary experiment runs to achieve as precise information as possible
129 with the smallest number of experiments. The type of plan used depends on the experiment’s
130 objective. Finding the optimal design table, i.e. the least amount of experiments that is
131 necessary to depict the systems behavior correctly has been subject to intense research. Using a
132 set of statistical evaluation methods on the resulting experiment data, the impact, effect and

3
133 interactions of the factors in relation to a chosen performance indicator is evaluated. The results
134 of the experiments can be used to formulate a mathematical surrogate model, also called a
135 metamodel in engineering disciplines [26, 27].

136 2.2.1 Application for Building Energy Performance

137 DoE has only rarely been applied in a building context. Chlela et al. [21] and Jaffal et al. [22]
138 present the application of the DoE methodology for selected design parameters of new low
139 energy buildings, targeting thermal performance. The focus of their work is on the exploration
140 of various existing design tables that reduce the amount of simulation runs to derive a
141 polynomial metamodel. Describing the mean and maximum error of the resulting metamodels
142 as the main outcome and key evaluation criterion, the work focuses rather on the technical
143 aspects of applying a DoE methodology for a new building design in general than on its
144 qualitative outcome and applicability for retrofit designs.
145 More recently, DoE in combination with dynamic energy simulation has been applied to
146 identify factors for cooling loads on a residential building [23], and similarly, for deriving a
147 metamodel for heating and cooling of low energy housing in Morocco [24]. For the simulation
148 of new buildings, the selection of factors for the experiments is only restricted by the simulation
149 capabilities whereas for building retrofit, the choice of factors and factor levels is constrained by
150 the condition of the existing building and the available design options. As it is not possible to
151 alter factors such as location, orientation or basic construction, the balancing of the remaining
152 factors is even more important.
153 The aforementioned examples address the design of new buildings and focus on the technical
154 application of DoE for building simulation. This work aims to move beyond in order to
155 advocate the use DoE in a performance-driven design workflow for creating understanding and
156 knowledge about the impact of retrofit measures, their interactions and the identification of
157 optimal retrofit strategies.

158 3 Linking BIM and Design of Experiments using the DPV toolset
159 We use and extend the Design Performance Viewer (DPV) toolset (Fig.1) for performance-
160 driven building design under development since 2008. It facilitates an integrated and bi-
161 directional link between a Building Information Model, using Autodesk Revit as BIM editor,
162 and
163 • energy/exergy simulation and analysis [25],
164 • dynamic building energy simulation using EnergyPlus [26],
165 • multi-scale building and urban energy co-simulation linking EnergyPlus and CitySim [27],
166 • scientific workflow management for defining custom simulation workflows [28], and
167 • roundtrip data exchange between a Building Information Model and a Building Energy
168 Model using a service oriented architecture [29].

169 The DPV addresses designers and engineers that are accustomed to software tools for modeling
170 and simulation as part of the design process. Case studies have shown its applicability in a
171 performance-driven design process [26], [30], [31]. A key feature of the toolkit is, unlike many
172 existing tools, the bi-directionality from design model to simulation/visualization and back,
173 rendering the usual export-import processes between tools, which are prone to data loss and
174 errors, unnecessary. One of the core components of the DPV which has gained popularity
175 among researchers and developers in the field is the Revit Python Shell (RPS) [32]. To keep a
176 fast and intuitive workflow, a variety of techniques were developed such as knowledge-based
177 defaults, material and construction databases and others. To aid the designer in validating the
178 model for correct simulation input, a semantic viewer for visual debugging and a testing harness
179 for automated validation of geometric design alternatives were developed and included.

4
180

181
182 Fig. 1: The Design Performance Viewer toolset

183 To integrate DoE as a methodology, the toolset was extended by an automated zoning algorithm
184 for fast identification of thermal zone affiliations, parallel computing capabilities for multiple
185 simulations and a connection to the ‘R’ software package [33] for simulation results data
186 analysis and results visualization.

187 3.1 DoE Design Process

188 The methodology proposed can be structured in several procedural steps (Fig.2) that begin with
189 the establishment of a Building Information Model (BIM) as the design database. In terms of
190 geometry, the model contains all structural elements of the envelope and interior walls and
191 floors. Also included is the volume and cubature of the neighboring buildings that may cast
192 shadows onto the building. For material and construction experiments, an external database can
193 be accessed, containing a variety of materials and construction prototypes. Internal gains for
194 people, lighting and electrical appliances can be scheduled and set according to Swiss Norm
195 SIA 2024 [34].
196 To allow for a high resolution for performance-driven design using DoE it is necessary to be
197 able to distinguish between different thermal zones to run zone-specific dynamic simulations.
198 We have developed a thermal zoning algorithm to automatically extract thermal zones, their
199 boundaries such as walls, windows and roofs and their neighboring relation to other zones. It
200 uses a zone box, which is a three-dimensional object that is placed inside the building geometry
201 in the BIM modeler and represents a zone or part of a zone. Rays are cast from the center of the
202 zone box in all directions and update the zone affiliation of the first surface hit by each ray. A
203 building surface’s zone affiliation is stored as an attribute in the BIM which also can be
204 manually changed. A building surface has at least one zone affiliation – the default zone. To
205 ensure correct simulation, special care has to be taken to ensure that each building surface is
206 affiliated with at most two zones before serializing the internal model to an EnergyPlus IDF file
207 for simulation. This requires splitting up surfaces by their zone affiliations, into sub-surfaces
208 with at most two affiliated zones.

5
209
210 Fig. 2. Procedural steps of integrating DoE

211 The DPV model builder is capable of reading, reducing and structuring the BIM into a valid
212 EnergyPlus input file. Visual debugging tools help ensuring that the relevant objects are
213 semantically correct. EnergyPlus, which is one of the most comprehensive and widespread
214 dynamic simulation engines available, serves as simulation engine for the whole building
215 including its construction and HVAC equipment. It is constantly validated against data of
216 existing systems and buildings [35]. The selected design factors such as insulation type and
217 thickness, window to wall ratio, material properties and heating/cooling inlet temperatures are
218 parameterized in the input file using the ‘parametric runs’ feature. This allows generating the
219 different simulations according to the chosen experiment setup.

220 4 Experiment Methodology and Execution

221 In the following section, we describe the methodology and exemplify its application using the
222 DPV toolset on a case study of a recently retrofitted office building.

223 4.1 Case Study Building

224 Finished in 1971, the building belongs to an original part of a campus ensemble worthy of
225 preserving (Fig.3). Maintaining its exterior appearance was stated as one of the key design
226 objectives, therefore the possible retrofit options were limited and had to be negotiated well. As
227 a case it is typical for many office buildings built in the 60’s and 70’s all over Europe with high
228 energy consumption that need to be retrofitted. For the simulation a long-term averaged weather
229 data file of Zurich was used. The average annual air temperature is 7.9 °C. During the winter
230 months of December, January and February the average monthly air temperature is 0° C, 1.0° C
231 and 0.2° C, respectively. The freestanding office building contains 2226 m2 of office space on
232 four levels, totaling in 8200 m3 of heated volume. For simulation and parametric control, the
233 volume is partitioned into 22 thermal zones that are either corner zones facing two directions,

6
234 intermediate zones facing one direction or core zones that have no boundary to the outside. All
235 internal loads are set and scheduled according to typical office occupation and operation
236 schedules provided by the Swiss Norm SIA. The floor area per person is 20m2, the lighting load
237 installed is 8 W/m2, and the load by appliances is 2.15 W/m2. The building structure consists of
238 a concrete core and slabs that are supported by concrete columns, providing a large amount of
239 thermal mass. The façade, which is still in its original condition, is made of aluminum sandwich
240 panels filled with thin insulation. The original glazing has been replaced once in 1989 by double
241 glazing. The window-to-wall ratio over the entire building is approximately 50%. The building
242 systems for heating, cooling and ventilation were idealized for load calculation.

243
244 Fig. 3: Case study building (left) BIM model with DPV semantic viewer, results viewer (right)

245 Measured heating end use energy data was used to validate the simulation model. Data was
246 available from a period of six years before the retrofit and one full year after. Both the
247 measurement and the simulation results were normalized using averaged 10-year heating degree
248 days from 2001 to 2010 for Zurich, Switzerland [36]. For the base case, a normalized average
249 annual building heating end use energy of 72.7 kWh/m2a was measured over a period of six
250 years before the retrofit. The normalized heating end use energy of the base case simulation,
251 using the factor levels that represent the original state of the building, is 72.9 kWh/m2a, which
252 constitutes a 0.3% difference. The measured and normalized annual heating energy
253 consumption of the first full year after retrofit was 67.3 kWh/m2a. The simulation of the
254 executed retrofit case using the factor levels that represent the retrofit conditions shows 63.75
255 kWh/m2a which is about a 6% difference. A detailed description of a comprehensive
256 model/measurement calibration process for retrofit using the DPV toolset is described in [26].

257 4.2 Case Study Experiment Definitions

258 Setting up the experiments requires the definition of performance indicators, the choice of
259 factors and appropriate factor levels as well as choosing an appropriate experiment plan.
260 Performance indicators are computable quantities, for example heating energy consumption,
261 that are used as categories to describe the performance of a solution. In the context of
262 performance-driven building design such indicators can be emissions, energy use, investment or
263 life-cycle cost or cost-efficiency. Factors are design parameters that specify selected properties
264 of a solution, for example the thermal resistance of a construction, the dimensions of an opening
265 or the inlet temperatures of a heating system. Varying these factors in a coordinated manner
266 using experiment matrices and simulating these parameter settings allows identifying their
267 impact on the performance indicators as well as interactions between each other.

268 4.2.1 Performance Indicator and Factor Definitions

269 For the experiments, we formulate a set of strategies representing different retrofit approaches
270 that are very different in architectural and constructive implications. These strategies are

7
271 encoded as factor/level combinations and their results, respectively. To conceptualize and
272 quantify such strategies, a thorough understanding of the effect and interactions of the relevant
273 parameters is necessary, which will be facilitated by the methodology proposed.
274 For the experiment, heating and cooling energy end use are chosen as performance indicators as
275 they are most influenced by the retrofit. Site energy was chosen over source energy to compare
276 the actual amounts of energy required to operate the building, neglecting source, generation and
277 losses, which are beyond our control due to a centralized energy system on campus. Additional
278 performance indicators could also be thermal comfort values such as operative interior
279 temperature or cost-efficiency of measures.
280 For the case study, factors and factor levels were chosen to correspond to realistic design
281 options in context of the building case. Level values for the base case and the executed retrofit
282 were chosen based on available design documentation. Each factor level represents a realistic
283 design measure, for example choosing a higher window to wall ratio means replacing the
284 existing facade by a new one. The following factor/level combinations were chosen:
285 • Window to wall ratio (RO): Aggregated over the entire building, the original facade has a
286 window to wall ratio of approximately 50%, which is depicted as level 2. Potentials
287 measures include smaller window elements (level 1), but also larger elements (levels 3 and
288 4). Level 4 represents a fully glazed facade. All changes in opening ratio imply the
289 replacement of the existing facade.
290 • U-value of vertical envelope (IV): The average u-value of the original vertical opaque
291 facade is approximately 1.5 (level 1). Level 2 represents a minimum additional interior
292 insulation, level 3 and 4 additional insulations that would make new facade elements
293 necessary.
294 • U-value of horizontal envelope (IH): The average u-value of the original horizontal
295 envelope is 0.8 (level 2). Levels 1 and 3 represent different thicknesses of insulation on the
296 roof; level 4 depicts a high standard as recommended by regulation.
297 • Solar energy transmittance (g-value) (WG): The approximate g-value of the original
298 vertical, glazed façade is 0.58 (level 2). Levels 1 and 3 represent glazing types with
299 different g-values; level 4 represents a low-e coated glazing with very little energy
300 transmittance.
301 • U-value windows (WU): The U-value of the vertical, glazed facade (glazing and frame)
302 before the retrofit is approximately 1.55 (level 2) according to thermal imaging
303 measurements. Level 1 represents the original state from 1971, levels 2 and 3 represent
304 exchanging the glazing to better performing double glazing whereas level 4 means
305 replacing the entire facade due to the use of triple glazing.
306 • Shading control (SC): The original shading (horizontal louvers) is manually controlled by
307 the occupants. This is approximated by using 150 W/m2 of irradiance as the threshold to
308 lower the shades (level 2). Level 1 represents the absence of external shading as
309 1000W/m2 on the vertical surfaces are never reached and thus the shading is never
310 activated. Levels 3 and 4 represent automated external shading with different controls set
311 points.

8
312
313 Tab. 1: Factors/levels, abbreviation, base case before retrofit and executed retrofit

314 4.2.2 Assessing Alternative Experiment Plans

315 The efficiency, applicability and validity of the experiments relies to a large extent on the
316 structure and resolution of the design, i.e. the experiment plan, which is expressed in the design
317 table [20]. The plan selection and thus the sizing of the design table depends on the number of
318 factors and the effects and interactions to be examined. In the following section, we assess
319 different plans in context of the case study to identify the plan with the least amount of runs
320 necessary yet still acceptable error, in case a full factorial cannot be used for reasons of
321 computational effort. Additionally, the reduction of the number of runs changes the duration of
322 the computation from hours to a couple of minutes, which would support a fast and iterative
323 design workflow.
324 To compare the performance of different table designs we first run a full factorial experiment
325 plan to compute every possible factor and level combination. This allows to emulate any
326 specific experiment plan including its results. Due to the exponential growth in number of
327 experiments this is only feasible for a limited number of factors and levels. Compared to full
328 factorial plans, fractional factorial plans use a selection of factor combinations to reduce the
329 number of experiment runs while maintaining the information gain. Being among the most
330 suitable approaches especially for complex computer experiments using a larger amount of
331 factor levels [18], [20], we focus on orthogonal arrays for generating suitable experiment plans.
332 To identify a suitable fractional factorial experiment plan for the retrofit experiments, a range of
333 plans different in generation algorithms and number of runs are compared. For the creation of
334 the matrices, two different generators are used. One version with 128 experiments is generated
335 using the ‘R’ Design of Experiments package including the tables prepared by Kuhfeld [37].
336 For comparison, additional arrays with 56, 128 and 256 experiments are created and tested
337 using the coordinate-exchange algorithm (CEA) [38]. Additionally, two random plans of 128
338 and 256 runs are generated, choosing factors levels at random over the design space. Fig. 4
339 shows the maximum deviation of functions fitted separately for each factor and type of
340 simulation response. The design plan generated by R and the tables of Kuhfeld (WKR 128)
341 show the worst performance probably caused by a lack of an appropriate base table for the size
342 and levels of the experiment. Fig. 5 shows the advantage of DoE tables generated by the CEA
343 compared to a random experiment plan (RND). The cumulative maximum error of all factors at
344 the right of the figure is about 5 kWh/m2a lower for the CEA tables with same size than for the
345 RND tables, which is an improvement of 25 to 30%. Furthermore, the doubling of the
346 experiment size from 128 runs to 256 runs improves the result quality by another 4 to 5
347 kWh/m2a. Therefore, as a compromise of run time and accuracy, the CEA 128 plan provides the
348 best results.

9
349

350 Fig. 4: Maximum deviation of regression models for total, heating and cooling energy using different experiment
351 plans and compared to the full factorial data.

352 An additional qualitative criterion of how accurate a DoE plan describes the design space is the
353 deviation of the design strategies using a DoE table from the ones selected as exemplary design
354 strategies (Table 2). To determine the closest design strategy with distance dmin, the Euclidian
355 distance by level steps x is determined excluding the original selected point jDoE and, in case of
356 equal distances, the design with the least difference in the total energy consumption is selected:

357 (1)

358 Fig. 5 shows the average, the minimum and the maximum deviation of the total energy
359 consumption comparing the strategy selected from the full factorial with its closest neighbours
360 in the DoE table. The data include tables created by the CEA, random generators (RND) and the
361 Box-Benken-Design (BBD) [39] method with sizes from 49 to 4096. The results indicate that
362 the use of a DoE table for selecting a design strategy instead of the full factorial causes an
363 average error of about 10 to 15 kWh/m2a for the test case. This includes, in adverse cases, a
364 peak error of 35 kWh/m2a for some design tables, such as the RND256. The BBD method
365 creates the smallest tables, which is of special interest for computationally intensive
366 experiments. In particular, the CEA 128 delivered the best result with a maximum of less than
367 14 kWh/m2a.
368

369

10
370 Fig. 5: Average, minimum and maximum deviation of the energy consumption in kWh/m2a (y-axis) between the
371 closest strategy in the DoE table and the selected strategy from the full factorial. The x-axis denotes the number of
372 experiment runs.

373 Concluding, the CEA 128 experiment plan requiring only 128 experiment runs can be used
374 instead of the full factorial with acceptable deviation, which means reducing computational time
375 to about 3 % with acceptable deviation. Given appropriate parallelization [40], the DoE
376 simulation results can be obtained in similar time as a single simulation run.

377 4.3 Simulation, Results Collection, Data Analysis and Visualization

378 For the visualization and analysis of the full factorial experiment results we introduce a factor
379 scatterplot (Fig. 6, 7). Each column of plots represents a different factor separated by the factor
380 levels; each row shows a different performance indicator. The graph allows the interactive
381 filtering and selection of experiments (dots on the scatterplot) showing their positioning in the
382 solution space as well as their characteristic parameter combinations. Introducing this combined
383 graph, different characteristics of the factors can be analyzed:
384 • Distribution and variance: The density of the scatter shows distribution and variance of the
385 results over the entire range.
386 • Direction and slope: The slope of minimum values of the factor levels determined by
387 regression analysis demonstrates the positive/negative influence and impact of the factor on
388 the performance indicator considering the best available solutions. Using the best results
389 xfac,min and y per factor level i, the linear regression is defined as

390 (1)

391 where γ is the y-axis intersection and ε the error.

392 Furthermore, the graphs allow the manual selection and exploration of retrofit strategies: As
393 each experiment is represented with its results and factor levels, picking a desired result with a
394 certain characteristic (for example a well performing result with an opening ratio of 50%) also
395 shows all other related factor levels that lead to the result. This allows for the identification of
396 favorable factor settings for the formulation of design strategies.
397

398

11
399 Fig.6: Scatterplots of full factorial results per factor/level: Ratio of Openings (RO),
400 Vertical Insulation (IV), Horizontal Insulation (IH)

401
402 Fig. 7: Scatterplots of full factorial results per factor/level: Solar Gain (WG),
403 Window Insulation (WU), Shading Control (SC)

404 4.3.1.1 Interpretation of the Results

405 The scatterplots can now be used to interpret the experiment results of the case study, first for
406 heating energy as performance indicator. In the top row of Fig.6, for example, one can observe
407 that the higher the window to wall ratio, the larger the distribution of results but also the lower
408 the minimum values are. This is the opposite for the vertical and horizontal insulation as well as
409 the thermal resistance of the windows, where the higher the thermal resistance, the smaller the
410 distribution and the heating energy consumption are. The lowering of the g-value of the glazing,
411 as depicted in Fig.7, results in a decrease of solar gains which causes an increase in heating
412 energy consumption. The type of shading control, on the other hand, has little influence on the
413 heating energy consumption.
414 Similar to the heating energy demand, the cooling energy demand is influenced by different
415 factors. In the bottom row of Fig. 6 one can observe that for each factor level of window to wall
416 ratio (RO) there are solutions with zero or near-zero cooling energy consumption. The
417 variability of the solution space however drastically increases. The higher the window to wall
418 ratio, the larger the potential solution space and the bigger the difference between best case and
419 worst case. As this is related to the amount of solar gains received this also holds true for the
420 factor g-value of the glazing; the higher the g-value (WG) and thus the larger the solar gains, the
421 higher the variability of solutions.
422 Concerning the shading control (ST), one can observe that for heating energy the lowering of
423 the threshold to activate the shading has nearly no effect whereas, not surprisingly, for cooling
424 energy a strong effect on the distribution of results can be observed. Especially when the
425 threshold is so high that the shading is never activated, the results show a large distribution,
426 meaning that there are high performing solutions with very responsive external shading as well
427 as without shading.

12
428 4.3.2 Factor Effects
429 In addition to the distribution and variance of the results, the main effect of the factors Ef on the
430 performance indicators can be calculated using the following equation [20]:
431 (2)

432 where represents the average response of the factor at level 4 and the average
433 response at level 1. An effects plot (Fig. 8) is used to display importance and magnitude of
434 effects of single and combined factors, which have been evaluated against the maximum
435 variation, as described above, and ranked according to their strength.

436 4.3.2.1 Interpretation of the Results

437 For the case study, the thermal resistance of the horizontal (IH), vertical (IN) and glazed facade
438 (WU) has the strongest effect on reducing heating energy demand, followed by combinations of
439 these factors. The strongest effect on increasing heating energy demand is caused by the g-value
440 (WG) of the glazed facades, followed by combined factors involving the window to wall ratio
441 (RO), g-value (WG) and insulation properties (IN, IH).
442

443
444 Fig 8: Plot of absolute effects (reduction/increase in kWh/m2a) for heating (left) and cooling energy demand (right);

445 For cooling energy consumption, all factors that control the solar gains (primary and secondary
446 factors) have the strongest effect on reducing cooling energy demand. The thermal resistance of
447 the opaque and glazed facades has only little effect. Consequently, the combination of the
448 windows’ g-value with the shading threshold show the strongest effect on increasing cooling
449 energy demand.

450 4.3.3 Factor Interactions

451 In addition to identifying important effects and determining their magnitude, the interactions
452 between effects are crucial. Interactions occur when the effect of a factor is dependent on the
453 level of another factor. A design measure always addresses multiple factors. Understanding how
454 these factors interact in which magnitude allows for choosing the best combination of measures
455 by revealing factor combinations with cumulative or degrading effect. The interaction IA,B
456 between two factors A and B can be calculated using the equation [20]

457 (3)

13
458 where is the effect of factor A if factor B is at level 4 and is the effect of factor A
459 if factor B at level 1. The interaction plot (Fig.9) allows for easily identifying interactions
460 between two factors. It plots the mean response of two factors for all occurring combinations. If
461 the lines are parallel, no interaction is present, non-parallel and intersecting lines indicate that
462 there is an interaction between the factors. The more non-parallel the lines are the stronger the
463 interaction is.

464 4.3.3.1 Interpretation of the Results

465 Using the interaction plot, the factor interactions of the case study can be interpreted (Fig.9).
466 For heating energy, the main factor interactions can be observed between the window to wall
467 ratio (RO) and the thermal properties of vertical opaque (IN) and transparent (WU, WG)
468 surfaces. The levels of those factors should therefore be chosen carefully and taking the
469 interdependencies into account. Other parameters show no or only very little interactions.
470

471
472 Fig 9: Interaction plot for heating energy demand (kWh/m2a) (Ratio of Openings (RO), Vertical Insulation (IV),
473 Horizontal Insulation (IH), Solar Gain (WG), Window Insulation (WU), Shading Control (SC))

474 For cooling energy demand, the strongest interactions can be found between the window to wall
475 ratio (RO) and the window g-value and the shading threshold, respectively. Both findings
476 substantiate but quantify a known relation: as the surface where heat gain/heat losses occurs
477 increases or decreases, the thermal properties of the surface become more or less important in
478 relation to the energy demand.

479 4.4 Identification of Retrofit Design Strategies

480 Based on the results, effect and interactions, we identify different exemplary design strategies
481 that perform above average and that each represent an exemplary retrofit approach for the given
482 case study. A strategy is defined by a set of measures. As described in 4.2, measures are
483 represented by factor settings. Table 2 below compares the base case (strategy A), the executed
484 retrofit (strategy B) and three alternative strategies regarding their factor levels and resulting
485 energy performance.

14
486
487 Table 2: Factor levels for design strategies

488 The retrofit strategy actually executed (B) intends to improve the envelope by exchanging the
489 existing glazing with double layer glazing and adding insulation on the vertical surfaces. This
490 way the existing facade and therefore the historic appearance can be maintained. Elements not
491 visible from the street level such as the roof are fully insulated. Even though glazing with a very
492 low g-value is used, the manual shading is kept.
493 The maximum reduction strategy (C), as the title implies, targets maximum energy savings.
494 Using the optimal factor levels, heating energy demand can be reduced by 75% as compared to
495 the base case. This strategy however means high efforts on the building envelope, including the
496 exchange of the existing aluminum facade by a new, highly insulated and triple glazed one,
497 featuring a window to wall ratio of 90%.
498 The factor settings of strategy (D) represent the retrofit measures realized, however with
499 alternative glazing. Different from the actual retrofit strategy it uses standard glazing with high
500 g-value that allows for higher solar energy transmittance and higher passive solar heat gains.
501 These heat gains are controlled using more reactive shading devices at the exterior. This results
502 in a reduction of heating energy demand of 42% compared to the base case before the retrofit.
503 For the façade conservation strategy (E) the façade remains untouched; only maximum thermal
504 insulation of the roof is applied. The heat losses through the badly insulated façade can to some
505 extent be counterbalanced by the high thermal gains through the openings. The existing manual
506 shading systems can control solar heat gains in summer. In practice this solution, however
507 might lead to uncomfortable states because of overheating, as this was observed for the case
508 study building. Already with this minimal measure, the heating energy demand can be lowered
509 by 30% compared to the base case.

510 5 Discussion
511 This work presents a methodology linking BIM and DoE to provide a better understanding of
512 the impact and interactions of architectural and technical design factors on building
513 performance. The methodology is integrated into the Design Performance Viewer toolset, which
514 allows establishing a workflow from a Building Information Model (BIM) to parametric
515 simulation, results collection and statistical analysis.
516 Applied on the actual case study retrofit, we conducted experiments regarding the building
517 envelope, analyzing and comparing different experimental plans. The results allow the

15
518 identification of the main effects, corresponding factor level settings and their interactions. The
519 choice of an appropriate experiment plan allows the reduction of the amount required
520 experiments, which is especially crucial for computationally intensive simulations. We show
521 that a small subset of 3% of the possible combination in the design space leads to results with
522 less than 15% deviation as compared to the full factorial of the given case. The experiment plan
523 identified is certainly specific to the case study. For statements on optimal experiments plans for
524 groups or typologies of buildings, a larger number of cases would need to be studied.
525 Nonetheless, we expect a similar performance of experiment plans for retrofit cases with similar
526 design factor structures.
527 We introduced the factor scatterplot as a visualization technique that, in its digital form, allows
528 interactive filtering, solution tracing in different combinations of factors and performance
529 indicators and further statistical analyses, such as linear regressions for the best solutions per
530 level. The plot represents an effective means to identify the distribution and variability of
531 results, impact of factors, tendencies, dependencies, factor combinations and thus the
532 identification of well-performing strategies. The impact of design decisions can thus not only be
533 observed qualitatively but also quantified in terms of the impact and interdependencies of
534 levels, allowing for a better understanding of the energetic behavior of the building and
535 extending the flexibility in design. The designer can choose a solution (i.e. a dot) in the
536 scatterplot and immediately observe the related factor levels, which correspond to different
537 measures on the building envelope. As we can demonstrate on the case study, the methodology
538 allows for the identification of differing retrofit strategies that equally yield good performance
539 but stress different architectural or technical design factors.
540 With this workflow, we tackle the complexity that decision makers are facing if they strive to
541 deliver energy-efficient and sustainable building designs. There is neither one solution nor one
542 key factor nor is a factor itself positive or negative without its context. The interdependency of
543 factors opens a design space with large combinatorics that the decision maker often cannot
544 overview efficiently. In this situation, we encourage decision makers to use BIM-based DoE as
545 a tool for examining the design space efficiently. The result of such an examination and its
546 visualization provides the decision maker with vital information to tackle the complexity and to
547 understand, which factors are influential, which factors interact and what configurations
548 perform well. Seeing the design space in diagrams and being able to interact with these
549 diagrams, e.g., by selecting well performing configurations and learning how factors are
550 configured, allows the designer to experience the design space with its dependencies between
551 factors and performance. This information helps decision makers to identify beneficial
552 strategies for retrofitting that go beyond that what can be manually identified, as we have
553 demonstrated for the case. This offers a substantial potential to reduce environmental impact
554 and costs by efficient retrofit while maintaining flexibility in design.
555 The proposed workflow requires BIM data as basis, which might not always be available.
556 However, as BIM as a design environment is increasingly applied in practice for both new
557 construction and retrofit of buildings, we believe that the proposed workflow is feasible for the
558 informed and technically apt designer. As the workflow can be automated, we imagine that this
559 could become a feature of the Design Performance Viewer toolset, which has successfully been
560 applied in research practice. As for any energy simulation, the designer however needs
561 sufficient knowledge on energy performance and related design parameters.

562 6 Conclusion
563 The approach presented in this paper suggests using a set of simulations and DoE as a
564 methodology to build up an understanding of the impact of technical and architectural design
565 factors rather than relying on the numerical outputs alone. Dynamic building energy simulation
566 is known to be very sensitive to specific parameter input. Normally, especially in practice, only

16
567 a single or a very few simulations are executed to assess the effectiveness of retrofit measures.
568 This bears the risk of neglecting crucial parameters or setting their values to inappropriate
569 levels, thus distorting the results and leading to wrong conclusions. Employing many
570 simulations using ranges of input values instead of just a single input value better depicts the
571 buildings’ energy behavior within a bandwidth of performance, which allows trends to become
572 more visible. Increasingly available computing power and availability of parallel computing,
573 such as employed for this work, puts the application of DoE experiment plans into perspective
574 as it is increasingly easy and fast to run large numbers of simulations in parallel at low cost.
575 Nevertheless, the smaller the amount of necessary experiment runs to obtain the results, the
576 more likely a real-time application and thus its employment in the design process is. The choice
577 of a small experiment plan thus has both advantages and drawbacks. It provides similar
578 performance results with acceptable error, equally provides information on effects and
579 interactions with much less computational time, however also reduces the variety of available
580 design solutions for the designer to choose from.
581 As an extension of this work, the experiments could be set up more fine-grained, for example
582 focusing on single thermal zones and their interactions. This could reveal how optimal settings
583 for one thermal zone/orientation interacts with the settings of the adjacent zones with different
584 use and orientation. Adding investment or lifecycle costs as a performance indicator would add
585 an additional dimension and benefit for the application of DoE as a methodology without
586 compromising the ease of application.

587 7 Acknowledgements
588 The authors would like to thank Daren Thomas for supporting the implementation of the DoE
589 workflow.

590 References
591 [1] O. Lucon, D. Ürge-Vorsatz, A. Zain Ahmed, H. Akbari, P. Bertoldi, L. F. Cabeza, N. Eyre, A. Gadgil, L. D. D. Harvey, Y.
592 Jiang, E. Liphoto, S. Mirasgedis, S. Murakami, J. Parikh, C. Pyke, and M. V. Vilariño, “Buildings,” in Climate Change
593 2014: Mitigation of Climate Change. Contribution of Working Group III to the Fifth Assessment Report of the
594 Intergovernmental Panel on Climate Change [Edenhofer, O., R. Pichs-Madruga, Y. Sokona, E. Farahani, S. Kadner, K.
595 Seyboth, A. Adler, I. Baum, S. Brunner, P. Eickemeier, B. Kriemann, J. Savolainen, S. Schlömer, C. von Stechow, T. Zwickel
596 and J.C. Minx (eds.)]. Cambridge University Press, Cambridge, United Kingdom and New York, NY, USA., 2014.
597 [2] “Buildings - Energy - European Commission,” Energy. [Online]. Available: https://ec.europa.eu/energy/en/topics/energy-
598 efficiency/buildings. [Accessed: 04-Dec-2015].
599 [3] M. Jakob, G. Martius, G. Catenazzi, and H. Berleth, “Energetische Erneuerungsraten im Gebäudebereich - Synthesebericht
600 zu Gebäudehülle und Heizanlagen,” Bundesamt für Energie (BFE), Feb. 2014.
601 [4] A. Wiencke and E. Meins, “Praxisbeitrag. Energieforschung Stadt Zürich. Bericht Nr. 5, Forschungsprojekt FP-2.2.2.”
602 [5] J. S. Gero, N. D’Cruz, and A. D. Radford, “Energy in context: a multicriteria model for building design,” Building and
603 Environment, vol. 18, no. 3, pp. 99–107, 1983.
604 [6] K. Papamichael, J. LaPorta, and H. Chauvet, “Building Design Advisor: automated integration of multiple simulation tools,”
605 Automation in construction, vol. 6, no. 4, pp. 341–352, 1997.
606 [7] L. G. Caldas and L. K. Norford, “A design optimization tool based on a genetic algorithm,” Automation in construction, vol.
607 11, no. 2, pp. 173–184, 2002.
608 [8] P. H. Janssen, “An evolutionary system for design exploration,” presented at the Proceedings of the International
609 Conference on Computer Aided Architectural Design Futures, Montréal, Canada17th-19th June, 2009, pp. 260–272.
610 [9] M. Turrin, P. von Buelow, R. Stouffs, and A. Kilian, “Performance-oriented design of large passive solar roofs,” presented
611 at the Future Cities: ECAADE 2010: Proceedings of the 28th Conference on Education in Computer Aided Architectural
612 Design in Europe, September 15-18, 2010, Zurich, Switzerland, ETH Zurich, 2010, p. 321.
613 [10] P. Heiselberg, H. Brohus, A. Hesselholt, H. Rasmussen, E. Seinre, and S. Thomas, “Application of sensitivity analysis in
614 design of sustainable buildings,” Renewable Energy, vol. 34, no. 9, pp. 2030–2036, 2009.
615 [11] H. Breesch and A. Janssens, “Performance evaluation of passive cooling in office buildings based on uncertainty and
616 sensitivity analysis,” Solar energy, vol. 84, no. 8, pp. 1453–1467, 2010.
617 [12] W. Wang, R. Zmeureanu, and H. Rivard, “Applying multi-objective genetic algorithms in green building design
618 optimization,” Building and Environment, vol. 40, no. 11, pp. 1512–1525, Nov. 2005.
619 [13] K. W. Chen, A. Schlueter, and P. Janssen, “A Design Method for Multicriteria Optimization of Low Exergy Architecture,”
620 presented at the Open Systems: Proceedings of the 18th International Conference of the Association of Computer-Aided
621 Architectural Design Research in Asia CAADRIA, Hong Kong, 2013.
622 [14] S. Attia, M. Hamdy, W. O’Brien, and S. Carlucci, “Assessing gaps and needs for integrating building performance
623 optimization tools in net zero energy buildings design,” Energy and Buildings, vol. 60, pp. 110–124, May 2013.
624 [15] W. Y. Fowlkes and C. M. Creveling, Engineering methods for robust product design. Addison-Wesley, 1995.

17
625 [16] Z. R. Lazic, Design of experiments in chemical engineering: a practical guide. John Wiley & Sons, 2006.
626 [17] D. R. Kuhn and M. J. Reilly, “An investigation of the applicability of design of experiments to software testing,” presented
627 at the Software Engineering Workshop, 2002. Proceedings. 27th Annual NASA Goddard/IEEE, 2002, pp. 91–95.
628 [18] K. Siebertz, D. T. van Bebber, and T. Hochkirchen, Statistische Versuchsplanung: Design of Experiments (DoE). Springer-
629 Verlag, 2010.
630 [19] H. J. Seltman, “Experimental design and analysis,” Online at: http://www. stat. cmu. edu/, hseltman/309/Book/Book. pdf,
631 2012.
632 [20] J. Antony, Design of experiments for engineers and scientists. Elsevier, 2014.
633 [21] F. Chlela, A. Husaunndee, C. Inard, and P. Riederer, “A new methodology for the design of low energy buildings,” Energy
634 and Buildings, vol. 41, no. 9, pp. 982–990, 2009.
635 [22] I. Jaffal, C. Inard, and C. Ghiaus, “Fast method to predict building heating demand based on the design of experiments,”
636 Energy and Buildings, vol. 41, no. 6, pp. 669–677, 2009.
637 [23] A. N. Sadeghifam, S. M. Zahraee, M. M. Meynagh, and I. Kiani, “Combined use of design of experiment and dynamic
638 building simulation in assessment of energy efficiency in tropical residential buildings,” Energy and Buildings, vol. 86, pp.
639 525–533, Jan. 2015.
640 [24] Z. Romani, A. Draoui, and F. Allard, “Metamodeling the heating and cooling energy needs and simultaneous building
641 envelope optimization for low energy building design in Morocco,” Energy and Buildings, vol. 102, pp. 139–148, Sep.
642 2015.
643 [25] A. Schlueter and F. Thesseling, “Building information model based energy/exergy performance assessment in early design
644 stages,” Automation in Construction, vol. 18, no. 2, pp. 153–163, Mar. 2009.
645 [26] C. Miller, D. Thomas, S. D. Irigoyen, C. Hersberger, Z. Nagy, D. Rossi, and A. Schlueter, “BIM-extracted EnergyPlus
646 model calibration for retrofit analysis of a historically listed building in Switzerland,” in Proceedings of SimBuild 2014:
647 2014 ASHRAE/IBPSA-USA Building Simulation Conference, 2014, pp. 331–338.
648 [27] D. Thomas, C. Miller, J. Kaempf, and A. Schlueter, “Multiscale co-simulation of EnergyPlus and CitySim models derived
649 from a building information model,” in Proceedings of Bausim 2014: Fifth German-Austrian IBPSA Conference, 2014, pp.
650 469–476.
651 [28] D. Thomas and A. Schlueter, “Customizable Continuous Building Simulation using the Design Performance Toolkit and
652 Kepler Scientific Workflows,” presented at the 9th European Conference of Product and Process Modeling (ECPPM),
653 Reykjavic, Iceland, 2012.
654 [29] H. Zhao, Z. Nagy, D. Thomas, and A. Schlueter, “Service-oriented Architecture for Data Exchange between A Building
655 Information Model and A Building Energy Model,” in Proceedings of CISBAT 2015, 2015, pp. 761–766.
656 [30] A. Schlueter and F. Thesseling, “Facilitating Environmental Performance Assessment in Architectural Design Competitions
657 utilizing a Model-based Workflow,” in eWork and eBusiness in Architecture, Engineering and Construction: ECPPM 2012,
658 Reykjavic, Iceland, 2012, pp. 181–186.
659 [31] A. Schlueter and F. Thesseling, “Balancing Design and Performance in Building Retrofitting – a Case Study Based on
660 Parametric Modeling,” presented at the ACADIA 2008 - Silkon and Skin, 2008.
661 [32] “revitpythonshell (github).” [Online]. Available: https://github.com/architecture-building-systems/revitpythonshell.
662 [Accessed: 01-Dec-2015].
663 [33] “R: The R Project for Statistical Computing.” [Online]. Available: https://www.r-project.org/. [Accessed: 08-Dec-2015].
664 [34] “sia| schweizerischer ingenieur- und architektenverein,” sia | schweizerischer ingenieur- und architektenverein. [Online].
665 Available: http://www.sia.ch/en/the-sia/. [Accessed: 08-Dec-2015].
666 [35] “Testing and Validation | EnergyPlus.” [Online]. Available: https://energyplus.net/testing. [Accessed: 08-Dec-2015].
667 [36] “Langjährige Mittelwerte - HEV Schweiz.” [Online]. Available: http://www.hev-schweiz.ch/vermieten/heiz-und-
668 nebenkosten/heizgradtage/langjaehrige-mittelwerte/. [Accessed: 03-Jan-2016].
669 [37] W. F. Kuhfeld, “Marketing research methods in SAS,” Experimental Design, Choice, Conjoint, and Graphical Techniques.
670 Cary, NC, SAS-Institute TS-722, 2005.
671 [38] R. K. Meyer and C. J. Nachtsheim, “The coordinate-exchange algorithm for constructing exact optimal experimental
672 designs,” Technometrics, vol. 37, no. 1, pp. 60–69, 1995.
673 [39] S. L. C. Ferreira, R. E. Bruns, H. S. Ferreira, G. D. Matos, J. M. David, G. C. Brandão, E. G. P. da Silva, L. A. Portugal, P.
674 S. dos Reis, A. S. Souza, and W. N. L. dos Santos, “Box-Behnken design: An alternative for the optimization of analytical
675 methods,” Analytica Chimica Acta, vol. 597, no. 2, pp. 179–186, Aug. 2007.
676 [40] V. Garg, K. Chandrasen, J. Mathur, S. Tetali, and A. Jawa, “Development and performance evaluation of a methodology,
677 based on distributed computing, for speeding EnergyPlus simulation,” Journal of Building Performance Simulation, vol. 4,
678 no. 3, pp. 257–270, 2011.
679

18