Installation Quality Framework: Investment Return Approach for Energy Savings on Building Product Installation

The construction and manufacturing industries have evaluated the interconnectedness of time, quality, and cost. The emphasis in project planning and scheduling has been on managing the relationship between time and cost, with an implicit assumption of a fixed level of quality that is seldom explicitly explained (Liberatore and Pollack-Johnson 2013). Cost is typically considered the most important factor in choosing a contractor, but often reduced cost comes with the risk of contractors using inferior materials or unskilled labor (Hu and He 2014). In addition, the failing or shortcoming in performance has been linked to materials and the services related to the building construction (Love 2002). Through continual evaluation, the trade-off among time, quality, and cost has provided insight on the optimum choice of a contractor for the job (Forcada et al. 2017); their skills ultimately ensure standard performance on the job with pristine quality and at the desired cost, while achieving maximum productivity. The effectiveness of the chosen contractor for the job is completely dependent on their acclaimed guarantee of meeting job requirements and producing quality work. The risk in the selection of a contractor has been related to varying issues and has the potential to negatively affect the outcome of the overall project (Cheung et al. 2006; Chen et al. 2014). Because the concept of quality is subjective in nature, the construction industry has struggled to define and quantify quality and value (Sullivan 2011).

In recent decades, there has been an effort to reduce the energy demands of residential and commercial buildings (Clean Energy Solutions Center 2015). To address this challenge, the US DOE Building America program established a research agenda targeting market-relevant strategies to achieve 40% reductions in existing home energy use by 2030. Deep energy retrofits are part of the strategy to meet and exceed this goal (Less and Walker 2014; Brennan and Iain 2015). Another method for reducing energy consumption has been focused on closing the performance gap. The term performance gap is used to denote deviations between a building’s planned and actual performances (Frei et al. 2017). This measure corresponds to an increase in energy use and, therefore, a decrease in energy savings. The discrepancy between actual and theoretical savings is caused (among other factors) by the energy performance gap, which is the discrepancy between the actual and calculated energy consumption of a building. One study illustrates that it is not possible to explain residential energy consumption by solely relying on building simulation models (van den Brom et al. 2019). Closing this gap and moving toward buildings with predictable energy demands will aid in achieving the widespread goal of decreased energy consumption. To move toward this goal, building performance is broken down in terms of the performance of individual products that make up the building and its envelope. Throughout this research, a framework is provided for analyzing the impact of product performance on building performance through an installation-quality perspective.

A performance approach is concerned with matching the description of what a product is required to achieve in terms of functional, technical, and sometimes economic requirements (Almeida et al. 2010). In Product Reliability: Specification and Performance (Murthy et al. 2008), manufacturers design a product with certain performance requirements specified during the product development process. Before production begins, prototypes are designed with the goal of meeting this desired performance, set forth by the manufacturer, which is then modified until the desired performance is reached (Murthy et al. 2008). In terms of whole building performance, the International Energy Agency states in Annex 55 that achieving the expected performance requires that the factors that the design concept is based upon fulfill certain standards and are within expected ranges. These factors could include artisanship, interior and exterior climate, and maintenance and/or material properties (Hagentoft 2017). Although it is recognized there are many factors that can affect performance, this present research focuses heavily on the impact of the craft during installation and not the quality of the products themselves. Furthermore, improper installation can lead to poor performance and energy inefficiency. This discrepancy will be used to develop a return on investment (ROI) to save costs upfront.

In a field study, the airtightness barriers of 12 buildings with identical designs were measured, and the results revealed that the least performing building was more than twice as leaky as the most airtight building (Pallin et al. 2017). In another study of indoor air quality and ventilation systems, it was noted that complex ventilation systems were more prone to installation and performance issues (Less et al. 2015). Because the product performance gap is considered to be the difference between a product’s expected performance and its actual performance, if this manufacturer’s claimed performance is not met once the product is in use, this would be a performance gap on the product level. Within the scope of a building, there are different occurrences identified from product performance gaps, and the present research focused on products that were specifically susceptible to poor performance due to errors that occurred during installation as a result of the craft. Pallin et al. (2017) suggested that a traditional blower door test is used to measure the ACH50 (air change per hour at 50 Pa) of buildings. Such a blower door test represents the building’s overall performance and, consequently, not the airtightness of different building elements, such as roofs, walls, or foundations. Without such distinctions, it is difficult to identify which locations, the connections and/or details, and whether air leakage exists. Using a nontraditional methodology, Pallin et al. (2017) were able to measure the airtightness of building elements by simultaneously measuring the air leakage through roofs, walls, foundations, and their connections individually. This segmental approach is similar to what is being proposed in this paper; more specifically, product installations are examined to capture the individual faults that makeup up overall building inefficiency.

Certain building products rely more on proper craft performed by skilled laborers than others. Heating, ventilation, and air-conditioning (HVAC) systems, heat pumps, and insulation systems are products that are notably susceptible to poor performance due to installation errors. The lack of broader educational content and deficiencies in engineering knowledge will have profound negative impacts on both the performance and market acceptance of heat pumps (Gleeson 2016). A further sensitivity analysis of heat pump installation faults performed by Domanski et al. (2014) researched the impact of installation on performance in five different climatic zones and concluded that installation faults could be responsible for a 30% increase in energy usage. This study was further substantiated by the American Council for Energy Efficient Economy (2019), which claimed that energy savings from high-efficiency air conditioners and heat pumps could be negated by a 20%–40% loss in energy efficiency due to poor installation (“2019 Efficiency Programs: Promoting High Efficiency Residential Air Conditioners and Heat Pumps,” 2019). To raise awareness and promote the prevention of these defects, the Air Conditioner Contractors of America have set forth the HVAC quality installation specification, which states that there is a need to establish a performance bar to improve the core competencies of contractors to ensure that quality installations occur (ACCA 2015). In addition, the performance of other building products and systems are also affected by craft. In research by Langmans et al. (2017), the insulation installed in cavity walls, especially rigid-board-insulated, was highly dependent on the installation quality. Sulakatkoa et al. (2017) reported on the performance of the External Thermal Insulation Composite System with respect to installation and concluded that onsite shortcomings during the construction process led to a loss of technical performance of the product and reduced thermal efficiency.

In a broader sense, the knowledge of contractors and their crews on building retrofit projects affects the quality of the retrofit. In Deep Residential Retrofits in East Tennessee, 10 homeowners were guided through the energy retrofit process. For one particular retrofit experience in this project, the authors stated that for homeowners who are not actively involved in their project or who have little or no knowledge of how to do energy retrofit work correctly, the quality of the work would vary significantly and depend on the knowledge, training, and pride of work of the contractors and construction crews (Boudreaux et al. 2012). The implication of this observation has an impact on the success of the project, as it was also asserted that the quality of the retrofit work would correlate to energy savings (Boudreaux et al. 2012). It is then necessary to ensure high-quality construction to reach the expected energy savings proposed in deep energy retrofit projects.

Zero Carbon Hub (2014) identified poor installation of insulation as a contributor to the building performance gap in Closing the Gap Between Design and As-Built Performance. In their findings, some of the test houses were constructed carefully in a manner that was described as good quality, while others were constructed in such a way as to mimic poor quality. Features associated with poor quality could, in some cases, cause the U-value to rise by as much as 310% (Zero Carbon Hub 2014). Other investigations in which a thermographic test and visual inspection were performed found that the quality of installation of insulation can be very important and that areas, where insulation is poorly fitted, can incur high levels of heat loss (Doran and Carr 2008). From these examples, it is evident that poor quality during installation can be a contributor to poor building performance. This research study by Zero Carbon Hub (2014) also stated that a total of 15 issues had been found to be both supported by strong evidence from multiple sources and likely to have a significant impact on the performance gap, and it was determined that the majority of these issues result from a lack of knowledge and skills. Similarly, it was noted that the feasibility of installation for a product is related to an installer’s ability and training; their training is composed of the knowledge and skills acquired and their use of standard industry practices. In 2009, the DOE announced that approximately $450 million in the Recovery Act Funding would be allocated to weatherization programs in 13 states. Nonetheless, 20% of the funds were planned to be spent on hiring and training workers that would have a positive long-term impact on the economy. In addition, opportunities would be presented to many inexperienced green job workers to serve and do important work for our communities in the market where experience is key (DOE 2009; Soratana and Marriott 2010). The parties involved in the installation process contribute their time and knowledge, which can vary significantly given their time constraints and educational backgrounds; therefore, the building performance becomes sensitive to the product installation process. This framework can be universally applied across multiple products to understand the implications of installation.

This study focuses on the relationship between a person’s skill level and work knowledge and the role they play on installation quality and building energy performance. There are multiple factors that influence job productivity, which adversely affects product installation. In chapter four of Project Management for Construction, labor, material, and equipment used were discussed in terms of factors that affect jobsite productivity (Hendrickson and Au 1989; Smith 2016). Regardless of whether the installer formally gained knowledge in an area or gained it through experience, the overall understanding of information in related areas has an influence on performance in said field.

In the manufacturing industry, product development knowledge is defined by production efficiency. Development knowledge and other intangible properties have become the most important and valuable assets for product development (Wu et al. 2014). The same study revealed that for most manufacturing companies in China, the shortage of knowledgeable workers or experts is one of the biggest gaps standing in the path of the companies’ growth. There is a gap in acquiring knowledge and achieving optimized productivity within the manufacturing industry due to the lack of inclusion of installation perspectives throughout the development of products. In today’s knowledge economy, the primary factor in determining economic development is the worker’s own knowledge and ability (Quintino et al. 2012).

Despite the obvious room for potential savings, there is still a slight implementation of market-based solutions within the residential energy efficient market. This issue was further investigated in Increasing Innovation in Home Energy Efficiency: Monte Carlo Simulation of Potential Improvements, in which the importance of the auditor’s experience was included during the home energy audit process (Soratana and Marriott 2010). In their study, an auditor is defined as an installer, and gathered savings were collected by deducting the costs of improvements during interviews and inspections. These improvements were vital to the proposed reduction energy consumption goals but, more importantly, were strongly based on the auditor’s experience from initial home visits to the decision-making process to determine the feasibility of the improvements. The auditor’s experience factor was named the energy savings threshold and used as one of the variables in the Monte Carlo simulation model. The auditor only installed improvements if the energy savings efficiency exceeded the energy-saving threshold. Soratana and Marriott (2010) found that the residential energy services company (RESCO) model results determined that the energy-saving threshold percentage is a critical factor and depends substantially on an auditor’s experience. Their RESCO-based study concluded that the energy improvement market should rely on technology as well as the qualification of the worker. As evidenced in their results, an experienced auditor can choose improvements better than a tool and aid RESCOs in allocating their improvements effectively. From the understanding that individual products contribute to the overall performance of a building and that the performance is affected by the quality of the installation, a framework for analyzing the impact of installation was developed. The proposed framework emphasizes the effects of installation and its importance for ensuring performance and reducing energy consumption. Inevitably, an optimized performance would result in a decrease in operational cost during the lifetime of the product. The overall goal of this paper is to introduce the probability investment return (PIR) metric that accounts for variations in cost and energy performance for any given product or system. From this metric, consumers can gain an understanding of how the installation quality of a product and its associated energy savings can provide information on the expected payback and its variance. This research explores building system performance variance and highlights optimized installation as a key factor in improving energy performance gaps.

PIR is based on the probabilistic variation of installation and performance aspects of building products, materials, and systems. PIR is found from the inputs displayed in the concept map, which reveals the breakdown of the influencing variables contributing to the definition of PIR (Fig. 1). The methods provided in this framework consider the difficulty of installation, variance in product performance, energy consumption as a result of performance, and a resulting probabilistic investment return. The value of performance can be used toward a probabilistic assessment to estimate the returns on product investment. Performance depends on the trade-off between installers’ knowledge and time spent on installation. The conceptual map of Fig. 1 reflects the process of how each part within the developmental process of the PIR value was considered in the overall framework.

For most products, materials, and systems, assuming a unique installation time for each knowledge level is less than realistic. Instead, a variation for the installation time is expected within each knowledge level. Fig. 4 reveals how the installation cost factors can take shape under a probabilistic viewpoint. In this display, the expected installation time is defined by a normal distribution, and the area under the curve represents a combination of time and knowledge, which together are a representation of the effort required to reach the expected performance (Hughes et al. 2020). As knowledge levels vary, so does the probabilistic range of installation time. This display (Fig. 4) represents the perfect scenario in which a total understanding exists of the installation time required for a product; however, many times, such complete analysis is difficult to conduct. In cases in which time data can be collected with little to no understanding of the installers’ knowledge levels, such data points can still be useful, assuming the installers’ knowledge from the collected data is representative of average levels. Fundamentally, a probability distribution is a function that represents the likelihood of obtaining possible values that a random value can assume. This statistical tool is useful in the investigation of the likely outcomes, potential values, and varied results. An example of how installation times may be presented by this graphical representation was presented in the installation of offshore wind turbines and their distribution (Lacal-Arántegui et al. 2018). If a standard random variable is used to characterize the distribution of activity durations, then only a few parameters are required to calculate the probability of any particular duration (Hendrickson and Au 1989). Fig. 5 illustrates how data sets are expected to look based on previous examples in the literature (Hendrickson and Au 1989; Lacal-Arántegui et al. 2018).

Because there is a correlation between installation time, $t$, and knowledge level, time can be described

The discrete-time distribution in Fig. 5 can be expressed as a continuous probability distribution—in this case a normal distribution. Hence, the installation time, $t$, is defined under the following conditionin whichand the expected mean value ofwhere $t$ = installation time ($\mathrm{h}/\mathrm{unit}$); $\mathrm{K}$ = knowledge level (0–9); ${\sigma }_t$ = standard deviation of time ($\mathrm{h}/\mathrm{unit}$); ${\mu }_t$ = average installation time (h/unit); and $n$ = sample size.

This work also seeks to understand costs associated with installation, that is, the labor cost. To determine the most cost-efficient knowledge level to install a product, the cost range for different knowledge levels must be accounted for, as seen in Fig. 6. Because the knowledge and skills of a laborer are often reflected in the hourly wage, the knowledge level can assumingly represent the hourly wages of the installer.

Depending on the expected performance level of a product, the associated energy savings can be found by a comparison with existing conditions. To the improved performance to which a system/product/material will contribute, the estimated savings, ${\xi }_s$, can be foundwhere ${\xi }_{\mathrm{pre}}$ (joule or watt) is the energy performance before the installation; and ${\xi }_{\mathrm{post}}$ (joule or watt) is the performance afterward. Obviously, the difference between the energy performance values represents the energy savings. Eq. (5) is valid for both deterministic values and/or probabilistic variations of ${\xi }_{\mathrm{pre}}$ and ${\xi }_{\mathrm{post}}$. Fig. 7 illustrates a probabilistic approach and how the expected energy savings can be found from two arbitrary normal distributions—one representing preinstallation and one representing postinstallation.

For probabilistic distributions, as seen in Fig. 7, performance savings can be expressed using Eq. (6)

In cases in which the existing performance is known, meaning ${\xi }_{\mathrm{pre}}$ has an exact value, while the expected performance after the installation is assumed to follow a probabilistic distribution, a combination of Eqs. (5) and (6) applies

Once the improved performance has been found (i.e., the energy savings), the associated cost savings is givenwhere $S$ = time-dependent cost savings ($/year); and $E$ = cost of energy ($/J).

Using Eqs. (6) and (8), the probabilistic energy savings becomes a variable under the normal distribution

The costs of installation, product, and associated energy savings provide a general ROI or payback. The number of years required to receive an ROI for installing a product is defined by the ratio between installation and material cost, $c_{\mathrm{tot}}^i$, over energy savings, $s$. The energy savings, as previously noted, would be dependent on the performance level of the product reachedwhere $c_{tot}^i$ = exact value of installation cost ($); and $s$ = exact value of cost savings ($/year).

However, the focus of the present study is on understanding and appreciating the importance and impact of probabilistic variations. So, a probabilistic payback metric, known as the probabilistic investment return (PIR), is proposed. The PIR represents the expected variation of a product/system/material payback time utilizing Eqs. (8)–(10) combining to formwhere the variables are from the following relationships

Because both the numerator and denominator of the PIR include a probabilistic varying variable, the solution will hold a probabilistic variation as well. The solution for Eq. (11) can be found by using mathematical tools (Palisade 2019) or by using a ratio distribution approximation (Hinkley 1969) for uncorrelated noncentral normal distributionswhere

In Eq. (12), PIR is a variable under the probability density function of $P(t)$—a probability density as a function of time, $t$.

Naturally, neither the probabilistic variation of installation and material cost nor the associated savings always take the form of a normal distribution. In such cases, Eq. (11) can be written to be valid for any arbitrary probabilistic distributionwhere

An arbitrary distribution of PIR is depicted in Fig. 8. In this figure, PIR is a variable under the probabilistic distribution of $f(t)$, in which $t$ is the time in years. Notably, Fig. 8 allows the reader to appreciate how much the ROI can vary for a product and why it is preferable to treat such an assessment in a probabilistic manner. In addition, PIR can be compared with different products, systems, and materials to evaluate the most cost-efficient alternative.

Oak Ridge National Laboratory has studied the performance of installed air barriers in 44 homes built between 2010 and 2016 by measuring the airtightness of each building. Typically, the airtightness of a building is measured by conducting a blower door test. For residential buildings, the airtightness is many times measured as ACH50. This unit stands for air changes per hour at a 50 Pa air pressure gradient. Even if buildings do not typically see this high air pressure gradient, it allows the evaluation of air barrier performance, code compliance, and comparisons between other buildings. The results of the measurements are presented in Fig. 9. Twenty-two of the homes were installed with a traditional house wrap as the air barrier, and 18 homes were constructed with the ZIP System. The two materials and systems are displayed in Fig. 10. The house wrap is typically mounted and fastened on the outside of the sheathing by staples. The ZIP System acts both as the sheathing and weather barrier. The seams between the boards are sealed using a ZIP System approved tape.

The performance of the air barrier is most relevant for the overall energy performance of a building. The financial benefit will be evaluated for installing the ZIP System as an air barrier over a traditional house wrap using the proposed PIR metric. This evaluation would allow for the expected installation cost, $C_{tot}$ and the expected savings, $S$, associated with using the ZIP System over house wrap to be ascertained.

The costs associated with installing a house wrap and a ZIP System are given in Table 1. For house wrap, the cost is estimated using RSMeans data (Gordian 2021) and includes material, labor, and overhead. For the ZIP System, the material cost is approximately twice that of house wrap (JLS Construction 2011; Luig 2020). The labor cost between the two materials and systems are almost identical (Dupont 2021). For the estimated variations in installation cost, the RSMeans City Cost Index was utilized (Gordian 2019). Using the data in Table 1, the probabilistic variation of installation cost between house wrap and the ZIP System can be defined. Using Eq. (6), the difference in cost between the two materials and systems is foundwhere $C_{\mathrm{ZIP}}(\$/{\mathrm{m}}^2)$ = probabilistic variation in cost for the ZIP System; and $C_{HW}$ ($\$/{\mathrm{m}}^2$) = equivalent for house wrap. It was necessary to determine the time needed for the more expensive ZIP System to become financially beneficial in terms of payback, so the cost difference between the two systems, $C_{tot}(\$/{\mathrm{m}}^2)$, was now defined.

Estimation of the performance difference between a house wrap and the ZIP System based on the conducted blower door tests in the 44 homes was then conducted. Measured data given in Fig. 9 was employed as input data for whole-building energy performance simulations. One of the prototype buildings given by the Department of Energy (EnergyCodes 2020) was used for analysis, which was a small office building, and it is very similar in construction to a residential building. The house was simulated in Climate zone 4A (Baltimore), with a thermal characteristics meeting code (ANSI/ASHRAE/IESNA 2016). The coefficient of performance of the heating and cooling system was set to 0.9 and 3.5, respectively. Further, the cost of electricity and gas was assumed at $0.11 and $\$0.05\mathrm{per}\mathrm{kW}·\mathrm{h}$.

The variation in energy usage and cost of energy for the simulated buildings is presented in Fig. 11. According to the two probabilistic distributions, the average cost of energy is somewhat lower than that of house wrap. The difference in cost between the two systems will define the savings

It was determined that $3,170 and $458 are the average annual cost of energy and the standard deviation for house wrap, while $2,840 and $429 are the average annual cost and standard deviation for the ZIP System.

Finally, the PIR can be calculated, using Eq. (13)

Utilizing Eq. (12), the following was foundwhere $IG$ = inverse gaussian distribution, which is also depicted in Fig. 12. According to the PIR value from Eq. (17), it takes on average 12.9 years for the ZIP System to outperform house wrap in terms of return of investment but with a significant spread in variance. Fig. 12 only presents the probabilistic distribution for which installing the ZIP System over a house wrap is favorable. Because the two distribution curves in Fig. 11 overlap, it is understood that the value of PIR is not always positive, which signifies that there will be times for which the house wrap outperforms the ZIP System. By utilizing a risk assessment simulation tool (Palisade 2019), the PIR values were estimated in both the positive and negative range. The simulation result is presented in Fig. 13.

By comparing Fig. 12 with Fig. 13, it can be stated that the distribution of PIR in Fig. 12 is also represented in Fig. 13. However, a significant part of the expected variation of PIR in the negative range can also be seen, which represents the portion for which house wrap outperforms the ZIP System.

The present research provided a methodology to quantify the impact of installation quality on overall energy performance and cost. To accomplish this, the PIR metric was introduced, which accounts for the probabilistic nature of installation time, knowledge, performance, cost, and energy savings. The research also presented an approach and framework to visually describe the correlation between installation time and installer knowledge level.

PIR represents the relationship between the required installation time of a product, system, or material and an installer’s level of expertise in relation to the expected variation of performance. This metric allows the user to appreciate how much the installation time, and thus cost, can vary depending on the skill set of the installer and how the difficulty of installation can be compared relative to other products. PIR is basically an approach to evaluate the ROI in a probabilistic manner. Such assessment can then be considered a framework for a holistic understanding of the product installation process and its far-reaching effects on building performance.

A case study compared the estimated energy performance for two air barrier systems: a traditional house wrap and the ZIP System. Both air barrier systems had similar labor costs, but choosing the option that had higher material/overhead costs led to an investment return that will continue to pay for itself. This may be intuitive but is often not done due to cheap labor regardless of another system that could be more energy efficient. This example showcased the obvious choice but also introduced how a probabilistic relationship could sway what is actually utilized during construction. Most notably, a negative PIR value represented what would be cost-effective and energy-efficient upfront, especially when comparing two systems or materials that have similar labor costs and total overhead and profit (O&P) almost doubling in comparison.

Considering the PIR metric during the planning stages of building construction would be a major step toward addressing the challenges of building energy inefficiencies. Designers would have a better understanding of how counteracting compromised airtightness practices with increased budgets on installation quality could result in significant energy savings. The PIR metric could be used by designers to carry out a cost-benefit analysis and help them understand how installation quality impacts energy performance prior to construction so that more informed decisions can be made regarding products and contractors/installers.

All data, models, and code generated or used during the study appear in the published article.

The authors would like to thank Oak Ridge National Laboratories for the hosting and implementation of this project work. We would also like to express gratitude to the Program of Excellence in STEM (PE-STEM) at Florida A&M University supported by the US Department of Education’s MSEIP Program, Grant No. P120A160115, and the GAANN Program in Civil Engineering at Florida A&M University (Grant No. P200A180074) for sponsorship of the lead author.