Extracting Building Characteristics Essential for Building Energy Consumption Predictions: Learning from Façade Images through Deep Learning

Buildings account for 40% of the energy consumption and more than 50% of the greenhouse gas (GHG) emissions worldwide. Various building energy efficiency policies have been issued in many cities around the US, requiring the disclosure of buildings’ energy performance and encourage building owners to take this information into consideration for retrofit investment decisions. However, various types of building information at different scales are meant to be collected and disclosed with costs that have substantial variations accordingly. Hence, complying with these policies is expensive and time-consuming for both government agencies and building owners. This paper proposes a non-intrusive and scalable data-driven approach to automatically capture energy-critical building characteristics to declare energy benchmarking or retrofitting policies. This approach first integrated open city data sets (building-related) and identified principal building variables in relation to energy performance indexes (i.e., electricity usage), then utilized transfer learning to retrain deep learning models on building façade images (retrieved from Google Street View) to extract their corresponding principal building variables. To elaborate, given a fully captured façade image of any building, this approach provides the data values for the identified principal variables (e.g., for a façade image given as an input, the outputs will include buildingType = Office). The best accuracy of the extraction model exceeds 80% in extracting building types from images, which indicates that the model is capable of extracting data values for the principal building variables that are essential for energy performance of buildings.