Abstract
Distributed generation (DG) refers to small-scale generation resources that are located at or near end-consumers, such as photovoltaic (PV) solar systems. DG systems have become increasingly popular in recent years owing to their economic efficiency, reliability, and sustainability. However, the increasing adoption of DG is creating new obstacles for system operators due to the uncertainty in forecasting future demand. One concern is the possibility of facing a utility death spiral as a feedback loop between, on the one hand, the increasing adoption of DG and increasing electricity rates to cover generation and transmission overheads with, on the other hand, reduced demand from the grid. The goal of this study was to investigate the effects of the penetration of DG on the power infrastructure and wholesale power markets considering the dynamic pricing of electric power. To achieve that goal, a complex system-of-systems (SoS) simulation for wholesale power markets and infrastructure is developed using agent-based modeling (ABM), Optimal Power Flow (OPF), and reinforcement learning (RL). The simulation framework is enabled by (1) consumer behavior that compares the benefits of installing DG versus the costs of conventional power from the grid; and (2) the dynamic pricing of electric power. Several RL algorithms are compared, including basic RL, multiplicative RL, Roth-Erev RL, a modified Roth-Erev, and a variation of the aforementioned algorithms using a Gibbs-Boltzmann cooling factor by means of a grid of learning parameters relevant to each technique. The results show that (1) low-cost generators, such as nuclear power plants, are the least affected by the penetration of DG, while high-cost generators are the most affected, (2) a utility death spiral is unlikely to occur, and (3) a modified Roth-Erev RL algorithm can be used by generating companies to maximize their gross profits by optimizing their supply curves considering the feedback effect on DG adoption rates. Overall, the proposed simulation framework can assist policymakers and future researchers in studying the interaction between dynamic pricing in wholesale power markets and the adoption of DG. By simulating the long-term effect of DG on the electric power infrastructure and market, policymakers can introduce regulations and incentives to strategically influence the rate of adoption of DG.
Practical Applications
DG using PV systems has been growing in recent years. However, there are concerns that the increasing penetration of DG would start a feedback loop where the lower demand for power from the grid would push utilities and generating companies to increase electricity rates to maintain profits and cover overhead, which can again motivate consumers to install DG systems. To investigate that feedback loop, this study developed a model that simulates the interacting effect of consumer behavior to adopt DG and dynamic pricing by utility companies. The results show that low-cost generators, such as nuclear plants, would be the least affected. The results also show that the best economic decision for generating companies may be to keep electricity rates unchanged to avoid accelerating DG penetration. The framework presented in this research can be used to create simulations of wholesale power markets affected by DG, which can guide decisions related to DG regulations, consumer incentives, and future grid expansion plans.
Get full access to this article
View all available purchase options and get full access to this article.
Data Availability Statement
All data, models, and code generated or used during the study appear in the published article.
Acknowledgments
This manuscript is based on work supported by the National Science Foundation under Grant No. 1901740. It is worth noting, though, that any opinions, findings, and conclusions or recommendations expressed in this paper are those of the authors and do not necessarily reflect the views of the National Science Foundation.
References
Abraham, Y. S., C. J. Anumba, and S. Asadi. 2018. Exploring agent-based modeling approaches for human-centered energy consumption prediction, 368–378. Reston, VA: ASCE.
Ahmed, M. O., I. H. El-adaway, K. T. Coatney, and M. S. Eid. 2016. “Construction bidding and the winner’s curse: Game theory approach.” J. Constr. Eng. Manage. 142 (2): 04015076. https://doi.org/10.1061/(ASCE)CO.1943-7862.0001058.
Ali, G. G., and I. H. El-adaway. 2020. “Relationship between electric-power sector development and socioeconomic parameters: Statistical analysis approach.” J. Energy Eng. 146 (5): 04020045. https://doi.org/10.1061/(ASCE)EY.1943-7897.0000690.
Arora, S., E. Hazan, and S. Kale. 2012. “The multiplicative weights update method: A meta-algorithm and applications.” Theory Comput. 8 (1): 121–164. https://doi.org/10.4086/toc.2012.v008a006.
ASCE. 2021. Infrastructure report card 2021: Energy. Reston, VA: ASCE.
Assaad, R., M. O. Ahmed, I. H. El-adaway, A. Elsayegh, and V. S. Siddhardh Nadendla. 2021. “Comparing the impact of learning in bidding decision-making processes using algorithmic game theory.” J. Manage. Eng. 37 (1): 04020099. https://doi.org/10.1061/(ASCE)ME.1943-5479.0000867.
Azar, E., and C. C. Menassa. 2012. “Agent-based modeling of occupants and their impact on energy use in commercial buildings.” J. Comput. Civ. Eng. 26 (4): 506–518. https://doi.org/10.1061/(ASCE)CP.1943-5487.0000158.
Barbose, G., N. Darghouth, S. Elmallah, S. Forrester, S. H. Kristina, D. Millstein, J. Rand, W. Cotton, S. Sherwood, and E. O’Shaughnessy. 2019. Tracking the sun: Pricing and design trends for distributed photovoltaic systems in the United States. Berkeley, CA: Lawrence Berkeley National Laboratory.
BNEF (Bloomberg New Energy Finance). 2019. A behind the scenes take on lithium-ion battery prices. New York: BNEF.
Burke, P. J., and A. Abayasekara. 2018. “The price elasticity of electricity demand in the United States: A three-dimensional analysis.” Energy J. 39 (2): 123–145. https://doi.org/10.5547/01956574.39.2.pbur.
Castaneda, M., M. Jimenez, S. Zapata, C. J. Franco, and I. Dyner. 2017. “Myths and facts of the utility death spiral.” Energy Policy 110 (Nov): 105–116. https://doi.org/10.1016/j.enpol.2017.07.063.
Chen, Y., M. Tanaka, and R. Takashima. 2021. “Energy expenditure incidence in the presence of prosumers: Can a fixed charge lead us to the promised land.” IEEE Trans. Power Syst. 37 (2): 1591–1600. https://doi.org/10.1109/TPWRS.2021.3104770.
Clements, J. 2019. “Solar panel lifespan guide: How long do they last?” Accessed July 2, 2021. https://greencoast.org/solar-panel-lifespan/.
Crago, C. L., and I. Chernyakhovskiy. 2017. “Are policy incentives for solar power effective? Evidence from residential installations in the Northeast.” J. Environ. Econ. Manage. 81 (Jan): 132–151. https://doi.org/10.1016/j.jeem.2016.09.008.
Denning, L. 2013. “Lights flicker for utilities.” Wall Street Journal, December 22, 2013.
DOE. 2021. “Solar energy glossary.” Accessed July 2, 2021. https://www.energy.gov/eere/solar/solar-energy-glossary.
Dong, C., and B. Sigrin. 2019. “Using willingness to pay to forecast the adoption of solar photovoltaics: A ‘parameterization+ calibration’ approach.” Energy Policy 129 (Jun): 100–110. https://doi.org/10.1016/j.enpol.2019.02.017.
Driesen, J., and F. Katiraei. 2008. “Design for distributed energy resources.” IEEE Power Energy Mag. 6 (3): 30–40. https://doi.org/10.1109/MPE.2008.918703.
EIA (Energy Information Administration). 2015. “EIA electricity data now include estimated small-scale solar PV capacity and generation.” Accessed February 11, 2021. https://www.eia.gov/todayinenergy/detail.php?id=23972.
EIA (Energy Information Administration). 2018. “Solar photovoltaic costs are declining, but estimates vary across sources—Today in Energy.” Accessed July 2, 2021. https://www.eia.gov/todayinenergy/detail.php?id=35432.
EIA (Energy Information Administration). 2020. “Electricity explained: Electricity in the United States.” Accessed April 18, 2020. https://www.eia.gov/energyexplained/electricity/electricity-in-the-us.php.
EIA (Energy Information Administration). 2021. “Use of electricity.” Accessed March 28, 2021. https://www.eia.gov/energyexplained/electricity/use-of-electricity.php.
Eid, C., E. Koliou, M. Valles, J. Reneses, and R. Hakvoort. 2016. “Time-based pricing and electricity demand response: Existing barriers and next steps.” Util. Policy 40 (Jun): 15–25. https://doi.org/10.1016/j.jup.2016.04.001.
Eid, M. S., and I. H. El-adaway. 2017. “Sustainable disaster recovery: Multiagent-based model for integrating environmental vulnerability into decision-making processes of the associated stakeholders.” J. Urban Plann. Dev. 143 (1): 04016022. https://doi.org/10.1061/(ASCE)UP.1943-5444.0000349.
El-adaway, I., C. Sims, M. Eid, and Y. Liu. 2019. Multi-agent based model for studying electric grid transition to distributed energy resources. Reston, VA: ASCE.
Erev, I., and A. E. Roth. 1998. “Predicting how people play games: Reinforcement learning in experimental games with unique, mixed strategy equilibria.” Am. Econ. Rev. 88 (4): 848–881.
Farajbakhsh Mamaghani, F., and M. Çakanyıldırım. 2021. Metin, harvesting solar power foments prices in a vicious cycle: Breaking the cycle with price mechanisms. Amsterdam, Netherlands: Elsevier.
Felder, F. A., and R. Athawale. 2014. “The life and death of the utility death spiral.” Electr. J. 27 (6): 9–16. https://doi.org/10.1016/j.tej.2014.06.008.
Feldman, D. J., and P. D. Schwabe. 2018. Terms, trends, and insights on PV project finance in the United States, 2018. Golden, CO: National Renewable Energy Laboratory.
Goldfarb, D., and A. Idnani. 1983. “A numerically stable dual method for solving strictly convex quadratic programs.” Math. Program. 27 (1): 1–33. https://doi.org/10.1007/BF02591962.
González, D. M. L., and J. G. Rendon. 2022. “Opportunities and challenges of mainstreaming distributed energy resources towards the transition to more efficient and resilient energy markets.” Renewable Sustainable Energy Rev. 157 (Apr): 112018. https://doi.org/10.1016/j.rser.2021.112018.
Hagberg, A., P. Swart, and S. D. Chult. 2008. Exploring network structure, dynamics, and function using Networkx. Los Alamos, NM: Los Alamos National Laboratory.
Haruna, H., S. Itoh, T. Horiba, E. Seki, and K. Kohno. 2011. “Large-format lithium-ion batteries for electric power storage.” J. Power Sources 196 (16): 7002–7005. https://doi.org/10.1016/j.jpowsour.2010.10.045.
Hunter, J. D. 2007. “Matplotlib: A 2D graphics environment.” Comput. Sci. Eng. 9 (3): 90. https://doi.org/10.1109/MCSE.2007.55.
Hyder, Z. 2021. What is a peak sun hour? What are peak sun hour numbers for your state?. Denver: Solar Reviews.
Khurshaid, T., A. Wadood, S. G. Farkoush, C. H. Kim, N. Cho, and S. B. Rhee. 2019. “Modified particle swarm optimizer as optimization of time dial settings for coordination of directional overcurrent relay.” J. Electr. Eng. Technol. 14 (1): 55–68. https://doi.org/10.1007/s42835-018-00039-z.
Kluyver, T., B. Ragan-Kelley, F. Pérez, B. E. Granger, M. Bussonnier, J. Frederic, K. Kelley, J. B. Hamrick, J. Grout, and S. Corlay. 2016. Jupyter Notebooks—A publishing format for reproducible computational workflows. Amsterdam, Netherlands: IOS Press.
Lam, S. K., A. Pitrou, and S. Seibert. 2015. “Numba: A LLVM-based Python JIT compiler.” In Proc., 2nd Workshop on the LLVM Compiler Infrastructure in HPC, LLVM ’15, 1–6. New York: Association for Computing Machinery.
Lamperti, F., A. Roventini, and A. Sani. 2017. “Agent-based model calibration using machine learning surrogates.” Preprint, submitted March 30, 2017. https://arxiv.org/abs/1703.10639.
Laws, N. D., B. P. Epps, S. O. Peterson, M. S. Laser, and G. K. Wanjiru. 2017. “On the utility death spiral and the impact of utility rate structures on the adoption of residential solar photovoltaics and energy storage.” Appl. Energy 185 (Jan): 627–641. https://doi.org/10.1016/j.apenergy.2016.10.123.
Lijesen, M. G. 2007. “The real-time price elasticity of electricity.” Energy Econ. 29 (2): 249–258. https://doi.org/10.1016/j.eneco.2006.08.008.
Liu, X., E. G. O’Rear, W. E. Tyner, and J. F. Pekny. 2014. “Purchasing vs. leasing: A benefit-cost analysis of residential solar PV panel use in California.” Renewable Energy 66 (Jun): 770–774. https://doi.org/10.1016/j.renene.2014.01.026.
McKinney, W. 2011. “Pandas: A foundational Python library for data analysis and statistics.” Python High Perform. Sci. Comput. 14 (9): 1–9.
Millman, K. J., and M. Aivazis. 2011. “Python for scientists and engineers.” Comput. Sci. Eng. 13 (2): 9–12. https://doi.org/10.1109/MCSE.2011.36.
Mori, S., Y. Makishita, and K. Kamegai. 2017. “Two-stage approach for the assessment of photovoltaic and cogeneration systems: Integration of regional distributed energy systems and power-expansion planning.” J. Energy Eng. 143 (3): F4016005. https://doi.org/10.1061/(ASCE)EY.1943-7897.0000370.
Muaafa, M., I. Adjali, P. Bean, R. Fuentes, S. O. Kimbrough, and F. H. Murphy. 2017. “Can adoption of rooftop solar panels trigger a utility death spiral? A tale of two U.S. cities.” Energy Res. Social Sci. 34 (Dec): 154–162. https://doi.org/10.1016/j.erss.2017.06.041.
Nicolaisen, J., V. Petrov, and L. Tesfatsion. 2001. “Market power and efficiency in a computational electricity market with discriminatory double-auction pricing.” IEEE Trans. Evol. Comput. 5 (5): 504–523. https://doi.org/10.1109/4235.956714.
NREL (National Renewable Energy Laboratory). 2021. “PVWatts calculator.” Accessed July 2, 2021. https://pvwatts.nrel.gov/pvwatts.php.
Oliphant, T. E. 2006. A guide to NumPy. Utah: Trelgol Publishing.
Oliphant, T. E. 2007. “Python for scientific computing.” Comput. Sci. Eng. 9 (3): 10–20. https://doi.org/10.1109/MCSE.2007.58.
Omar, N., Y. Firouz, H. Gualous, J. Salminen, T. Kallio, J. M. Timmermans, T. Coosemans, P. Van den Bossche, and J. Van Mierlo. 2015. “Aging and degradation of lithium-ion batteries.” In Rechargeable lithium batteries, Woodhead Publishing series in energy, edited by A. A. Franco, 263–279. Sawston, Cambridge: Woodhead Publishing.
Pedregosa, F., et al. 2011. “Scikit-learn: Machine learning in Python.” J. Mach. Learn. Res. 12: 2825–2830.
Pentapalli, M. 2008. “A comparative study of Roth-Erev and modified Roth-Erev reinforcement learning algorithms for uniform-price double auctions.” Ph.D. dissertation, Dept. of Computer Science, Iowa State Univ.
Pront-Van Bommel, S. 2016. “A reasonable price for electricity.” J. Consum. Policy 39 (2): 141–158. https://doi.org/10.1007/s10603-015-9300-x.
Ringler, P., D. Keles, and W. Fichtner. 2016. “Agent-based modelling and simulation of smart electricity grids and markets: A literature review.” Renewable Sustainable Energy Rev. 57 (May): 205–215. https://doi.org/10.1016/j.rser.2015.12.169.
Roth, A. E., and I. Erev. 1995. “Learning in extensive-form games: Experimental data and simple dynamic models in the intermediate term.” Games Econ. Behav. 8 (1): 164–212. https://doi.org/10.1016/S0899-8256(05)80020-X.
Sarre, G., P. Blanchard, and M. Broussely. 2004. “Aging of lithium-ion batteries.” J. Power Sources 127 (1): 65–71. https://doi.org/10.1016/j.jpowsour.2003.09.008.
Seabold, S., and J. Perktold. 2010. “Statsmodels: Econometric and statistical modeling with Python.” In Proc., 9th Python in Science Conf. Austin, TX: Scipy.
Sheppard, K. 2017. “Linear model estimation: Linearmodels 4.5 documentation.” Accessed January 31, 2020. https://bashtage.github.io/linearmodels/doc/index.html#.
Shinde, P., and M. Amelin. 2019. “Agent-based models in electricity markets: A literature review.” In Proc., 2019 IEEE Innovative Smart Grid Technologies-Asia (ISGT Asia), 3026–3031. New York: IEEE.
Siegfried, R. 2014. Modeling and simulation of complex systems: A framework for efficient agent-based modeling and simulation. Wiesbaden, Germany: Springer.
Sun, J., and L. Tesfatsion. 2007. “Dynamic testing of wholesale power market designs: An open-source agent-based framework.” Comput. Econ. 30 (3): 291–327. https://doi.org/10.1007/s10614-007-9095-1.
Sun, P. C., H. M. Wang, H. L. Huang, and C. W. Ho. 2020. “Consumer attitude and purchase intention toward rooftop photovoltaic installation: The roles of personal trait, psychological benefit, and government incentives.” Energy Environ. 31 (1): 21–39. https://doi.org/10.1177/0958305X17754278.
Tervo, E., K. Agbim, F. DeAngelis, J. Hernandez, H. K. Kim, and A. Odukomaiya. 2018. “An economic analysis of residential photovoltaic systems with lithium-ion battery storage in the United States.” Renewable Sustainable Energy Rev. 94 (Oct): 1057–1066. https://doi.org/10.1016/j.rser.2018.06.055.
Tesfatsion, L. 2006. “Agent-based computational economics: A constructive approach to economic theory.” In Handbook of computational economics, edited by L. Tesfatsion and K. L. Judd, 831–880. Amsterdam, Netherlands: Elsevier.
Tungadio, D. H., B. P. Numbi, M. W. Siti, and A. A. Jimoh. 2015. “Particle swarm optimization for power system state estimation.” Neurocomputing 148 (Jan): 175–180. https://doi.org/10.1016/j.neucom.2012.10.049.
Van Der Walt, S., S. C. Colbert, and G. Varoquaux. 2011. “The NumPy array: A structure for efficient numerical computation.” Comput. Sci. Eng. 13 (2): 22–30. https://doi.org/10.1109/MCSE.2011.37.
Virtanen, P., et al. 2020. “SciPy 1.0: Fundamental algorithms for scientific computing in Python.” Nat. Methods 17 (3): 261–272. https://doi.org/10.1038/s41592-019-0686-2.
Waskom, M., et al. 2018. mwaskom/seaborn: v0.9.0. Switzerland: Zenodo.
Yin, L. 2013. “Assessing walkability in the City of Buffalo: Application of agent-based simulation.” J. Urban Plann. Dev. 139 (3): 166–175. https://doi.org/10.1061/(ASCE)UP.1943-5444.0000147.
Zhou, Z., F. Zhao, and J. Wang. 2011. “Agent-based electricity market simulation with demand response from commercial buildings.” IEEE Trans. Smart Grid 2 (4): 580–588. https://doi.org/10.1109/TSG.2011.2168244.
Information & Authors
Information
Published In
Journal of Energy Engineering
Volume 148 • Issue 5 • October 2022
Copyright
© 2022 American Society of Civil Engineers.
History
Received: Sep 27, 2021
Accepted: May 18, 2022
Published online: Aug 5, 2022
Published in print: Oct 1, 2022
Discussion open until: Jan 5, 2023
Authors
Metrics & Citations
Metrics
Citations
Download citation
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.
Cited by
- Gasser G. Ali, Islam H. El-adaway, Distributed Solar Generation: Current Knowledge and Future Trends, Journal of Infrastructure Systems, 10.1061/JITSE4.ISENG-2399, 30, 1, (2024).
- Gasser Ali, Islam H. El-Adaway, Agent-Based Modeling for Understanding Incentives Associated with Distributed Solar Generation, Computing in Civil Engineering 2023, 10.1061/9780784485248.123, (1030-1038), (2024).
- Ying Wang, Chang Liu, Weihong Yuan, Lili Li, MRL-Based Model for Diverse Bidding Decision-Makings of Power Retail Company in the Wholesale Electricity Market of China, Axioms, 10.3390/axioms12020142, 12, 2, (142), (2023).
- Alejandro Platas‐López, Alejandro Guerra‐Hernández, Marcela Quiroz‐Castellanos, Nicandro Cruz‐Ramirez, A survey on agent‐based modelling assisted by machine learning, Expert Systems, 10.1111/exsy.13325, (2023).