Intelligent Stochastic Agent-Based Model for Predicting Truck Production in Construction Sites by Considering Learning Effect
Publication: Journal of Construction Engineering and Management
Volume 148, Issue 5
Abstract
Predicting truck production in construction projects is one of the basic tasks within project planning and control. This paper presents an original and novel intelligent stochastic agent-based model to maximize truck production at construction sites by considering the impact of learning. The proposed model was developed to overcome limitations of existing models, including a lack of the inclusion of a training mechanism and a reward/penalty framework for truck performance. Ideas of reinforcement learning theory were used. A reward/penalty function was designed based on minimum travel time. Traffic and fuel volume were treated as stochastic variables. A worked example and a real case study are presented to show the applicability and efficiency of the proposed model. The paper shows that the results of the proposed model accurately predict truck production. The paper also shows that the proposed model demonstrates a shorter truck travel time and, thus, higher production compared to the Monte Carlo simulation logic. The method proposed here offers an original contribution to the analysis of truck production and will be of use to practitioners engaged in project planning and control, especially in large earth-moving operations.
Get full access to this article
View all available purchase options and get full access to this article.
Data Availability Statement
Data generated or analyzed during the study are available from the corresponding author by request.
References
AbouRizk, S. 2010. “Role of simulation in construction engineering and management.” J. Constr. Eng. Manage. 136 (10): 1140–1153. https://doi.org/10.1061/(ASCE)CO.1943-7862.0000220.
Ahn, C., W. Pan, S. Lee, and F. Peña-Mora. 2010. “Enhanced estimation of air emissions from construction operations based on discrete-event simulation.” In Proc., Int. Conf. on Computing in Civil and Building Engineering. Nottingham, UK: Nottingham Univ.
Ahn, S., P. S. Dunston, A. Kandil, and J. C. Martinez. 2015a. “Process mining technique for automated simulation model generation using activity log data.” Comput. Civ. Eng. 2015 (1): 636–643. https://doi.org/10.1061/9780784479247.079.
Ahn, S., J. Kim, P. S. Dunston, A. Kandil, and J. C. Martinez. 2015b. “Characterizing travel time distributions in earthmoving operations using GPS data.” Comput. Civ. Eng. 2015 (1): 288–295. https://doi.org/doi.org/10.1061/9780784479247.036.
Amirkhanian, S. N., and N. J. Baker. 1992. “Expert system for equipment selection for earth-moving operations.” J. Constr. Eng. Manage. 118 (2): 318–331. https://doi.org/10.1061/(ASCE)0733-9364(1992)118:2(318).
Bureau of Public Roads. 1964. Bureau of public roads, traffic assignment manual. Washington, DC: US Department of Commerce.
Carmichael, D. G. 2006. Project planning, and control. 1st ed. New York: Taylor & Francis.
Carmichael, D. G., B. J. Bartlett, and A. S. Kaboli. 2013. “Surface mining operations: Coincident unit cost and emissions.” Int. J. Min. Reclam. Environ. 28 (1): 47–65. https://doi.org/10.1080/17480930.2013.772699.
Chanda, E. K., and S. Gardiner. 2010. “A comparative study of truck cycle time prediction methods in open-pit mining.” Eng. Constr. Archit. Manage. 17 (5): 446–460. https://doi.org/10.1108/09699981011074556.
Chaowasakoo, P., H. Seppälä, H. Koivo, and Q. Zhou. 2017. “Digitalization of mine operations: Scenarios to benefit in real-time truck dispatching.” Int. J. Min. Sci. Technol. 27 (2): 229–236. https://doi.org/10.1016/j.ijmst.2017.01.007.
Chen, C., Z. Zhu, A. Hammad, and M. Akbarzadeh. 2021. “Automatic identification of idling reasons in excavation operations based on excavator—Truck relationships.” Comput. Civ. Eng. 35 (5): 04021015. https://doi.org/10.1061/(ASCE)CP.1943-5487.0000981.
Chen, Y., S. Huang, F. Liu, Z. Wang, and X. Sun. 2019. “Evaluation of reinforcement learning-based false data injection attack to automatic voltage control.” IEEE Trans. Smart Grid 10 (2): 2158–2169. https://doi.org/10.1109/TSG.2018.2790704.
Coldren, G. M., and F. S. Koppelman. 2005. “Modeling the competition among air-travel itinerary shares: GEV model development.” Transp. Res. Part A Policy Pract. 39 (4): 345–365. https://doi.org/10.1016/j.tra.2004.12.001.
Dawid, H. 2006. “Agent-based computational economics 2.” In Handbook of computational economics, edited by L. Tesfatsion and K. L. Judd. New York: IEEE.
Ditlevsen, O., and H. O. Madsen. 1996. “Structural reliability.” In Methods. 1st ed. New York: Wiley.
El Naqa, I., and M. J. Murphy. 2015. “What is machine learning?” In Machine learning in radiation oncology, 3–11. Berlin: Springer.
Friedrich, M., I. Hofsaess, and S. Wekeck. 2001. “Timetable-based transit assignment using branch and bound techniques.” Transp. Res. Rec. 1752 (1): 100–107. https://doi.org/10.3141/1752-14.
Gabel, T., and M. Riedmiller. 2007. “On a successful application of multi-agent reinforcement learning to operations research benchmarks.” In Proc., IEEE Int. Symp. on Approximate Dynamic Programming and Reinforcement Learning, 68–75. New York: IEEE.
Gosavi, A. 2008. “On step sizes, stochastic shortest paths, and survival probabilities in Reinforcement Learning.” In Proc., 40th Conf. on Winter Simulation, Winter Simulation Conf. New York: IEEE.
Gosavi, A. 2014a. “Simulation-based optimization: An overview.” In Operations research/computer science interfaces series, 29–35. Berlin: Springer.
Gosavi, A. 2014b. Simulation-based optimization: Parametric optimization techniques and reinforcement learning. 2nd ed. New York: Springer.
Grimmett, G., and D. Welsh. 2014. Probability: An introduction. Oxford, UK: Oxford University Press.
Hájek, A. 2003. “What conditional probability could not be.” Synthese 137 (3): 273–323. https://doi.org/10.1023/B:SYNT.0000004904.91112.16.
Hajji, A. M., and P. Lewis. 2013. “Development of productivity-based estimating tool for energy and air emissions from earthwork construction activities.” Smart Sustainable Built Environ. 2 (1): 84–100. https://doi.org/10.1108/20466091311325863.
Hammad, A. W. A., A. Akbarnezhad, and D. Rey. 2016. “A multi-objective mixed integer nonlinear programming model for construction site layout planning to minimize noise pollution and transport costs.” Autom. Constr. 61 (Jan): 73–85. https://doi.org/10.1016/j.autcon.2015.10.010.
Hu, J., and M. P. Wellman. 1998. “Multiagent reinforcement learning: Theoretical framework and an algorithm.” In Proc., 15th Int. Conf. on Machine Learning. San Francisco: Morgan Kaufmann.
Huntsinger, L. F., and N. M. Rouphail. 2011. “Bottleneck and queuing analysis: Calibrating volume–Delay functions of travel demand models.” Transp. Res. Rec. 2255 (1): 117–124. https://doi.org/10.3141/2255-13.
Jabri, A., and T. Zayed. 2017. “Agent-based modeling and simulation of earthmoving operations.” Autom. Constr. 81 (Sep): 210–223. https://doi.org/10.1016/j.autcon.2017.06.017.
Jayawardane, A. K. W., and F. C. Harris. 1990. “Further development of integer programming in earthwork optimization.” J. Constr. Eng. Manage. 116 (1): 18–34. https://doi.org/10.1061/(ASCE)0733-9364(1990)116:1(18).
Jun, D. H., and K. El-rayes. 2011. “Fast and accurate risk evaluation for scheduling large-scale construction projects.” J. Comput. Civ. Eng. 25 (5): 407–417. https://doi.org/10.1061/(ASCE)CP.1943-5487.0000106.
Kaboli, A. S., and D. G. Carmichael. 2012. “Emission and cost configurations in earthmoving operations.” Organ. Technol. Manage. Constr. 4 (1): 393–402.
Karshenas, S., and X. Feng. 1992. “Application of neural networks in earthmoving equipment production estimating.” In Proc., Computing in Civil Engineering and Geographic Information Systems Symp., 841–847. Reston, VA: ASCE.
Kim, K., and K. J. Kim. 2010. “Multi-agent-based simulation system for construction operations with congested flows.” Autom. Constr. 19 (7): 867–874. https://doi.org/10.1016/j.autcon.2010.05.005.
Kormushev, P., S. Calinon, and D. Caldwell. 2013. “Reinforcement learning in robotics: Applications and real-world challenges.” Robotics 2 (3): 122–148. https://doi.org/10.3390/robotics2030122.
Li, C.-Q., G. Zhang, and S. M. Hosseinian. 2017. “A fast and accurate method to predict reliability of project completion time.” J. Civ. Eng. Manage. 23 (1): 37–46. https://doi.org/10.3846/13923730.2014.953570.
Macal, C. M. 2016. “Everything you need to know about agent-based modelling and simulation.” Simulation 10 (2): 144–156. https://doi.org/10.1057/jos.2016.7.
Macal, C. M., and M. J. North. 2010. “Tutorial on agent-based modelling and simulation.” Simulation 4 (3): 151–162. https://doi.org/10.1057/jos.2010.3.
Maghrebi, M., C. Sammut, and T. Waller. 2013. “Reconstruction of an expert’s decision making expertise in concrete dispatching by machine learning.” Civ. Eng. Archit. 7 (12): 1540–1547. https://doi.org/10.17265/1934-7359/2013.12.009.
Marceau, M. L., M. A. Nisbet, and M. G. Van Geem. 2007. Life cycle inventory Portland cement concrete. Skokie, IL: Portland Cement Association.
Maxizm Company. 2019. “Construction machinery.” Accessed July 17, 2020. https://maxizm.en.made-in-china.com/.
McArthur, S. D. J., E. M. Davidson, V. M. Catterson, A. L. Dimeas, N. D. Hatziargyriou, F. Ponci, and T. Funabashi. 2007. “Multi-agent systems for power engineering applications—Part I: Concepts, approaches, and technical challenges.” IEEE Trans. Power Syst. 22 (4): 1743–1752. https://doi.org/10.1109/TPWRS.2007.908471.
Mitrovic, N., and A. Stevanovic. 2019. “Estimating peak-hour traffic profiles for selection of appropriate day-of-year signal timing plans.” Transp. Res. Rec. 2673 (10): 199–213. https://doi.org/10.1177/0361198119841860.
Mohamed, Y., and S. M. AbouRizk. 2005. “Framework for building intelligent simulation models of construction operations.” Comput. Civ. Eng. 19 (3): 277–291. https://doi.org/10.1061/(ASCE)0887-3801(2005)19:3(277).
Moradi, M. H., S. Razini, and S. M. Hosseinian. 2016. “State of art of multiagent systems in power engineering: A review.” Renewable Sustainable Energy Rev. 58 (May): 814–824. https://doi.org/10.1016/j.rser.2015.12.339.
Morozs, N. 2015. “Accelerating reinforcement learning for dynamic spectrum access in cognitive wireless networks.” Ph.D. thesis, Dept. of Electronics, Univ. of York.
Nunnally, S. W. 2013. Construction methods and management: Pearson new international edition PDF eBook. London: Pearson.
Pan, Y. 2016. “Heading toward artificial intelligence 2.0.” Engineering 2 (4): 409–413. https://doi.org/10.1016/J.ENG.2016.04.018.
Riedl, M. O., and H. Brent. 2016. “Using stories to teach human values to artificial agents.” In Proc., Workshops at the 30th AAAI Conf. on Artificial Intelligence. Menlo Park, CA: Association for the Advancement of Artificial Intelligence.
Roche, R., B. Blunier, A. Miraoui, V. Hilaire, and A. Koukam. 2010. “Multi-agent systems for grid energy management: A short review.” In Proc., IECON 2010–36th Annual Conf. on IEEE Industrial Electronics Society, 3441. New York: IEEE.
Salling, K. B., and D. Banister. 2009. “Assessment of large transport infrastructure projects: The CBA-DK model.” Transp. Res. Part A Policy Pract. 43 (9–10): 800–813. https://doi.org/10.1016/j.tra.2009.08.001.
Shibasaki, R., T. Azuma, T. Yoshida, H. Teranishi, and M. Abe. 2017. “Global route choice and its modelling of dry bulk carriers based on vessel movement database: Focusing on the Suez Canal.” Res. Transp. Bus. Manage. 25 (Dec): 51–65. https://doi.org/10.1016/j.rtbm.2017.08.003.
Shojaei, A., H. I. Moud, and I. Flood. 2018. “The need for remote artificial intelligence control of space-based construction projects: Multi-agent-based last planners, local centralized controllers, and hybrid solutions to decision-making.” In Earth and space 2018. Reston, VA: ASCE.
Stathopoulos, A., and M. G. Karlaftis. 2002. “Modeling duration of urban traffic congestion.” Transp. Eng. 128 (6): 587–590. https://doi.org/10.1061/(ASCE)0733-947X(2002)128:6(587).
Suh, S., C. H. Park, and T. J. Kim. 1990. “A highway capacity function in Korea: Measurement and calibration.” Transp. Res. Part A General 24 (3): 177–186. https://doi.org/10.1016/0191-2607(90)90055-B.
Sutton, R. S., and A. G. Barto. 2018. Reinforcement learning: An introduction. 2nd ed. Cambridge, MA: MIT Press.
Tang, T. Q., W. F. Shi, X. B. Yang, Y. P. Wang, and G. Q. Lu. 2013. “A macro traffic flow model accounting for road capacity and reliability analysis.” Physica A 392 (24): 6300–6306. https://doi.org/10.1016/j.physa.2013.07.035.
Tolliver, D., P. Lu, and D. Benson. 2013. “Comparing rail fuel efficiency with truck and waterway.” Transp. Res. Part D Transp. Environ. 24 (Oct): 69–75. https://doi.org/10.1016/j.trd.2013.06.002.
Vahdatikhaki, F., A. Hammad, and S. M. Langari. 2015. “Multi-agent system for improved safety and productivity of earthwork equipment using real-timelocation systems.” In Proc., Canadian Society for Civil Engineering’s 5th Int./11th Construction Specialty Conf. (ICSC15). Vancouver, Canada: Univ. of British Columbia. https://doi.org/10.14288/1.0076351.
Van der Waerden, P. J. H. J., A. W. J. Borgers, and H. J. P. Timmermans. 2004. “Choice set composition in the context of pedestrians’ route choice modeling.” In Proc., 83rd Annual Meeting of the Transportation Research Board. Washington, DC: National Academies of Sciences, Engineering, and Medicine.
Wang, H., X. Chen, Q. Wu, Q. Yu, X. Hu, Z. Zheng, and A. Bouguettaya. 2017. “Integrating reinforcement learning with multi-agent techniques for adaptive service composition.” ACM Transp. Autom. Adapt. Syst. 12 (2): 1–42. https://doi.org/10.1145/3058592.
Wang, Y., Y. Liu, W. Cheng, Z. M. Ma, and T. Y. Liu. 2020. “Target transfer Q-learning and its convergence analysis.” Neurocomputing 392: 11–22. https://doi.org/10.1016/j.neucom.2020.02.117.
Wang, Z., and H. Hu. 2017. “Improved precast production–Scheduling model considering the whole supply chain.” Comput. Civ. Eng. 31 (4): 04017013. https://doi.org/10.1061/(ASCE)CP.1943-5487.0000667.
Wang, Z., and R. S. Srinivasan. 2017. “A review of artificial intelligence based building energy use prediction: Contrasting the capabilities of single and ensemble prediction models.” Renewable Sustainable Energy Rev. 75 (Aug): 796–808. https://doi.org/10.1016/j.rser.2016.10.079.
Watkins, C. J. C. H., and P. Dayan. 1992. “Q-learning.” Mach. Learn. 8 (3–4): 279–292. https://doi.org/10.1007/BF00992698.
Watkins, M., A. Mukherjee, N. Onder, and K. Mattila. 2009. “Using agent-based modeling to study construction labor productivity as an emergent property of individual and crew interactions.” Constr. Eng. Manage. 135 (7): 657–667. https://doi.org/10.1061/(ASCE)CO.1943-7862.0000022.
Wojtusiak, J., T. Warden, and O. Herzog. 2012. “Machine learning in agent-based stochastic simulation: Inferential theory and evaluation in transportation logistics.” Comput. Math. Appl. 64 (12): 3658–3665. https://doi.org/10.1016/j.camwa.2012.01.079.
Wooldridge, M. 2009. “An introduction to multi agent.” In Systems. 2nd ed. New York: Wiley.
Yan, S., W. Lai, and M. Chen. 2008. “Production scheduling and truck dispatching of ready mixed concrete.” Transp. Res. Part E Logis. Transp. Rev. 44 (1): 164–179. https://doi.org/10.1016/j.tre.2006.05.001.
Yip, H., H. Fan, and Y. Chiang. 2014. “Predicting the maintenance cost of construction equipment: Comparison between general regression neural network and Box–Jenkins time series models.” Autom. Constr. 38 (Mar): 30–38. https://doi.org/10.1016/j.autcon.2013.10.024.
Zankoul, E., H. Khoury, and R. Awwad. 2015. “Evaluation of agent-based and discrete-event simulation for modeling construction earthmoving operations.” In Proc., 32nd Int. Symp. on Automation and Robotics in Construction and Mining. Brussels, Belgium: International Association for Automation and Robotics in Construction.
Zhan, X., Y. Zheng, X. Yi, and S. V. Ukkusuri. 2017. “Citywide traffic volume estimation using trajectory data.” IEEE Trans. Knowl. Data Eng. 29 (2): 272–285. https://doi.org/10.1109/TKDE.2016.2621104.
Zhang, L. 2011. “Behavioral foundation of route choice and traffic assignment: Comparison of principles of user equilibrium traffic assignment under different behavioral assumptions.” Transp. Res. Rec. 2254 (1): 1–10. https://doi.org/10.3141/2254-01.
Zhang, L., and X. Xia. 2015. “An integer programming approach for truck-shovel dispatching problem in open-pit mines.” Energy Procedia 75 (Aug): 1779–1784. https://doi.org/10.1016/j.egypro.2015.07.469.
Zhou, M., D. Wang, Q. Li, Y. Yue, W. Tu, and R. Cao. 2017. “Impacts of weather on public transport ridership: Results from mining data from different sources.” Transp. Res. Part C Emerging Technol. 75 (Feb): 17–29. https://doi.org/10.1016/j.trc.2016.12.001.
Zhu, W., and R. Xu. 2016. “Generating route choice sets with operation information on metro networks.” Traffic Transp. Eng. 3 (3): 243–252. https://doi.org/10.1016/j.jtte.2016.05.001.
Information & Authors
Information
Published In
Journal of Construction Engineering and Management
Volume 148 • Issue 5 • May 2022
Copyright
© 2022 American Society of Civil Engineers.
History
Received: Aug 20, 2021
Accepted: Jan 9, 2022
Published online: Mar 11, 2022
Published in print: May 1, 2022
Discussion open until: Aug 11, 2022
Authors
Metrics & Citations
Metrics
Citations
Download citation
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.
Cited by
- Weiwei Qi, Shufang Zhu, Jinsong Hu, Correlation Analysis of Real-Time Warning Factors for Construction Heavy Trucks Based on Electrified Supervision System, Sustainability, 10.3390/su141710944, 14, 17, (10944), (2022).