Traffic Density versus Rear-End Crash Risk on Freeways: Empirical Model, Mechanism Model, and Transfer to Automated Vehicles

The Highway Safety Manual (HSM) (AASHTO 2010) offers evidence-based tools for predicting the safety effects of design-related decisions. These tools are applicable to a variety of road types (segments and intersections on two-lane rural highways, multi-lane rural highways, and urban and suburban arterials), they allow for a wide range of roadway features (e.g., lane and shoulder widths, presence of curves, roadside features), and they support a range of crash types (e.g., angle, rear-end, or pedestrian-involved). The HSM uses statistical regression models, called safety performance functions (SPF), to predict expected crash frequencies for specific base conditions and empirically-derived crash modification factors (CMF) to predict the effects of deviations from the base conditions. Although the HSM is a major advance in practical knowledge regarding road safety, HSM also has, as currently implemented, certain limitations. First, the HSM quantifies associations between aggregated traffic measures, such as posted speed limits or annual average daily traffic volumes (AADT), and crash frequencies accumulated over several years. Since crash risk can vary as traffic conditions vary, the current HSM offers only limited guidance on responding to exceptional events or managing within-day variations in traffic. This issue has been recognized and is currently the focus of an NCHRP project (NCHRP 2019b). Second, the predictive tools in the HSM were derived largely from statistical summaries reflecting the population of drivers and vehicles on the roads during the recent past. This population is expected to change, perhaps substantially, when and if automated vehicles (AV) achieve a significant market share, and it is not yet clear how design and operational practices should change in response. This is also the subject of an NCHRP initiative (NCHRP 2019a) and brings us to a more general issue, assessing the external validity of safety-related findings. In their classic work, Campbell and Stanley (1966) defined external validity as “generalizability of empirical findings to new environments, settings, or populations…” but, in road safety, assessing external validity has usually been presented as a problem of determining the transferability of SPFs and CMFs (FHWA 2015, p 25). Once the connection between transferability and external validity is made, however, we can see that this issue goes beyond road safety. In policy analysis, Cartwright and Hardie (2012, p. 6) have argued that determining if a policy “will work here” from the knowledge that “it worked somewhere” requires “facts about the causal role the policy plays and facts about the support factors that must be in place for the policy to work.” In medicine, Parkkinen et al. (2018) have argued that determining the external validity of a claim that “A causes B” requires that “the mechanism responsible for B in the target population is sufficiently similar to that responsible for B in the study population” (Section 2.3). Taking this as our guide, it appears that transferring the safety knowledge gained in one set of conditions to other situations requires understanding the processes, i.e., mechanisms, generating the relevant crash phenomena. Although the nature of mechanisms is an ongoing topic of philosophical debate, one reasonable definition has been put forward by Illari and Williamson (2012, p. 120): “A mechanism for a phenomenon consists of entities and activities organized in such a way that they are responsible for the phenomenon.” A case can be made that a significant portion of scientific activity, especially in the life sciences, is devoted to discovering and elucidating mechanisms (e.g., Machamer et al. 2000), and mechanism-based arguments, applied to reconstructed crashes, have been used to estimate potential safety benefits arising from vehicle automation (e.g., Haus et al. 2019; Kusano and Gabler 2012; Rosen et al. 2010). In these studies, kinematic models for types of road crashes were combined with numerical inputs describing the actions of the involved parties and the capabilities of vehicle/road systems. Inputs were then changed to reflect new situations, and the kinematic models were used to simulate revised outcomes. The fractions of crashes that, according to the simulations, would have been prevented were then taken as measures of AV impacts. In highway design, mechanism-based descriptions of critical events have been used to derive design constraints (AASHTO 2017), but for safety prediction, the focus has been on regression models like those supporting the HSM and their relatives, such as advanced heterogeneity models (Mannering et al. 2016).

In what follows, we argue that mechanism-based models provide tools both for addressing short-term variations in crash risk and for transferring current safety knowledge to environments containing AVs. Our argument is by example, using rear-end crashes on freeways. The section “Regression-Based Modeling of Rear-End Crash Risk” outlines our regression-based work that found inverted U-shaped relationships between traffic density and rear-end crash risk on several segments of Interstate Highway 35W. The section “Mechanism-Based Model of Rear-End Crash Risk” describes a mechanism model, originally due to Brill (1972), that when coupled with a fundamental diagram of traffic flow reproduces such inverted U-shaped relationships. In this model, the crash probability for a platoon of vehicles is related to the first-passage time of a random walk, and a tractable procedure for computing this probability is presented. The section “Transferring Knowledge to an AV Environment” then describes how the model developed in the section “Mechanism-Based Model of Rear-End Crash Risk” can be used to characterize the crash risk for a platoon of vehicles equipped with adaptive cruise control (ACC) and automatic emergency braking (AEB). Finally, the section “Summary and Conclusion” presents our conclusions and suggestions for future work.

This section briefly reviews the literature on associations between traffic flow measures and crash risk on freeways and then introduces our recent work where logistic regression was used to control for the effects of traffic conditions in a before/after study of freeway improvements.

Compared to single-vehicle road departures, rear-end crashes on busy urban freeways are less likely to produce serious or fatal injuries, but they do generate much of the nonrecurring delay familiar to urban commuters. The fact that crash risk can vary with traffic conditions has been recognized for at least 20 years (Liu and Popoff 1997), leading to efforts at identifying those situations where crashes are more likely. Pioneering work in this area was performed by Oh et al. (2001) and Lee et al. (2002), with a focus on using inductive loop-detector data to identify short-term crash precursors. Abdel-Aty et al. (2004) introduced a case-control approach, where crash reports were reviewed in order to identify crashes occurring on a section of Interstate-4, and archived data from loop detectors were used to characterize traffic conditions. These crash events provided the cases for a case-control analysis, while control events were selected from the same locations, and at times similar to those of the crash events, but when no crashes had occurred. Logistic regression was then used to identify traffic measures that discriminated the cases from the controls. A model using average lane occupancy and the coefficient of variation of traffic speed as predictors detected 69% of the crash events. Subsequently, variants of this approach have led to significant improvements in predictive power, and Roshandel et al. (2015) have reviewed this work.

Our interest in modeling traffic conditions versus freeway crash risk is connected to recent work on estimating the safety-related impacts of a major set of improvements to Interstate Highway 35W (I-35W) (Davis et al. 2017). These improvements sought to reduce congestion on that part of I-35W running through the Twin Cities of Minnesota, and they included the conversion of the shoulder on a section of northbound I-35W to a priced dynamic shoulder lane (PDSL). Following the completion of the project, however, this PDSL section experienced both a substantial increase in traffic congestion due to the removal of an upstream bottleneck and a roughly three-fold increase in the frequency of rear-end crashes. It was unclear if this change in crash frequency was a direct effect of the PDSL or an indirect effect due to the relocation of traffic congestion. To untangle these effects, we employed an interrupted time-series design, with logistic regression being used to control for changes in traffic conditions.

Fig. 1 shows a Google Earth image of northbound I-35W; the PDSL section was divided into three segments, each of which contained an inductive loop-detector station. Segment N37 was 1.34 km (0.83 mi) long, segment N38 was 0.84 km (0.52 mi) long, and segment N40 was 1.26 km (0.78 mi) long. Computerized records for crashes occurring in this area were identified for a before period running from January 1, 2004, to December 31, 2006, and an after period running from January 1, 2011, to December 31, 2013; hard copies of the original crash reports were reviewed to verify the type, location, direction, and time for each crash. Each hour in both the before and the after periods was then classified as to whether or not at least one rear-end crash had been reported during that hour. 30-second traffic counts and lane occupancy measurements were obtained for each segment’s loop-detector station for both the before and after periods. Lane occupancy was of particular interest because it is approximately proportional to traffic density (Garber and Hoel 2015). That iswhere $o$ = lane occupancy (percent), $d$ = traffic density (vehicles/km), and $L_e$ = effective vehicle length (meters).

The raw loop-detector data were screened for missing or obviously erroneous values, such as records showing negative volume or lane occupancy, records with occupancies greater than 100%, or records showing the repetitive patterns caused by failures to update the recording system’s buffers. Traffic flow, average lane occupancy, and the standard deviation of lane occupancy were then computed for each analysis segment and for each hour during both the before and after periods. In order to not confound traffic conditions leading to a crash with those caused by a crash, for each hour when a crash occurred, the traffic conditions were computed for the half-hour preceding the reported time of the crash. Eq. (2) shows the logistic regression model that was used to relate the probability of crash occurrence to traffic and weather measurements and to a dummy variable that captured the direct effect, if any, of the PDSLwhere $Y_t=0$, no crash occurred during hour $t$; $Y_t=1$, crash occurred during hour $t$; ${\mathit{x}}_t$ = row $t$ of the predictor variable matrix; and ${\beta }_0$, $\mathit{\beta }$ = parameters to be estimated.

Initially, weather conditions (whether or not rainy or snowy conditions were present during that hour), traffic volume (vehicles/hour), average lane occupancy, lane occupancy standard deviation, and whether or not that hour occurred before or after the construction of the PDSL were all included as predictors of rear-end crash risk, and the maximum likelihood method was used to fit and evaluate different combinations of the predictors. This process has been described in detail (Davis et al. 2017), and here we simply summarize some especially relevant findings. We found that our initial models had difficulty passing routine goodness-of-fit tests, and Fig. 2 plots the occurrence or nonoccurrence of rear-end crashes versus lane occupancy for PDSL segment N38. Consistent with findings reported by Xu et al. (2014), crashes were most frequent when lane occupancy took on mid-range values and were less frequent when lane occupancy was either low or high. These considerations suggested that our models should allow for a possibly concave relationship between lane occupancy and crash risk, and a quadratic term for lane occupancy was added to the predictor list. Table 1 shows the final model selected for segment N38 and includes only those predictors whose coefficients were significantly different from zero. In particular, the coefficient for the variable indicating the presence/absence of the PDSL was insignificant, indicating no direct effect from the PDSL. Note though that both lane occupancy and lane ${\text{occupancy}}^2$ were included and that the coefficient for lane ${\mathrm{occupancy}}^2$ was negative, indicating a concave region for the fitted risk curve.

Fig. 3 shows the relationship between lane occupancy and rear-end crash risk, as predicted by the logit model summarized in Table 1. Taking the derivative of Eq. (2) with respect to lane occupancy, setting this equal to zero, and solving for lane occupancy gives an expression for the point where crash risk was greatest. Using the estimates from Table 1, for segment N38 the crash probability was greatest when lane occupancy (%) was approximately equal towhere $\overline{o}$ = average lane occupancy over all hours in the study period, which for N38 was equal to 7.26%.

Similar patterns, with rear-end crash risk showing an inverted U-shaped relationship to lane occupancy, were found for segments N37 and N40 as well as for four of five additional segments on I35W. Since lane occupancy is roughly proportional to traffic density, these results indicated that rear-end crash risk should also show an inverted U-shaped relation to traffic density.

The logistic regression model illustrated in Fig. 3 summarized the association between crash risk and lane occupancy for drivers and vehicles present during 2004–2013. As noted earlier, though, transferring this model to a population of AVs is problematic. Fig. 6, on the other hand, shows the probability that a stopping wave produces a crash or near-crash event and was developed from a mechanism-based model, where a fundamental diagram described steady-state car-following behavior and where Brill’s random walk model described how a stopping wave leads to a crash or near-crash event. In principle, a fundamental diagram describing traffic flow for AVs, along with data on their braking rates and reaction times, should make it possible to construct a similar diagram for the AVs. In what follows, we illustrate how this can be done for a population of AVs equipped with ACC and AEB. All vehicles in our hypothetical population are assumed to have the same ACC and AEB systems. Car-following is assumed to be governed by the ACC system while braking in a stopping wave is governed by the AEB system. Estimates from recently conducted field studies of ACC and AEB systems currently on the road are then used to characterize crash risk in a hypothetical platoon of AVs.

Regarding AEB, researchers have recently conducted field tests of a system implemented in two 2017 Toyota Corollas (Yang et al. 2018; Xing et al. 2018). This system used radar mounted behind the front grill and a monocular camera mounted just in front of the rear-view mirror to identify possible hazards. The system issued auditory and visual alarms when predicted time-to-collision fell below a threshold; if the driver did not respond, or if the driver’s response appeared insufficient to prevent the collision, the system initiated full braking autonomously. In the field tests, the timing of these and related events were obtained from the system’s vehicle control history (VCH). The tests involved approaching a stationary object with a constant accelerator pedal position and no braking by the driver and then recording the vehicle’s final position along with data from the VCH. Xing et al. (2018, p. 5) noted that autonomous deceleration appeared to correlate with an event labeled FPB in the VCH, while Yang et al. (2018, p. 5) noted that the average time between the warning and the start of autonomous deceleration was approximately 0.94 s. They also reported a maximum braking rate deceleration of $0.9g$ and suggested that a plausible standard deviation for the time lag between the warning signal and the start of deceleration was 0.12 s. Adding 0.1 s to account for the time needed by the system to identify a hazard gave the following characteristics for our hypothetical AEB system: mean reaction time $\mu =1.04\mathrm{s}$, reaction time standard deviation $\sigma =0.12\mathrm{s}$, and maximum braking rate $a_{\max }=0.9g$.

Regarding ACC, field experiments seeking to characterize the car-following behavior of existing ACC systems have been described in Gunter et al. (2019a, b). These involved recording the positions of leading and following vehicles during car-following situations where the following vehicles were operating under ACC. Parameters characterizing different car-following models were then estimated by minimizing the squared error between observed and predicted speed histories for seven different vehicle models. Gunter et al. (2019b, p. 3053) compared three different car-following models and concluded that the optimal velocity with relative velocity term (OVRV) model and the intelligent driver model (IDM) “perform roughly the same.” Gunter et al. (2019a) fit OVRV models to data from the seven vehicle models and then used stability analysis methods to assess the string stability of the estimated car-following behavior. Interestingly, the authors found that the fitted models for each of the seven ACC systems were string unstable, i.e., they tended to amplify disturbances and so, potentially, could generate stopping waves. Recognizing that replication and extension of this pioneering work is needed before conclusions can be finalized, we will use some of these results to illustrate how the model developed in in the section “Mechanism-Based Model of Rear-end Crash Risk” might be transferred to an AV environment.

The version of the OVRV model used in Gunter et al. (2019a) has as its steady-state relationship between mean traffic speed and traffic density$t_h$ = ACC’s target time headway; and $\eta$ = jam spacing, and this is a version of the right-hand side ($d>d_c$) of Newell’s model shown in Eq. (7). For a given free-flow speed $v_f$, the corresponding wave speed and critical density can be computed via

Gunter et al. (2019a) fit separate models for when the vehicles’ ACCs were at their minimum (shortest headway) and maximum (longest headway) settings. For their vehicle A at the minimum setting, the estimated target headway was $t_h=0.82\mathrm{s}$ and the estimated jam spacing was $\eta =8.03\mathrm{m}$, while at the maximum setting, the estimated parameters were $t_h=2.05\mathrm{s}$ and $\eta =5.96\mathrm{m}$. For our hypothetical AV, we used estimates for both settings to construct fundamental diagrams, in each case with a free-flow speed of $88.5\mathrm{km}/\mathrm{h}$ (55 mi/h).

Fig. 9 shows the steady-state following times for vehicle A at both its minimum and maximum settings. Also shown is the mean reaction time for the AEB system, 1.04 s. For the minimum setting, the mean following time is generally longer than the mean braking reaction time, but it dips below the mean reaction time when traffic density is roughly between 33 vehicles/km/lane and 45 vehicles/km/lane. Brill’s model then tells us that for traffic densities in this range, and for long platoons, a stopping wave is likely to result in a crash. At the maximum setting, however, the steady-state mean following time is never below 2.0 s, suggesting that stopping waves would most likely be benign.

Fig. 10 shows the probability a stopping wave produces a crash/near-crash in a 0.4 km (0.25 mi) long section of one freeway lane. Car-following and braking behavior were the same as those for the model A vehicle at its minimum setting, and the initiating deceleration was, as for Fig. 6, $3.05{\mathrm{mps}}^2$ ($10{\mathrm{fps}}^2$). Crash probabilities were computed using the finite Markov chain approximation described in the section “Mechanism-Based Model of Rear-End Crash Risk.” The greatest risk occurs at densities of approximately 43 vehicles/km/lane. For the ACCs at the maximum setting, on the other hand, all computed crash probabilities were less than $1\times {10}^{-10}$.

Using data from I-35W in Minneapolis, empirical logistic regression analyses found inverted U-shaped relationships between crash risk and lane occupancy. Macroscopic traffic flow models imply that vehicle headways should show a roughly convex relation to traffic density; headways tend to be long when densities are low because vehicles are widely separated, and headways tend to be long when densities are high because vehicles have slow speeds. Brill’s microscopic model of how a platoon of braking vehicles can produce a rear-end crash emphasizes the differences between reaction times and following times and shows how the probability of a crash occurring in a stopping wave can be computed via the first passage of a random walk. By combining Brill’s model with a macroscopic traffic flow model, it was possible to predict a relationship between traffic density and rear-end crash risk. Computational difficulties related to passage times for random walks were alleviated by approximating the random walks with finite Markov chains. The resulting combined model predicted an inverted U-shape to the relation between traffic density and the probability that a stopping wave produced a crash/near-crash. Although it is not yet clear how to transfer empirical regression models, such as the HSM’s SPF, to situations where AVs are present, it is possible to transfer the mechanism-based model described in the “Mechanism-Based Model of Rear-End Crash Risk” section. This possibility was illustrated using data from field tests of vehicles equipped with ACC and AEB.

Our findings should be regarded as promising rather than final. What we have presented is an example of how a mechanism-based model can (1) relate crash risk to traffic conditions and (2) be transferred to an environment containing AVs. The empirical support for our model came from detailed study of one section of I-94, so replication using other freeway sections is needed. Also, our model has four main components: a steady-state relationship between traffic density and mean speed, a probability model for drivers’ reaction times, a random walk model describing how a stopping wave produces a rear-end crash, and an empirical model relating traffic density to the probability that a stopping wave occurs. It is possible that, with additional work, each of these component models could be replaced. In particular, the random walk in Eq. (4) could be replaced by a more detailed stochastic process that allows vehicles/drivers in a wave to have different speeds, braking strategies, or reaction models, while the empirical relationship between traffic density and stopping wave probability could, in principle, be replaced by one derived from car-following theory. Research into the capabilities of AVs is at an early stage and undoubtedly the ACC and AEB models described in the “Transferring Knowledge to an AV Environment” section can, and probably will, be replaced in the future. In the near term, it will also be important to consider traffic with mixtures of AVs and legacy vehicles. We would also like to note that while the model presented here does not include infrastructure-related features, recent work has illustrated how mechanism-based modeling can explain CMFs associated with changes in left-turn lane offset (Davis 2021) and installation of pedestrian hybrid beacons (Davis 2019). The challenge here is not so much model construction as it is our limited understanding of how crash-related modifications achieve their effects.

Finally, Brill’s model is a hypothesized mechanism for explaining how rear-end crashes occur. This contrasts with the statistical models that underpin the HSM, which can produce useful results even when little is known about how crashes occur. An ongoing debate concerns the value of mechanistic models and explanations for highway safety engineering (Bonneson and Ivan 2013), and interestingly, there is a similar debate about the role of mechanisms in evidence-based medicine (Howick 2011). Proponents of evidence-based medicine generally regard randomized controlled trials as providing the strongest evidence for the effectiveness of treatments, and an open question concerns what role knowledge of physiological mechanisms should play in prescribing treatments (Andersen 2012). Bluhm (2013) has argued that mechanistic knowledge in medical research could be used to guide and focus the design of statistical studies, and something similar occurred in our I-35W study (Davis et al. 2017). Brill’s mechanism, coupled with Greenshield’s fundamental diagram, was used to suggest the quadratic form for lane occupancy predictors in the logistic regressions, leading to noticeable improvements in model fit. More generally, in addition to guiding decisions about external validity, the heuristic value of mechanisms for suggesting productive lines of research should not be discounted.

In the section “Mechanism-Based Model of Rear-End Crash Risk,” we approximated the random walk in Eq. (5) with a finite, discrete-state Markov chain. Derivations of this approximation have been given elsewhere (Brook and Evans 1972, p. 543; Chatterjee 2016, pp. 59–61); here, we simply illustrate how the approximation is constructed. Consider one lane in a 0.4 km (0.25 mi) segment of freeway, where the traffic is well-described by Newell’s model with a capacity flow of 2,250 vehicles/hour/lane, a free-flow speed of $88.5\mathrm{km}/\mathrm{h}$ (55 mi/h), and a wave speed of $24.1\mathrm{km}/\mathrm{h}$ (15 mi/h). Driver reaction times are normally distributed, with a mean of 1.5 s and a standard deviation of 0.2 s, and the maximum feasible braking rate is $a_{\max }=6.1{\mathrm{mps}}^2$ ($20{\mathrm{fps}}^2$). In the lane of interest, traffic density is currently $37.3\mathrm{vehicles}/\mathrm{km}/\mathrm{lane}$ ($60\text{vehicles}/\mathrm{mi}/\mathrm{lane}$) when a driver at the segment’s downstream boundary brakes to a stop at $a_0=3.0{\mathrm{mps}}^2$ ($10{\mathrm{fps}}^2$). Newell’s model with the above parameters, and the given traffic density, gives the common speed in this lane as $v_0=14.6$ mps ($48\mathrm{ft}/\mathrm{s}$), and there are $N=15$ vehicles in this lane. The common following time is then $h=1.52\mathrm{s}$, and the critical time is $t_{\mathrm{crit}}=1.2\mathrm{s}$.

To illustrate the computation of the Markov chain’s transition probabilities, we will, unrealistically, approximate the state space for $S_n$ using only $m=5$ states. The range ($-tcrit,tcri\mathrm{t})=(-1.2,1.2$) is first divided into $m-2=3$ intervals of width $w=(tcrit-(-tcrit))/(m-2)=0.8\mathrm{s}$. The absorbing state is given index 5 and corresponds to the crash condition $S_n>t_{\mathrm{crit}}=1.2\mathrm{s}$. State 1 corresponds to $S_n<-t_{\mathrm{crit}}$, while states 2, 3, and 4 represent intermediate values for $S_n$. State 3, with a midpoint of 0 s, represents $S_n$ between $0-w/2=-0.4\mathrm{s}$ and $0+w/2=0.4\mathrm{s}$, while state 4 has a midpoint of 0.8 s and represents $S_n$ between 0.4 and 1.2 s. A transition from state 3 to state 4 occurs when the reaction time/following time difference $r-h$, plus the midpoint of state 3, is greater than the lower boundary for state 4 but not greater than the upper boundary of state 4. For this example, $r-h$ would need to be greater than 0.4 s, so that $0+(r-h)>0.4$, but less than 1.2, so that $0+(r-h)<1.2$. Since $r$ is assumed to be normally distributed, with a mean of 1.5 s and a standard deviation of 0.2 s, while $h$ is constant at 1.52 s, we have $P_{3,4}=P(0.4<r-h<1.2)=0.018$. Table 2 shows the resulting transition matrix P for this five-state approximation to $S_n$. Here $\mathrm{E}[r-h]=-0.02<0$, leading to the mild negative drift shown in the transition matrix.

Taking the submatrix consisting of the first four rows and columns of this matrix to be the matrix R in Eq. (8) and letting $N=15$ be the number of vehicles in this lane, we have

The probability that this stopping wave produces a crash would then be 0.022. In practice, a finer discretization of the state space should be used, and in our experiments, $m>100$ resulted in collision probabilities that were stable to four decimal places. Computing the probabilities shown in Figs. 6, 8, and 10 was done using Mathcad (e.g., Maxfield 2009), and copies of this Mathcad document are available from the corresponding author upon reasonable request.

Some or all data, models, or code that support the findings of this study are available from the corresponding author upon reasonable request.

This research was supported in part by the Minnesota DOT. The authors would like to thank Raphael Stern for his insight and guidance regarding the ACC models in the section “Transferring Knowledge to an AV Environment.”