Study of the Factors Affecting Road Roughness Measurement Using Smartphones

The importance of maintaining roads at an appropriate standard to encourage economic development, minimize road use costs (i.e., travel time, fuel efficiency, vehicle repair costs, and accidents), provide social benefit, and reduce environmental impacts of transport is well documented (Robinson 2008). However, road agencies worldwide generally have insufficient maintenance budgets to treat their entire road networks, and therefore in order to make best use of scarce resources they are obliged to prioritize maintenance according to the perceived socioeconomic returns (Asphalt Industry Alliance 2018). To achieve this in a rational manner, the condition of the road networks must be assessed periodically using appropriate means at suitable frequencies, accuracies, and levels of detail. Ideally, data collection strategies associated with road surface condition should consider the functional performance of the road (e.g., roughness, surface cracking, and fretting), the structural condition (e.g., deflection and rutting), and measures of the road surface associated with the safety of the running surface (e.g., skid resistance and texture depth) (Paterson and Scullion 1990). Because the cost of data collection is considerable, data collection strategies for strategic road networks often are adopted whereby the functional condition, skid resistance, and texture depth of the network is measured in its entirety on an annual basis to identify problem areas rapidly. The structural condition, which is much more expensive to assess, is measured less frequently (McGhee 2004). The assessment of the condition of secondary, or local, roads is even less frequent, and often structural condition is ignored. On most road networks, road roughness usually is adopted as the measure of functional condition because it can be related readily to road use costs and its measurement can be automated. The most accurate automated methods of assessing road roughness use vehicles fitted with lasers to measure the road’s longitudinal profile. The obtained road profile then is converted to road roughness using a standard mathematical procedure. Nevertheless, even assessing the roughness of a reasonably sized network can be costly. McGhee (2004) found that the cost of collecting road roughness data in the United States is between $1.4 ($2.23) and $6.2 per km ($10 per mile), depending on the state. For example, in Illinois, which has 224,719 km (139,577 mi) of roads, the cost of data collection is $1.4 million annually. An attractive solution for measuring the functional condition of large road networks economically is to make use of the acceleration sensors which are built into the majority of smartphones to assess road roughness by relating measured vertical acceleration to the road’s longitudinal profile. Because smartphone ownership and use are widespread, an approach could be envisioned whereby the assessment of the condition of road networks is facilitated using crowd-sourced data.

To investigate the feasibility of using smartphone technology for road roughness assessment, the following methodology was used:

The International Roughness Index (IRI) is the most commonly used measure of road roughness and is computed from a road’s longitudinal profile using a mathematical procedure described by Sayers (1995). The mathematical procedure simulates a quarter-car model of a standardized vehicle travelling at a speed of $80\mathrm{km}/\mathrm{h}$ over the road profile (ASTM 2015). IRI is the unit of slope (e.g., meters per kilometer) calculated from the accumulated suspension motions of the quarter-car model divided by the length of the road assessed. ASTM requires the IRI to be calculated over a minimum distance of 160 m (0.1 mi) (ASTM 2012). $\mathrm{IRI}\le 2.5\mathrm{m}/\mathrm{km}$ is considered to represent roads in very good condition, $2.5<\mathrm{IRI}\le 3.5\mathrm{m}/\mathrm{km}$ represents roads in good condition, $3.5<\mathrm{IRI}\le 6\mathrm{m}/\mathrm{km}$ represents roads in fair condition, and $6<\mathrm{IRI}\le 10\mathrm{m}/\mathrm{km}$ represents roads in poor condition (Archondo-Callao 2008). Commercially available devices for measuring road roughness are categorized into four levels according to their accuracy (Table 1) (Sayers et al. 1986). The most accurate of these devices (i.e., Class 1 and Class 2 devices) determine a road section’s IRI by measuring the road section’s profile, typically with a laser, and converting the profile to an IRI value using the procedure described by Sayers (1995). The World Bank introduced the concept of information quality levels (IQLs) as a guide to collecting data to inform road management decision-making (Paterson and Scullion 1990). Four IQLs are specified. IQL-1 and IQL-2 are associated with the highest level of accuracy and typically are used for research and project-level analyses, respectively. Information at IQL-3 and IQL-4 is at a coarser level of detail and is recommended for programming and strategic planning (i.e., network-level). By analogy with the information presented in Table 1, Class 1 devices could be used to collect IQL-1 information, Class 2 devices could be used to collect IQL-2–IQL-3 information and Class 3 devices could be used to collect IQL-3–IQL-4 information. Class 4 devices are suitable only for collecting IQL-4 information on roads with high inherent roughness such as gravel or earth roads.

Modern smartphones have in-built three-axis accelerometers and GPS capabilities. The measurement of road roughness utilizes the accelerometer to measure vertical vehicle-body acceleration. Discrete defects, such as potholes, can be recognized by identifying relatively large perturbations in continuous acceleration recordings. Road roughness can be determined by transforming the continuous vertical acceleration recordings and matching these with the GPS-determined locations (Buttlar and Islam 2014). A variety of smartphone-based applications for measuring road condition have been developed using an empirical approach. Table 2 summaries nine commonly available applications. Despite the availability and use of approaches such as those listed in Table 2, their accuracy and degree to which they can measure roughness and identify other defects, such as potholes, in real time over large road networks is unclear. Furthermore, because the models are empirical, they may be valid only for a narrow range of vehicle classes. Moreover, empirical studies trialing such systems suggested that they may be regarded as Class 2 or 3 devices (Scholotjes, Visser and Bennett 2014; Chugh et al. 2014). This is partly because a large number of factors need to be taken into account when determining road roughness from acceleration readings determined from a smartphone. These factors are associated with the smartphones themselves, vehicle-related aspects, and the form of the road surface profile.

Mathematical models based on a multivariate linear regression analysis and machine learning were developed to predict road roughness from vehicle body vertical acceleration. The purpose of developing the models was to identify the most suitable model forms and the influencing variables which need to be included in the development of similar predictive models, and to gauge the accuracy which might be expected from the models. The models were developed from a data set produced from 6,000 CarSim simulations of two vehicle types travelling over road sections with different road profiles. The accuracies of the developed models were determined by comparing the IRI values predicted by the models with the IRI values (i.e., target variable values) of the road profiles determined using the industry-standard approach used to determine IRI from data collected by devices which measure the road profile (Sayers 1995). For the simulation, a large European van and a D-class sedan were modeled and subject to the variables which were found to have a high or moderate impact on vertical vehicle body acceleration (Table 8). The variables considered in the simulation are given in Table 9. The data obtained from the CarSim simulation were split into three separate data sets, one associated with the large European van only, one associated with the D-class sedan, and a third which combined the data from both data sets. The vertical acceleration data were converted to Grms values as discussed previously.

Table 10 gives the results of the multivariate regression analysis; the data therein were used to develop a linear model for a large European van (Eq. 2), a linear model for a D-class sedan (Eq. 3), and a linear model which does not include the vehicle type (Eq. 4)where SE = standard error. The coefficient of multiple determination ($R^2$) of the three linear models was 0.83 for the LEV model, 0.79 for the DSD model, and 0.74 for the model which does not take into account vehicle type. Compared with the target variable values of IRI determined from the road profile using the standard method given by Sayers (1995), the standard error of the three linear models was 1.39, 1.54, and $1.69\mathrm{m}/\mathrm{km}$, respectively. Commonality analysis was carried out to determine how much variance in predicted IRI each of the six variables uniquely contribute (i.e., the significance of the variable). The results show that Grms contributes 82% of the variance in IRI in the case of the LEV and 79% for the DSD (Table 10). The Pearson’s correlation also indicated strong correlation between Grms and the target IRI values (0.81 and 0.83 respectively). Thus, as is self-evident, the simulated Grms is the dominant factor influencing the prediction of IRI from the simulated data sets. Speed was found to have the second largest influence on the prediction of IRI, accounting for 14% of the variance in the case of the DSD and 10% for the LEV. The other influencing variables do not appear to significantly affect the prediction of IRI when using multivariate linear regression.

Accordingly, simplified regression models which predict IRI from Grms, speed, and vehicle type were developed as follows:

For a large European van

For a D-class sedan

Ignoring vehicle type

The values of the $R^2$ and SE of the preceding sets of equations which consider all the influencing variables, and that which considers speed alone, suggest that multivariate linear regression could be used to determine IRI to a similar degree of accuracy as might be expected from a visual inspection (Table 1), i.e., IQL-4, even without taking into account the class of vehicle. However, considering all of the influencing variables does not seem to improve the results to an extent which would allow roughness to be measured to a higher IQL.

Three machine learning algorithms based on a decision regression tree, an artificial neural network (ANN), and a random forest were developed. The three machine learning algorithms were described by (Han et al. 2011). The three model types were chosen because their structures (i.e., dendritic and neuron based) have been shown to be applicable to large and complex data sets, yet they can be trained quickly using a relatively small sample of test data (without overfitting) to produce accurate predictions (Chandra et al. 2012; Melhem and Cheng 2003). Furthermore, the model types have been used successfully in related applications which predict road condition from field measurements (Jang et al. 2016; Seyfi et al. 2013; Nitsche et al. 2012; Soleimani and Sahebi 2012).

A classification and regression tree (CART) decision regression tree type, as advocated by Breiman et al. (1984), was selected with an unlimited tree depth and no pruning. The ANN utilized the RProp algorithm described by Riedmiller and Braun (1993) and consisted of two hidden layers with four neurons per layer. For each analysis the RProp algorithm was iterated 500 times. The random forest algorithm was of the form suggested by Breiman et al. (1984) and had two hundred trees with an unlimited tree depth.

The values chosen for the aforementioned parameters (number and depth of trees, number of layers, and so forth) for the three algorithms were determined on a trial-and-error basis to achieve a reasonable balance between model accuracy and the length of the required training time.

A 10-fold cross-validation process was used to train and test the machine learning algorithms. Such a validation process strongly increases the randomness (i.e., variance) in the data modeling and thereby reduces the likelihood of the model achieving a good accuracy by chance (Han et al. 2011). The validation process consisted of portioning the LEV, DSD, and combined data sets randomly into 10 equal-sized subsets. Each machine learning algorithm was trained using nine subsets, and the remaining subset was used for testing. This process was repeated a further nine times until all subsets were tested. The results of the 10 different trials were averaged to give an indication of the overall performance of each algorithm. Table 11 presents the results of this analysis for the three data sets. The three machine learning algorithms performed better than the regression approach described previously, as evidenced by higher $R^2$ and lower RMS error (RMSE) values. Furthermore, the neural network and random forest performed better for all three data sets than did the decision tree.

A feature elimination approach was used to remove each variable in turn in order to better understand the influence of the six selected variables on the prediction of IRI. The results of this process also are given in Table 11. The simulated Grms, as expected, was found to have the greatest influence on predicting IRI. Speed was found to be the second most important variable for all three data sets. For example, when speed was not taken into account, for the LEV data set the RMSE increased from 0.91 to $1.74\mathrm{m}/\mathrm{km}$ and the $R^2$ value decreased from 0.92 to 0.72. In comparison, the four other variables had a relatively minor influence on the prediction of IRI. This supports the findings of the regression analysis described previously.

Comparing the performance of all three machine learning algorithms in analyzing the three data sets indicated that the ANN and random forest algorithms were better predictors of IRI than was the decision regression tree. Both algorithms performed better for the LEV and DSD data sets than for the combined data set. Removing the vehicle type variable from the analysis using the ANN and random forest algorithms increased the RMSE by between 3% and 29%. For both algorithms, the RMSE for the LEV data set was higher than for the DSD data set. This suggests that it is important to take into account vehicle type when predicting IRI. However, removing all variables that were found from the parametric study to have a medium influence on IRI prediction, i.e., number of people, stiffness, damper force, and tire pressure (Table 11), did not affect the error in the prediction. For example, for the neural network model when considering the LEV and combined data sets, the RMSE value increased by 27% (0.91–$1.16\mathrm{m}/\mathrm{km}$) and by 17% (0.97–0.94), respectively.

The suitability of using a smartphone suitably fixed inside a moving vehicle to determine road roughness (IRI) was assessed by reviewing existing approaches using a parametric study carried out with the aid of CarSim, a vehicle dynamics package. CarSim has been shown to accurately replicate actual vehicle behavior, and its use allowed a large number of controlled simulations to be carried out in a relatively short time.

It was found that an important aspect of measuring road roughness with a smartphone is to convert the smartphone’s vertical acceleration data to RMS body accelerations (i.e., Grms). The simulation results showed that because Grms is strongly correlated with the actual road IRI in a real-world system, the calculation of the Grms of potentially millions of measured acceleration signals accruing from road users requires less computational processing than approaches that require the additional transformation of the data to the frequency domain.

The parametric study showed that vehicle speed and type and Grms (i.e., vertical vehicle body acceleration) had a large effect ($>30\%$) on the assessment of IRI. Sprung mass, suspension stiffness, longitudinal profiles giving the same IRI, and tire pressure had a moderate effect (i.e., between 10% and 30%), whereas acceleration/braking and suspension damping had a relatively minor influence ($<10\%$). Multivariate linear regression and machine learning algorithms were developed to predict IRI from the vertical acceleration (i.e., Grms) values obtained from the CarSim simulation. The results suggest that a multivariate linear regression which takes into account vehicle speed could be used to predict IRI to an accuracy equivalent to that which might be expected by visual observation (i.e., a Class 4 device at IQL-4). However, using a machine learning algorithm as part of a suitable smartphone-based system could enable road roughness to be predicted in accordance with the requirements of a Class 3 device and to an IQL of 3 (Table 1). For example, the ANN that was trialed was able to predict road roughness to a RMSE of between 0.73 and $0.91\mathrm{m}/\mathrm{km}$ when vehicle class and speed were considered. Data at IQL-3, and at the frequency of collection possible with a smartphone, would enable long-term strategic road management decision-making. Such decision-making enabled by roughness data collected using such a smartphone system would allow for (Sayers et al. 1986)

To support such strategic road asset management, the routine inspection of the condition of a road network could be achieved using low-cost data collection systems which utilize smartphones with similar characteristics inside a fleet of vehicles of similar types, travelling at normal traffic speeds.

Without knowing the vehicle type and speed, vertical acceleration data obtained from smartphones could be analyzed using machine learning algorithms to enable IRI to be predicted to a similar accuracy as would be expected from a visual inspection, but with arguably improved repeatability and reproducibility. Such data could be suitable for the routine analysis of the condition of local road networks. A particularly useful application could be the assessment of the condition of low-volume rural road networks in developing countries where the majority of rural roads are constructed from either gravel or earth and where smartphone ownership is surprisingly high (World Bank Group 2016).

It is recognized that the work presented herein is based on simulation, and therefore the work has a number of possible limitations. An objective of the research was to compare the simulated vertical accelerations measured by a smartphone of a vehicle travelling on actual road sections, with the roughness of the road sections determined from a profile of the road using a standard method (Sayers 1995). The former was simulated using the CarSim vehicle dynamics package. The road roughness values of the road sections were computed using a standard procedure developed by Sayers (1995) which is used by the road data collection industry to determine IRI from data collected by devices which measure the road profile. CarSim and the method provided by Sayers (1995) use models of vehicle motion to determine the vertical vehicle body acceleration and the IRI, respectively, from the summation of model vertical displacements. CarSim replicates the behavior of actual vehicles, whereas the quarter-car model used by Sayers (1995) is a simplified model of a standardized vehicle. Although CarSim has been shown by independent studies to closely replicate the performance of real vehicles, the components of real vehicles that might affect vehicle vertical body acceleration will vary because of age, use, and maintenance (i.e., suspension stiffness, damping, tire pressure, and mass). The combined effects of these parameters on the vertical vehicle accelerations for a single vehicle class were not considered in this work. Furthermore, the work presented assumed that the simulated vertical vehicle body accelerations were equivalent to the accelerations measured by a smartphone fixed inside a vehicle. However, the simulation did not replicate the effect of varying the mounting system nor the position of the smartphone in a vehicle. These collectively can affect the recorded vertical acceleration values by $\pm 15\%$ (Kropáč and Múčka 2005; Belzowski and Ekstrom 2015). For a practical system, the smartphone should be adhered to the windshield with a suction cup and a bracket that attaches to the dashboard to ensure a rigid mounting, preventing the smartphone from moving freely.

The following conclusions can be drawn from the research:

These factors notwithstanding, although the algorithms presented for the analysis of vertical vehicle body acceleration demonstrated promising results for the artificial data sets developed for the research, clearly they need further testing using data obtained in the field, calibration, and refinement to the conditions at hand for their practical application.

The practical support of the School of Engineering, at the University of Birmingham is noted with gratitude.