Let's Connect
Follow Us
Watch Us
(+385) 1 2380 262
journal.prometfpz.unizg.hr
Promet - Traffic&Transportation journal

Accelerating Discoveries in Traffic Science

Accelerating Discoveries in Traffic Science

Login Submit

PDF
PUBLISHED
30.09.2022
DOI
https://doi.org/10.7307/ptt.v34i5.4069
ISSUE
2022 (Vol 34), Issue 5
LICENSE
Copyright (c) 2024 Bing Tang, Yao Hu, Huan Chen
Creative Commons 4.0 logo
This work is licensed under a
Creative Commons Attribution-NonCommercial-
NoDerivatives 4.0 International License.

A Functional Data Approach to Outlier Detection and Imputation for Traffic Density Data on Urban Arterial Roads

Authors:Bing Tang, Yao Hu, Huan Chen

Abstract

In traffic monitoring data analysis, the magnitude of traffic density plays an important role in determining the level of traffic congestion. This study proposes a data imputation method for spatio-functional principal component analysis (s-FPCA) and unifies anomaly curve detection, outlier confirmation and imputation of traffic density at target intersections. Firstly, the detection of anomalous curves is performed based on the binary principal component scores obtained from the functional data analysis, followed by the determination of the presence of outliers through threshold method. Secondly, an improved method for missing traffic data estimation based on upstream and downstream is proposed. Finally, a numerical study of the actual traffic density data is carried out, and the accuracy of s-FPCA for imputation is improved by 8.28%, 8.91% and 7.48%, respectively, when comparing to functional principal component analysis (FPCA) with daily traffic density data missing rates of 5%, 10% and 20%, proving the superiority of the method. This method can also be applied to the detection of outliers in traffic flow, imputation and other longitudinal data analysis with periodic fluctuations.

Keywords:functional data, functional principal component analysis, traffic density, outliers, s-FPCA

References

  1. [1] Arasan VT, Dhivya G. Methodology for Determination of Concentration of Heterogeneous Traffic. Journal of Transportation Systems Engineering and Information Technology. 2010;10(4). doi: 10.1016/S1570-6672(09)60052-0.
  2. [2] Ramsay JO, Silverman BW. Functional Data Analysis. New York: Springer; 2005.
  3. [3] Wang J-L, et al. Functional data analysis. Annual Review of Statistics and Its Application. 2016;3: 257-295. doi: 10.1146/annurev-statistics-041715-033624.
  4. [4] Chiou J-M. Dynamical functional prediction and classification, with application to traffic flow prediction. The Annals of Applied Statistics. 2012;6(4). doi: 10.1214/12-AOAS595.
  5. [5] Chiou J-M, et al. A functional data approach to missing value imputation and outlier detection for traffic flow data. Transportmetrica B: Transport Dynamics. 2014;2(2). doi: 10.1080/21680566.2014.892847.
  6. [6] Li PL, Chiou J-M. Functional clustering and missing value imputation of traffic flow trajectories. Transportmetrica B: Transport Dynamics. 2020;9(1). doi:10.1080/21680566.2020.1781706.
  7. [7] Mu W. Application of functional data anomaly detection in spectral data. Xiamen University, 2019.
  8. [8] Chen J. Improvement and application of abnormal value diagnosis method of functional data. Jiangxi University of Finance and Economics. 2020. doi: 10.27175/d.cnki. gjxcu. 2020.000382.
  9. [9] Hyndman RJ, Shang HL. Rainbow plots, bagplots, and boxplots for functional data. Journal of Computational and Graphical Statistics. 2010;19(1). doi: 10.1198/jcgs.2009.08158.
  10. [10] Mondal MA, Rehena Z. Road traffic outlier detection technique based on linear regression. Procedia Computer Science. 2020;171(C): 2537-2555. doi: 10.1016/j.procs.2020.04.276.
  11. [11] Pu J, et al. STLP-OD: Spatial and temporal label propagation for traffic outlier detection. IEEE Access. 2019;(7): 63036-63044. doi: 10.1109/ACCESS.2019.2916853.
  12. [12] Chen K, Zou Q. [A traffic flow anomaly mining method incorporating time-correlated factor curve fitting]. Computer Engineering and Design. 2013;34(07): 2561-2565. Chinese.
  13. [13] Lu M-W, et al. [A traffic data pre-processing method based on curve-fitting anomaly detection]. Database Professional Committee of China Computer Society; 2006. p. 642-646. Chinese.
  14. [14] Schafer JL, Graham JW. Missing data: Our view of the state of the art. Psychological Methods. 2002;7(2): 147-177. doi: 10.1037/1082-989X.7.2.147.
  15. [15] Collins LM, Schafer JL, Kam CM. A comparison of inclusive and restrictive strategies in modern missing data procedures. Psychological Methods. 2001;6(4): 330-351. doi: 10.1037/1082-989X.6.4.330.
  16. [16] Rubin DB. Multiple imputation for nonresponse in surveys. John Wiley & Sons; 2004. p. 81.
  17. [17] Booth DE. Analysis of incomplete multivariate data. Technometrics. 2000;42(2): 213-214 doi: 10.1080/00401706.2000.10486013.
  18. [18] Schlittgen R. Analysis of incomplete multivariate data. Computational Statistics and Data Analysis. 1999;30(4): 478-479. doi: 10.1016/S0167-9473(99)90025-7.
  19. [19] Beale EML, Little RJA. Missing values in multivariate analysis. Journal of the Royal Statistical Society: Series B (Methodological). 1975;37(1): 129-145. doi: 10.1111/j.2517-6161.1975.tb01037.x.
  20. [20] Laird NM. Missing data in longitudinal studies. Statistics in Medicine. 1988;7(1-2): 305-315. doi: 10.1002/sim.4780070131.
  21. [21] Little RJA. Modeling the drop-out mechanism in repeated-measures studies. Journal of the American Statistical Association. 1995;90(431): 1112-1121.
  22. [22] Little RJA, Rubin DB. Statistical analysis with missing data. John Wiley & Sons, Inc; 2002. doi: 10.1002/9781119013563.
  23. [23] Molenberghs G. Applied longitudinal analysis. Journal of the American Statistical Association. 2005;100(470). doi: 10.1198/jasa.2005.s24.
  24. [24] Nihan NL. Aid to determining freeway metering rates and detecting loop errors. Journal of Transportation Engineering. 1997;123(6): 454-458. doi: 10.1061/(ASCE)0733-947X(1997)123:6(454).
  25. [25] Chen C, et al. Detecting errors and imputing missing data for single-loop surveillance systems. Transportation Research Record. 2003;1855(1): 160-167. doi: 10.3141/1855-20.
  26. [26] Zhong M, Sharma S, Lingras P. Genetically designed models for accurate imputation of missing traffic counts. Transportation Research Record. 2004;1879(1): doi: 10.3141/1879-09.
  27. [27] Zhang W-B, et al. [Traffic flow data restoration model for road networks based on self-attention mechanism and graph self-encoder]. Transportation Systems Engineering and Information. 2021;21(04): 90-98. doi: 10.16097/j.cnki.1009-6744.2021.04.011. Chinese.
  28. [28] Lu W-Q, et al. [Lane level traffic flow data restoration algorithm based on tensor decomposition theory]. Journal of Jilin University (Engineering and Technology Edition). 2021;51(05): 1708-1715. doi: 10.13229/j.cnki.jdxbgxb20200535. Chinese.
  29. [29] Li L, et al. Estimation of missing values in heterogeneous traffic data: Application of multimodal deep learning model. Knowledge-Based Systems. 2020;194: 105592. doi: 10.1016/j.knosys.2020.105592.
Show more

SPECIAL ISSUE CALL

Innovation and New Technologies in Transport and Logistics

Guest Editor: Eleonora Papadimitriou, PhD

Editors: Marko Matulin, PhD; Dario Babić, PhD; Marko Ševrović, PhD.

RECENT PAPERS

Browse through the selection of our recent research
Risk Analysis of Level Crossing Element Failures in a Fuzzy EnvironmentA Study on the Impact of Overtaking Lane-Changing Behaviour in Expressway Interchange Weaving AreasA Deep Learning Approach for Enhanced Real-Time Prediction of Winter Road Surface Temperatures in High-Altitude Mountain AreasBibliometric Analysis of Traffic Accident Prediction Studies from 2003 to 2023: Trends, Patterns and Future Directions

Accelerating Discoveries in Traffic Science |
2024 © Promet - Traffic&Transportation journal