Comparison of Data Reduction Algorithms for Real-Time Data Processing in Embedded Systems
DOI:
https://doi.org/10.58190/imiens.2024.113Keywords:
big-data processing, data reduction, embedded systems, real-time processAbstract
In embedded systems, large datasets are difficult to process in real-time due to limited processing power, memory capacity, and energy resources. In order to solve these difficulties, the use of algorithms that reduce data size and complexity has become a critical requirement. This study examines the techniques of five algorithms used for data reduction in embedded systems. The techniques of dimensionality reduction, numerosity reduction, data compression, data cube aggregation, and discretization algorithms are applied to a dataset. The dataset consists of load and angle data recorded every five seconds for three months. The selected data reduction techniques are evaluated to reduce data processing load, optimize storage requirements, and reduce energy consumption. The results show that each algorithm offers advantages according to different application requirements. The findings obtained in this study provide a guiding framework for the optimization of data processing processes in embedded systems. The results provide important information that can help system designers select algorithms suitable for application requirements. In the future, combining these algorithms with hybrid approaches can further increase the data processing capacity and efficiency of embedded systems.
Downloads
References
[1] M. Jiménez, R. Palomera, and I. Couvertier, Introduction to embedded systems. Springer, 2013.
[2] H. Kopetz and W. Steiner, Real-time systems: design principles for distributed embedded applications. Springer Nature, 2022.
[3] P. Marwedel, Embedded system design: embedded systems foundations of cyber-physical systems, and the internet of things. Springer Nature, 2021.
[4] P. Louridas and C. Ebert, "Embedded analytics and statistics for big data," IEEE software, vol. 30, no. 6, pp. 33-39, 2013.
[5] M. Wu, "Wearable technology applications in healthcare: a literature review," On-Line Journal of Nursing Informatics, vol. 23, no. 3, 2019.
[6] M. A. Wulder et al., "Lidar sampling for large-area forest characterization: A review," Remote sensing of environment, vol. 121, pp. 196-209, 2012.
[7] P. R. Evans, "An introduction to data reduction: space-group determination, scaling and intensity statistics," Acta Crystallographica Section D: Biological Crystallography, vol. 67, no. 4, pp. 282-292, 2011.
[8] P. M. Gleason, C. J. Boushey, J. E. Harris, and J. Zoellner, "Publishing nutrition research: a review of multivariate techniques—part 3: data reduction methods," Journal of the Academy of Nutrition and Dietetics, vol. 115, no. 7, pp. 1072-1082, 2015.
[9] J. P. Cunningham and B. M. Yu, "Dimensionality reduction for large-scale neural recordings," Nature neuroscience, vol. 17, no. 11, pp. 1500-1509, 2014.
[10] K. Kalegele, H. Takahashi, J. Sveholm, K. Sasai, G. Kitagata, and T. Kinoshita, "On-demand data numerosity reduction for learning artifacts," in 2012 IEEE 26th International Conference on Advanced Information Networking and Applications, 2012: IEEE, pp. 152-159.
[11] K. Sayood, Introduction to data compression. Morgan Kaufmann, 2017.
[12] D. P. S. Ghazali, R. Latip, M. Hussin, and M. H. Abd Wahab, "A review data cube analysis method in big data environment," ARPN J. Eng. Appl. Sci, vol. 10, pp. 8525-8532, 2015.
[13] S. Ramírez‐Gallego et al., "Data discretization: taxonomy and big data challenge," Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery, vol. 6, no. 1, pp. 5-21, 2016.
[14] A. Papageorgiou, B. Cheng, and E. Kovacs, "Real-time data reduction at the network edge of Internet-of-Things systems," in 2015 11th international conference on network and service management (CNSM), 2015: IEEE, pp. 284-291.
[15] E. A. Lee and S. A. Seshia, Introduction to embedded systems: A cyber-physical systems approach. MIT press, 2017.
[16] D. Hatley and I. Pirbhai, Strategies for real-time system specification. Addison-Wesley, 2013.
[17] N. Salem and S. Hussein, "Data dimensional reduction and principal components analysis," Procedia Computer Science, vol. 163, pp. 292-299, 2019.
[18] D. Quick and K.-K. R. Choo, "Big forensic data reduction: digital forensic images and electronic evidence," Cluster Computing, vol. 19, pp. 723-740, 2016.
[19] Y. Wang, H. Yao, and S. Zhao, "Auto-encoder based dimensionality reduction," Neurocomputing, vol. 184, pp. 232-242, 2016.
[20] A. Rossi, A. Visentin, S. Prestwich, and K. N. Brown, "Clustering-Based Numerosity Reduction for Cloud Workload Forecasting," in International Symposium on Algorithmic Aspects of Cloud Computing, 2023: Springer, pp. 115-132.
[21] D. Salomon and G. Motta, Handbook of data compression. Springer Science & Business Media, 2010.
Downloads
Published
Issue
Section
License
Copyright (c) 2024 Intelligent Methods In Engineering Sciences
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
