LSTM based Anomaly Detection in Time Series for United States exports and imports

Aggarwal, Sakshi (2023): LSTM based Anomaly Detection in Time Series for United States exports and imports.

Abstract

This survey aims to offer a thorough and organized overview of research on anomaly detection, which is a significant problem that has been studied in various fields and application areas. Some anomaly detection techniques have been tailored for specific domains, while others are more general. Anomaly detection involves identifying unusual patterns or events in a dataset, which is important for a wide range of applications including fraud detection and medical diagnosis. Not much research on anomaly detection techniques has been conducted in the field of economic and international trade. Therefore, this study attempts to analyze the time-series data of United Nations exports and imports for the period 1992 – 2022 using LSTM based anomaly detection algorithm. Deep learning, particularly LSTM networks, are becoming increasingly popular in anomaly detection tasks due to their ability to learn complex patterns in sequential data. This paper presents a detailed explanation of LSTM architecture, including the role of input, forget, and output gates in processing input vectors and hidden states at each timestep. The LSTM based anomaly detection approach yields promising results by modelling small-term as well as long-term temporal dependencies.

• Item Type: MPRA Paper
• Language: English
• Keywords: Anomaly detection, LSTM, Machine learning, Artificial intelligence, economic trade
• Subjects: C - Mathematical and Quantitative Methods > C5 - Econometric Modeling > C54 - Quantitative Policy Modeling; F - International Economics > F1 - Trade > F13 - Trade Policy ; International Trade Organizations; F - International Economics > F1 - Trade > F15 - Economic Integration

References

Agarwal, A., Dawson, S., McKee, D., Eugster, P., Tancreti, M., & Sundaram, V. (2017). Detecting abnormalities in IoT program executions through control-flow-based features. In Proceedings of the Second International Conference on Internet-of-Things Design and Implementation (pp. 339-340). Aggarwal, C. C. (2013). Outlier analysis. New York, NY: Springer. Aggarwal, S. (2016). Determinants of money demand for India in presence of structural break: An empirical analysis. Business and Economic Horizons (BEH), Prague Development Center (PRADEC), 12(4), 173-177. Aggarwal, S. (2017a). Smile curve and its linkages with global value chains. Journal of Economic Bibliography, 4(3). Aggarwal, S. (2017b). Sectoral Level Analysis of India’s Bilateral Trade over 2001-2015. MPRA Paper No 80099, University Library of Munich, Germany. Retrieved from https://mpra.ub.uni-muenchen.de/80099/ (Accessed on March 10, 2023) Aggarwal, S. (2020). Determinants of Intra-Industry Trade and Labour Market Adjustment: A Sectoral Analysis for India (Doctoral dissertation, Indian Institute of Foreign Trade). Aggarwal, S. (2023a). Machine Learning Algorithms, Perspectives, and Real - World Application: Empirical Evidence from United States Trade Data. International Journal of Science and Research, 12(3), pp. 292-313. https://www.ijsr.net/getabstract.php?paperid=SR23305084601 Aggarwal, S. (2023b). The empirical measurement and determinants of intra-industry trade for a developing country. MPRA Paper No. 117112, University Library of Munich, Germany. Retrieved from https://mpra.ub.uni-muenchen.de/117112/ (Accessed on April 21, 2023). Aggarwal, S., & Chakraborty, D. (2017). Determinants of India’s bilateral intra-industry trade over 2001–2015: Empirical results. South Asia Economic Journal, 18(2), 296–313. Aggarwal, S., & Chakraborty, D. (2019). Which factors influence India’s intra-industry trade? Empirical findings for select sectors. Global Business Review. Retrieved from https://journals.sagepub.com/doi/10.1177/0972150919868343 (Accessed on April 23, 2020). Aggarwal, S., & Chakraborty, D. (2020a). Labour market adjustment and intra-industry trade: Empirical results from Indian manufacturing sectors. Journal of South Asian Development, 15(2), 238-269. Aggarwal, S., & Chakraborty, D. (2020b). Determinants of vertical intra-industry trade: Empirical evidence from Indian manufacturing sectors. Prajnan: Journal of Social and Management Sciences, 49(3), 221-252. Aggarwal, S., & Chakraborty, D. (2020c). Is there any relationship between Marginal Intra-Industry Trade and Employment Change? Evidence from Indian Industries. Working Paper, No. EC-20-44, Indian Institute of Foreign Trade, Delhi. Aggarwal, S., Chakraborty, D., & Bhattacharyya, R. (2021). Determinants of Domestic Value Added in Exports: Empirical Evidence from India’s Manufacturing Sectors. Global Business Review. https://doi.org/10.1177/09721509211050138. Aggarwal, S., & Chakraborty, D. (2021). Which factors influence vertical intra-industry trade in India? Empirical results from panel data analysis. Working Paper, No. EC-21-54, Indian Institute of Foreign Trade, Delhi. Retrieved from http://cc.iift.ac.in/research/Docs/WP/EC-21-54.pdf (Accessed on March 20, 2023) Aggarwal, S., Chakraborty, D. (2022). Which Factors Influence India’s Bilateral Intra-Industry Trade? Cross-Country Empirical Estimates. Working Papers 2260, Indian Institute of Foreign Trade, Delhi. Aggarwal, S., Mondal, S., & Chakraborty, D. (2022). Efficiency Gain in Indian Manufacturing Sectors: Evidence from Domestic Value Addition in Exports. Empirical Economics Letters, 21(2): 69-83. Aggarwal, S., Chakraborty, D., & Banik, N. (2023). Does Difference in Environmental Standard Influence India’s Bilateral IIT Flows? Evidence from GMM Results. Journal of Emerging Market Finance, 22(1), 7–30. https://doi.org/10.1177/09726527221088412. Ahmed, M., Mahmood, A. N., & Hu, J. (2014). Outlier detection. In The state of the art in intrusion prevention and detection (Vol. 44, pp. 3-21). New York, NY, USA: CRC Press. Albelwi, S., & Mahmood, A. (2017). A framework for designing the architectures of deep convolutional neural networks. Entropy, 19(6), 242. Anderson, J. E. (1979). A theoretical foundation for the gravity equation. The American economic review, 69(1), 106-116. Barnett, V., & Lewis, T. (1994). Outliers in statistical data, 3(1). John Wiley and sons, New York. Batarseh, F., Gopinath, M., Nalluru, G., & Beckman, J. (2019). Application of machine learning in forecasting international trade trends. arXiv preprint arXiv:1910.03112. Beckman, R. J., & Cook, R. D. (1983). Outlier………. s. Technometrics, 25(2), 119-149. Bengio, Y., Courville, A., & Vincent, P. (2013). Representation learning: A review and new perspectives. IEEE transactions on pattern analysis and machine intelligence, 35(8), 1798-1828. Bhattacharya, K., Mukherjee, G., and Manna, S. S. (2007). The international trade network. New Economic Windows 139–147. Bolton, R. J., & Hand, D. J. (2001). Unsupervised profiling methods for fraud detection. Credit scoring and credit control VII, 235-255. Boukerche, A., Zheng, L., & Alfandi, O. (2020). Outlier detection: Methods, models, and classification. ACM Computing Surveys (CSUR), 53(3), 1-37. Callado, A., Kamienski, C., Szabó, G., Gero, B. P., Kelner, J., Fernandes, S., & Sadok, D. (2009). A survey on internet traffic identification. IEEE communications surveys & tutorials, 11(3), 37-52. Chalapathy, R., & Chawla, S. (2019). Deep learning for anomaly detection: A survey. arXiv preprint arXiv:1901.03407. Chandola, V., Banerjee, A., & Kumar, V. (2009). Anomaly detection: A survey. ACM computing surveys (CSUR), 41(3), 1-58. Charitou, C., Garcez, A. D. A., & Dragicevic, S. (2020). Semi-supervised GANs for fraud detection. International Joint Conference on Neural Networks (IJCNN) (pp. 1-8). IEEE. Chen, K., Lu, S. C., & Teng, H. S. (1990). Adaptive real-time anomaly detection using inductively generated sequential patterns. In Proceedings of IEEE Computer Society Symposium on Research in Security and Privacy. IEEE Computer Society Press, 278–284. Chen, J., Shen, Y., & Ali, R. (2018). Credit card fraud detection using sparse autoencoder and generative adversarial network. IEEE 9th Annual Information Technology, Electronics and Mobile Communication Conference (IEMCON) (pp. 1054-1059). IEEE. Fiore, U., De Santis, A., Perla, F., Zanetti, P., & Palmieri, F. (2019). Using generative adversarial networks for improving classification effectiveness in credit card fraud detection. Information Sciences, 479, 448-455. Fu, K., Cheng, D., Tu, Y., & Zhang, L. (2016). Credit card fraud detection using convolutional neural networks. In Neural Information Processing: 23rd International Conference, ICONIP 2016, Kyoto, Japan, October 16–21, 2016, Proceedings, Part III 23 (pp. 483-490). Springer International Publishing. Garcia-Teodoro, P., Diaz-Verdejo, J., Maciá-Fernández, G., & Vázquez, E. (2009). Anomaly-based network intrusion detection: Techniques, systems and challenges. computers & security, 28(1-2), 18-28. Gers, F. A., Schmidhuber, J., & Cummins, F. (2000). Learning to forget: Continual prediction with LSTM. Neural computation, 12(10), 2451-2471. Ghoddusi, H., Creamer, G. G., & Rafizadeh, N. (2019). Machine learning in energy economics and finance: A review. Energy Economics, 81, 709-727. Ghosh, S., & Reilly, D. L. (1994). Credit card fraud detection with a neural-network. In System Sciences, 1994. Proceedings of the Twenty-Seventh Hawaii International Conference on (Vol. 3, pp. 621-630). IEEE. Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., ... & Bengio, Y. (2014). Generative adversarial nets. arXiv. arXiv preprint arXiv:1406.2661. Gopinath, M., Batarseh, F. A., & Beckman, J. (2020). Machine learning in gravity models: An application to agricultural trade (No. w27151). National Bureau of Economic Research. Gupta, M., Gao, J., Aggarwal, C. C., & Han, J. (2013). Outlier detection for temporal data: A survey. IEEE Transactions on Knowledge and data Engineering, 26(9), 2250-2267. Hawkins, D. M. (1980). Identification of outliers (Vol. 11). Heidelberg, Germany: Springer. He, H., Wang, J., Graco, W., & Hawkins, S. (1997). Application of neural networks to detection of medical fraud. Expert systems with applications, 13(4), 329-336. Heryadi, Y., & Warnars, H. L. H. S. (2017). Learning temporal representation of transaction amount for fraudulent transaction recognition using CNN, Stacked LSTM, and CNN-LSTM. In 2017 IEEE International Conference on Cybernetics and Computational Intelligence (CyberneticsCom) (pp. 84-89). IEEE. Hochreiter, S., & Schmidhuber, J. (1997). Long short-term memory. Neural Computation, 9(8), 1735-1780. Hodge, V., & Austin, J. (2004). A survey of outlier detection methodologies. Artificial intelligence review, 22, 85-126. Javaid, A., Niyaz, Q., Sun, W., & Alam, M. A. (2016). A deep learning approach for network intrusion detection system, In: proceedings of the 9th EAI International Conference on bio-inspired information and communications technologies (formerly BIONETICS). formerly BIONETICS). Jurgovsky, J., Granitzer, M., Ziegler, K., Calabretto, S., Portier, P. E., He-Guelton, L., & Caelen, O. (2018). Sequence classification for credit-card fraud detection. Expert Systems with Applications, 100, 234-245. Kazemi, Z., & Zarrabi, H. (2017, December). Using deep networks for fraud detection in the credit card transactions. IEEE 4th International conference on knowledge-based engineering and innovation (KBEI) (pp. 0630-0633). Tehran, Iran. Khan, A. U. S., Akhtar, N., & Qureshi, M. N. (2014). Real-time credit-card fraud detection using artificial neural network tuned by simulated annealing algorithm. In Proceedings of international conference on recent trends in information, telecommunication and computing, ITC (pp. 113-121). Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2012). ImageNet Classification with Deep Convolutional Networks. 25th Advances in Neural Information Processing Systems. Grenada, Spain. Kucharski, D., Kleczek, P., Jaworek-Korjakowska, J., Dyduch, G., & Gorgon, M. (2020). Semi-supervised nests of melanocytes segmentation method using convolutional autoencoders. Sensors, 20(6), 1546. LeCun, Y., & Bengio, Y. (1995). Convolutional networks for images, speech, and time series. The handbook of brain theory and neural networks, 3361(10), 1995. LeCun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998). Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(11), 2278-2324. Lin, J., Keogh, E., Fu, A., & Van Herle, H. (2005). Approximations to magic: Finding unusual medical time series. In 18th IEEE Symposium on Computer-Based Medical Systems (CBMS'05) (pp. 329-334). IEEE. Maes, S., Tuyls, K., Vanschoenwinkel, B., & Manderick, B. (2002). Credit card fraud detection using Bayesian and neural networks. In Proceedings of the 1st international naiso congress on neuro fuzzy technologies (Vol. 261, p. 270). Malhotra, P., Vig, L., Shroff, G., & Agarwal, P. (2015). Long Short Term Memory Networks for Anomaly Detection in Time Series. In ESANN (Vol. 2015, p. 89). Markou, M., & Singh, S. (2003a). Novelty detection: a review—part 1: statistical approaches. Signal processing, 83(12), 2481-2497. Markou, M., & Singh, S. (2003b). Novelty detection: a review—part 2: neural network-based approaches. Signal processing, 83(12), 2499-2521. Michelucci, U. (2018). Feedforward neural networks. Applied Deep Learning: A Case-Based Approach to Understanding Deep Neural Networks, 83-136. Miljković, D. (2010). Review of novelty detection methods. In The 33rd International Convention MIPRO (pp. 593-598). IEEE. Mirza, M., & Osindero, S. (2014). Conditional generative adversarial nets. arXiv preprint arXiv:1411.1784. Monken, A., Haberkorn, F., Gopinath, M., Freeman, L., & Batarseh, F. A. (2021). Graph neural networks for modeling causality in international trade. In The International FLAIRS Conference Proceedings (Vol. 34). Nag, B., Chakraborty, D., & Aggarwal, S. (2021). India's Act East Policy: RCEP Negotiations and Beyond. Working Paper, No. EC-21-01, Indian Institute of Foreign Trade, Delhi. Noble, C. C., & Cook, D. J. (2003). Graph-based anomaly detection. In Proceedings of the ninth ACM SIGKDD international conference on Knowledge discovery and data mining (pp. 631-636). Nguyen, T. T., & Armitage, G. (2008). A survey of techniques for internet traffic classification using machine learning. IEEE communications surveys & tutorials, 10(4), 56-76. O'Shea, K., & Nash, R. (2015). An introduction to convolutional neural networks. arXiv preprint arXiv:1511.08458. Panford-Quainoo, K., Kigali, R., Bose, A. J., & Defferrard, M. (2020). Bilateral Trade Modeling with Graph Neural Networks. In ICLR Workshop on Practical ML for Developing Countries. Park, J. (2018). RNN based time-series anomaly detector model implemented in Pytorch. Published in website URL: https://github. com/chickenbestlover/RNN-Time-series-Anomaly-Detection. Patcha, A., & Park, J. M. (2007). An overview of anomaly detection techniques: Existing solutions and latest technological trends. Computer networks, 51(12), 3448-3470. Paula, E. L., Ladeira, M., Carvalho, R. N., & Marzagao, T. (2016). Deep learning anomaly detection as support fraud investigation in brazilian exports and anti-money laundering. In 2016 15th ieee international conference on machine learning and applications (icmla) (pp. 954-960). IEEE. Peng, H. K., & Marculescu, R. (2015). Multi-scale compositionality: identifying the compositional structures of social dynamics using deep learning. PloS one, 10(4), e0118309. Pimentel, M. A., Clifton, D. A., Clifton, L., & Tarassenko, L. (2014). A review of novelty detection. Signal processing, 99, 215-249. Pourhabibi, T., Ong, K. L., Kam, B. H., & Boo, Y. L. (2020). Fraud detection: A systematic literature review of graph-based anomaly detection approaches. Decision Support Systems, 133, 113303. Pumsirirat, A., & Liu, Y. (2018). Credit card fraud detection using deep learning based on auto-encoder and restricted boltzmann machine. International Journal of advanced computer science and applications, 9(1), 18-25. Rosseeuw, P. J., & Leroy, A. M. (1987). Robust regression and outlier detection. John Willey & Sons. Inc., New York. S. 1558 – AIIA (2020). Artificial Intelligence Initiative Act. U.S. Congressional Senate Committee on Commerce, Science, and Transportation. Saunders, R., & Gero, J. S. (2000). The importance of being emergent. In Proceedings of Artificial Intelligence in Design. Sperotto, A., Schaffrath, G., Sadre, R., Morariu, C., Pras, A., & Stiller, B. (2010). An overview of IP flow-based intrusion detection. IEEE communications surveys & tutorials, 12(3), 343-356. Sun, B., Yu, F., Wu, K., Xiao, Y., & Leung, V. C. (2006). Enhancing security using mobility-based anomaly detection in cellular mobile networks. IEEE Transactions on Vehicular Technology, 55(4), 1385-1396. Sun, B., Osborne, L., Xiao, Y., & Guizani, S. (2007a). Intrusion detection techniques in mobile ad hoc and wireless sensor networks. IEEE Wireless Communications, 14(5), 56-63. Sun, B., Xiao, Y., & Wang, R. (2007b). Detection of fraudulent usage in wireless networks. IEEE Transactions on Vehicular Technology, 56(6), 3912-3923. Sun, B., Wu, K., Xiao, Y., & Wang, R. (2007c). Integration of mobility and intrusion detection for wireless ad hoc networks. International Journal of Communication Systems, 20(6), 695-721. Tanaka, F. H., & Aranha, C. (2019). Data Augmentation Using GANs. Retrieved from arXiv:1904.09135. United States Central Bureau (undated), " U.S. International Trade in Goods and Services", available at: https://www.census.gov/foreign-trade/statistics/historical/index.html (accessed March 01, 2023). United Nations Conference on Trade and Development (2019). World Investment Report, Geneva: WTO. United Nations Conference on Trade and Development (2022). Trade and Development Report, Geneva: WTO. Wang, C., Wang, Y., Ye, Z., Yan, L., Cai, W., & Pan, S. (2018). Credit card fraud detection based on whale algorithm optimized BP neural network. 13th international conference on computer science & education (ICCSE) (pp. 1-4). IEEE. Wang, X., Du, Y., Lin, S., Cui, P., Shen, Y., & Yang, Y. (2020). adVAE: A self-adversarial variational autoencoder with Gaussian anomaly prior knowledge for anomaly detection. Knowledge-Based Systems, 190, 105187. Wiese, B., & Omlin, C. (2009). Credit card transactions, fraud detection, and machine learning: Modelling time with LSTM recurrent neural networks (pp. 231-268). Springer Berlin Heidelberg. Wohl, I., & Kennedy, J. (2018). Neural network analysis of international trade. US International Trade Commission: Washington, DC, USA. World Trade Organisation (2011). Trade patterns and global value chains in East Asia: From trade in goods to trade in tasks. In collaboration with institute of developing economies (IDE) and Japan external trade organization (JETRO). WTO. World Trade Organisation (2015). International Trade Statistics 2015, available at: www.wto.org/statistics (Accessed on September 5, 2022). Xiong, H., Pandey, G., Steinbach, M., & Kumar, V. (2006). Enhancing data analysis with noise removal. IEEE transactions on knowledge and data engineering, 18(3), 304-319. Yu, Y., Si, X., Hu, C., & Zhang, J. (2019). A review of recurrent neural networks: LSTM cells and network architectures. Neural computation, 31(7), 1235-1270. Zhang, W., Yang, Q., & Geng, Y. (2009). A survey of anomaly detection methods in networks. In 2009 International Symposium on Computer Network and Multimedia Technology (pp. 1-3). IEEE. Zimek, A., Schubert, E., & Kriegel, H. P. (2012). A survey on unsupervised outlier detection in high‐dimensional numerical data. Statistical Analysis and Data Mining: The ASA Data Science Journal, 5(5), 363-387.