Nejad, Amin Shoari and Alaiz-Rodríguez, Rocío and McCarthy, Gerard D. and Kelleher, Brian and Grey, Anthony and Parnell, Andrew (2024) SERT: A transformer based model for multivariate temporal sensor data with missing values for environmental monitoring. Computers & Geosciences, 188. p. 105601. ISSN 00983004

Download (2MB)

Abstract

Environmental monitoring is crucial to our understanding of climate change, biodiversity loss and pollution. The availability of large-scale spatio-temporal data from sources such as sensors and satellites allows us to develop sophisticated models for forecasting and understanding key drivers. However, the data collected from sensors often contain missing values due to faulty equipment or maintenance issues. The missing values rarely occur simultaneously leading to data that are multivariate misaligned sparse time series. We propose two models that are capable of performing multivariate spatio-temporal forecasting while handling missing data naturally without the need for imputation. The first model is a transformer-based model, which we name SERT (Spatio-temporal Encoder Representations from Transformers). The second is a simpler model named SST-ANN (Sparse Spatio-Temporal Artificial Neural Network) which is capable of providing interpretable results. We conduct extensive experiments on two different datasets for multivariate spatio-temporal forecasting and show that our models have competitive or superior performance to those at the state-of-the-art.

Item Type:	Article
Additional Information:	We would like to express our sincere gratitude to Dublin Port Company for providing us with the real dataset. This work was supported by an SFI Investigator award (16/IA/4520).
Keywords:	spatio-temporal; deep learning; transformers; environmental monitoring;
Academic Unit:	Faculty of Science and Engineering > Research Institutes > Hamilton Institute Faculty of Social Sciences > Research Institutes > Irish Climate Analysis and Research Units, ICARUS
Item ID:	18916
Identification Number:	https://doi.org/10.1016/j.cageo.2024.105601
Depositing User:	Andrew Parnell
Date Deposited:	24 Sep 2024 09:32
Journal or Publication Title:	Computers & Geosciences
Publisher:	Elsevier
Refereed:	Yes
URI:	Publisher
Use Licence:	This item is available under a Creative Commons Attribution Non Commercial Share Alike Licence (CC BY-NC-SA). Details of this licence are available here

Repository Staff Only(login required)

Item control page

Downloads

Origin of downloads