MURAL - Maynooth University Research Archive Library



    SERT: A transformer based model for multivariate temporal sensor data with missing values for environmental monitoring


    Nejad, Amin Shoari, Alaiz-Rodríguez, Rocío, McCarthy, Gerard D., Kelleher, Brian, Grey, Anthony and Parnell, Andrew (2024) SERT: A transformer based model for multivariate temporal sensor data with missing values for environmental monitoring. Computers & Geosciences, 188. p. 105601. ISSN 00983004

    [thumbnail of 1-s2.0-S0098300424000840-main.pdf]
    Preview
    Text
    1-s2.0-S0098300424000840-main.pdf

    Download (2MB) | Preview
    Official URL: https://doi.org/10.1016/j.cageo.2024.105601

    Abstract

    Environmental monitoring is crucial to our understanding of climate change, biodiversity loss and pollution. The availability of large-scale spatio-temporal data from sources such as sensors and satellites allows us to develop sophisticated models for forecasting and understanding key drivers. However, the data collected from sensors often contain missing values due to faulty equipment or maintenance issues. The missing values rarely occur simultaneously leading to data that are multivariate misaligned sparse time series. We propose two models that are capable of performing multivariate spatio-temporal forecasting while handling missing data naturally without the need for imputation. The first model is a transformer-based model, which we name SERT (Spatio-temporal Encoder Representations from Transformers). The second is a simpler model named SST-ANN (Sparse Spatio-Temporal Artificial Neural Network) which is capable of providing interpretable results. We conduct extensive experiments on two different datasets for multivariate spatio-temporal forecasting and show that our models have competitive or superior performance to those at the state-of-the-art.
    Item Type: Article
    Additional Information: We would like to express our sincere gratitude to Dublin Port Company for providing us with the real dataset. This work was supported by an SFI Investigator award (16/IA/4520).
    Keywords: spatio-temporal; deep learning; transformers; environmental monitoring;
    Academic Unit: Faculty of Science and Engineering > Research Institutes > Hamilton Institute
    Faculty of Social Sciences > Research Institutes > Irish Climate Analysis and Research Units, ICARUS
    Item ID: 18916
    Identification Number: 10.1016/j.cageo.2024.105601
    Depositing User: Andrew Parnell
    Date Deposited: 24 Sep 2024 09:32
    Journal or Publication Title: Computers & Geosciences
    Publisher: Elsevier
    Refereed: Yes
    Related URLs:
    URI: https://mural.maynoothuniversity.ie/id/eprint/18916
    Use Licence: This item is available under a Creative Commons Attribution Non Commercial Share Alike Licence (CC BY-NC-SA). Details of this licence are available here

    Repository Staff Only (login required)

    Item control page
    Item control page

    Downloads

    Downloads per month over past year

    Origin of downloads