Multivariate irregular time series anomaly detection based on Deep Learning (or even self-attention)?

Question

I work as a remote sensing researcher using satellite image time series (SITS) for forest change detection. I want to automatically detect anomalies in multivariate time series of forest pixels, which will then be identified as forest disturbance. The time series usually have a length of 4 years. Training needs to be unsupervised, since I don't have any labels.

I am encountering three main challenges:

1. I have irregular time series (optical satellites suffer from clouds), which is a problem rarely accounted for in the literature. My ideas for adjustments are time masking (putting zero weights on "non-observations" after insert padding) or using an explicit time embedding (such as Positional Encoding with the day of the year, as in the SITS-BERT model [1]).

2. Another problem is that I need a reconstruction method that does not work on windowing. That's because I have a large number (more than a million) of presumably non-anomalous, but short time series (about 200 observations). I don't think windowing makes sense here. I looked at the code of the TranAD model (which I wanted to adapt to my needs) [2], for example, and it turned out that each time series is fed sequentially to the model, and parallelization is only done across the windows. That seems to be the normal case, which does not fit my data. In contrast, I need to input time series in parallel, not windows.
   Anyway, windows are likely too short to capture the natural seasonality in my time series (deciduous trees not having leaves in winter, for instance). So I would rather take a full sequence processing approach.

3. I am not a Deep Learning developer, but an ecologist (with advanced Python skills). I would need a starting point to build upon, in terms of code. So I need a (preferably PyTorch + Python) code implementation to adjust to my needs.

I want to use state-of-the-art Deep Learning methods, if possible, which in my opinion would be based on self-attention. Since I could not find a self-attention based method with published code without windowing, I thought maybe Variational AutoEncoders (VAE) and/or Generative Adversarial Networks (GAN) or a mixture thereof would be a good idea, but haven't found anything useful.

Since I've been searching for a method fulfilling my requirements for weeks to no avail now, I want to reach out to the community, hoping to get some hints.

Thank you very much!

References: [1] Yuan, Y., & Lin, L. (2020). Self-supervised pretraining of transformers for satellite image time series classification. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 14, 474-487. [2] Tuli, S., Casale, G., & Jennings, N. R. (2022). Tranad: Deep transformer networks for anomaly detection in multivariate time series data. arXiv preprint arXiv:2201.07284.