WaveFormer: Wavelet Embedding Transformer for Biomedical Signals
This addresses the problem of classifying biomedical signals with complex temporal and frequency patterns for researchers in healthcare and AI, though it appears incremental by adapting existing transformer methods.
The paper tackled biomedical signal classification by proposing WaveFormer, a transformer architecture that integrates wavelet decomposition for embedding and positional encoding, achieving competitive performance on eight diverse datasets with sequence lengths from 50 to 3000 timesteps.
Biomedical signal classification presents unique challenges due to long sequences, complex temporal dynamics, and multi-scale frequency patterns that are poorly captured by standard transformer architectures. We propose WaveFormer, a transformer architecture that integrates wavelet decomposition at two critical stages: embedding construction, where multi-channel Discrete Wavelet Transform (DWT) extracts frequency features to create tokens containing both time-domain and frequency-domain information, and positional encoding, where Dynamic Wavelet Positional Encoding (DyWPE) adapts position embeddings to signal-specific temporal structure through mono-channel DWT analysis. We evaluate WaveFormer on eight diverse datasets spanning human activity recognition and brain signal analysis, with sequence lengths ranging from 50 to 3000 timesteps and channel counts from 1 to 144. Experimental results demonstrate that WaveFormer achieves competitive performance through comprehensive frequency-aware processing. Our approach provides a principled framework for incorporating frequency-domain knowledge into transformer-based time series classification.