Neural Networks for Seismic Data Denoising: Attention Mechanisms and Diffusion Models

Abstract

Seismic data, which is crucial for understanding the Earth’s subsurface structure, is frequently compromised by incoherent and coherent noise, complicating accurate geological imaging and thus making noise suppression one of the most important processing steps. Traditional denoising methods, although widely used, are typically time-intensive and struggle to differentiate adequately between signal and noise, often leading to primary signal damage. We therefore utilize advanced machine learning methods and in this regard introduce a novel self-supervised residual encoder-decoder network equipped with a local attention mechanism to effectively attenuate uncorrelated seismic noise. The self-supervised nature of this architecture allows the network to learn directly from the data itself, eliminating the need for explicit labels or prior knowledge about the noise, thereby simplifying us- age and enhancing efficiency. Residual connections within the network help retain critical seismic signal characteristics during the denoising process. A significant innovation in our approach is the integration of a local self-attention mechanism, enabling the model to concentrate on relevant segments of the input data, thus improving noise attenuation while preserving the seismic signals. Additionally, we employ a specialized loss function that combines Mean Squared Error (MSE) with the Structural Similarity Index (SSIM) to minimize primary damage and ensure better preservation of primary signals. To address the more complex challenge of coherent noise attenuation, we enhance the encoder-decoder network by incorporating attention gates alongside the already implemented local attention within the so-called Dual-Attention Residual Encoder-Decoder (DARED) network. This dual-attention mechanism allows the network to focus on both local and global features, further reducing the loss of primary signal. Given the predictable structure of the noise, we have to use supervised learning with labels, generated by rank-reduction-based denoising. Recognizing the time-intensive nature of traditional denoising parameter selection, we propose to train the network on a manually denoised small portion of the dataset before applying it to the entire dataset. This strategy offers a time-efficient enhancement to conventional methods. The results indicate that our approach can reduce the primary damage more effectively than the deterministic denoising label. Furthermore, this thesis tackles the challenge of limited labeled training data by introducing a novel data augmentation scheme based on Denoising Diffusion Probabilistic Models (DDPM). This approach generates new seismic data and corresponding labels that mirror the distribution of the seismic dataset used to train this generative network, effectively addressing the training data limitation. By integrating DDPM-based data augmentation with the Dual-Attention Residual Encoder-Decoder network, we achieve significant performance improvements in denoising. Our results demonstrate that this combined approach enhances the attenuation of noise and also better mitigates primary damage, compared to the application of the DARED network without data augmentation. Applications of these methods to both synthetic and field seismic data showcase their potential for seismic denoising. These methods offer efficient and less destructive noise attenuation techniques, underscoring the impact of using neural networks and integrating attention mechanisms alongside data augmentation based on generative AI in seismic data denoising. This thesis highlights the crucial role of advanced machine learning in modern geophysical research.