Seismic dataset-specific machine learning framework based on pre-training and fine-tuning

Every seismic dataset has its particular characteristics guided mainly by the properties of the subsurface, the data acquisition parameters (the survey), and the often unique noise conditions it experiences. Capturing such characteristics in a neural network model for the efficient application of processing tasks, like denoising, first arrival picking, velocity estimation, and so on, offer a more effective approach to incorporating machine learning in processing than training neural networks for specific tasks that may or may not transfer well to new data. We introduce a framework for seismic processing that allows us to pre-train a neural network to learn the features of a seismic dataset, and then fine-tune that network for any downstream processing task. We take advantage of the fact that most processing tasks utilize the same features embedded in the seismic dataset, and thus, these features can be stored in a common pre-trained network in a self-supervised manner, we refer to as StorSeismic. In this framework, we utilize a Bidirectional Encoder Representations from Transformers (BERT) model to promote pre-training for storing the features of a seismic dataset and then efficiently fine-tune it to adapt to a wide spectrum of seismic processing tasks. We apply this framework on field data, along with synthetically generated data, in the self-supervised pre-training step to store the seismic features. Then, we use the labeled synthetic data to fine-tune the pre-trained network in a supervised fashion to perform various seismic processing tasks, like denoising, low frequency extrapolation, first arrival picking, and velocity estimation, and finally obtain satisfactory inference results on the field data.