Scientific Machine Learning for the E&P
Machine learning is transforming scientific research on the one hand and is changing computational science and engineering in fundamental ways on the other hand. The machine learning revolution allows for the development of radical new techniques to address problems known to be very challenging with traditional methods. This short course provides an in-depth introduction to scientific machine learning with topics selected according to their relevance to applications in the energy industry.
ELEMENTS OF DEEP LEARNING: FROM NETWORKS IN EUCLIDEAN SPACES TO NEURAL OPERATORS AND TRANSFORMERS -- M.V. de Hoop, Rice University
In this tutorial, we will introduce various elements of deep learning and neural architecture design underpinning this transformation and revolution. We will begin with networks in Euclidean spaces, and guarantees for injectivity and invertibility leading to the introduction of flows in connection with geometric deep learning. Flows play an important role in generative modeling and variational inference. We present an application to imaging.
We then proceed with constructing neural operators, injective and bijective versions, and side-step discussing out-of-distribution risk bounds highlighting the importance of skip connections and stochastic depth. We discuss how physics can guide their design, and the importance of developing a framework in function and metric spaces encompassing discretization invariance. We present as an example learning the forward and inverse maps in the framework of "full-wave inversion''.
We finally, briefly introduce attention, in-context learning and transformers. We present as an example a foundation model for seismology.
GENERATIVE AI FOR SCIENTIFIC MACHINE LEARNING: MODELING SPATIO-TEMPORAL DATA -- Ben Erichson, UC Berkeley
Generative Artificial Intelligence (GenAI) presents transformative opportunities for understanding and predicting complex spatio-temporal phenomena in science and engineering. By enabling the generation of realistic and high-dimensional data, GenAI has revolutionized fields ranging from computer vision to climate modeling. Its versatility in learning complex patterns and generating synthetic data offers advantages for scientific discovery, particularly in addressing challenges of limited or noisy datasets. This tutorial introduces the foundational concepts of GenAI, with a focus on challenges such as super-resolution and forecasting.
We begin with an overview of state-of-the-art generative models (including Variational Autoenecoders (VAEs), Generative Adversarial Networks (GANs), and Diffusion Models), emphasizing their mathematical foundations and applicability to scientific data. Particular attention will be given to score-based diffusion models, which have shown remarkable capabilities in generating high-quality data reconstructions; and generative pretrained transformers (GPTs), which have shown remarkable capabilities for time-series modeling. Building on this, we explore the role of generative models in Uncertainty Quantification (UQ), demonstrating their potential to provide robust confidence intervals for predictions in dynamic systems.
Break
REINFORCEMENT LEARNING -- Moataz Ahmed, KFUPM
Reinforcement Learning (RL) is a powerful machine learning paradigm for designing systems capable of learning decision-making strategies through interaction with their environment. This tutorial provides an accessible yet comprehensive introduction to RL, starting with its core concepts: Agents, environments, actions, states, rewards, and the objective of maximizing cumulative rewards. Real-world examples, such as game playing, robotics, and autonomous systems, will be presented to illustrate practical applications. The discussion would then touch on foundational algorithms like Q-Learning, Deep Q-Networks (DQN), and Policy Gradient methods, highlighting their strengths and limitations. To engage the audience, the session would include a simple demonstration or visualization of an RL agent learning a task, such as navigating a maze or playing a basic game. Finally, challenges in RL, such as exploration vs. exploitation, scalability, and sample efficiency, wwill be briefly addressed to set the stage for further learning.
APPLICATIONS OF SCIML IN PETROLEUM ENGINEERING -- John Foster, UT Austin
Scientific Machine Learning (SciML) sits at the intersection of data science, machine learning, and physics-based computational simulation. SciML encompasses many ideas including physics-informed neural networks, universal differential equations, and the use of synthetic data generated from physical simulators or generative-AI in training machine learning models for rapid decision making. This tutorial will give an overview of SciML using simple examples and discuss recent results from our investigations using SciML in petroleum engineering, specifically for reservoir and drilling engineering applications.
THE SESSIONS AIM TO BALANCE TECHNICAL DEPTH AND ACCESSIBILITY, ENSURING THAT PARTICIPANTS LEAVE WITH A CLEAR UNDERSTANDING AND PRACTICAL RELEVANCE OF A BROAD RANGE OF DEVELOPMENTS IN MACHINE LEARNING.
