The Symbiosis of Deep Learning and Differential Equations (DLDE) - II
NeurIPS 2022 Workshop
In recent years, there has been a rapid increase of machine learning applications in computational sciences, with some of the most impressive results at the interface of deep learning (DL) and differential equations (DEs). DL techniques have been used in a variety of ways to dramatically enhance the effectiveness of DE solvers and computer simulations. These successes have widespread implications, as DEs are among the most well-understood tools for the mathematical analysis of scientific knowledge, and they are fundamental building blocks for mathematical models in engineering, finance, and the natural sciences. Conversely, DL algorithms based on DEs--such as neural differential equations and continuous-time diffusion models--have also been successfully employed as deep learning models. Moreover, theoretical tools from DE analysis have been used to glean insights into the expressivity and training dynamics of mainstream deep learning algorithms.
This workshop will aim to bring together researchers with backgrounds in computational science and deep learning to encourage intellectual exchanges, cultivate relationships and accelerate research in this area. The scope of the workshop spans topics at the intersection of DL and DEs, including theory of DL and DEs, neural differential equations, solving DEs with neural networks, and more.
|Submission Deadline||October 1st, 2022 - Anywhere on Earth (AoE)|
|Final Decisions||October 17th, 2022 - AoE|
|Workshop Date||December 9th, 2022 - AoE|
Call for Extended Abstracts
We invite high-quality extended abstract submissions on the intersection of DEs and DL. Some examples (non-exhaustive list):
- Using differential equation models to understand and improve deep learning algorithms:
- Incorporating DEs into existing DL models (neural differential equations, diffusion models, ...)
- Analysis of numerical methods for implementing DEs in DL models (trade-offs, benchmarks, ...)
- Modeling training dynamics using DEs to generate theoretical insights and novel algorithms.
- Using deep learning algorithms to create or solve differential equation models:
- DL methods for solving high-dimensional, highly parameterized, or otherwise challenging DE models.
- Learning-augmented numerical methods for DEs (hypersolvers, hybrid solvers ...)
- Specialized DL architectures for solving DEs (neural operators, PINNs, ...).
Accepted submissions will be presented during joint poster sessions and will be made publicly available as non-archival reports, allowing future submissions to archival conferences or journals. Exceptional submissions will be either selected for four 15-minute contributed talks or eight 5-minute spotlight oral presentations.
Submissions should be up to 4 pages excluding references, acknowledgements, and supplementary material, and should be DLDE-NeurIPS format and anonymous. Long appendices are permitted but strongly discouraged, and reviewers are not required to read them. The review process is double-blind.
We also welcome submissions of recently published work that is strongly within the scope of the workshop (with proper formatting). We encourage the authors of such submissions to focus on accessibility to the wider NeurIPS community while distilling their work into an extended abstract. Submission of this type will be eligible for poster sessions after a lighter review process.
Authors may be asked to review other workshop submissions.
Please submit your extended abstract to this address.
If you have any question, send an email to : [firstname.lastname@example.org]
If you have any question for the Invited Speakers, send it through this form.
- 04:00 : Introduction and opening remarks
- 04:10 : Provable Active Learning of Neural Networks for Parametric PDEs (Spotlight)
- 04:25 : PIXEL: Physics-Informed Cell Representations for Fast and Accurate PDE Solvers (Spotlight)
- 04:40 : Bridging the Gap Between Coulomb GAN and Gradient-regularized WGAN (Spotlight)
- 04:55 : How PINNs cheat: Predicting chaotic motion of a double pendulum (Spotlight)
- 05:10 : Poster Session 1
- 06:05 : Yang Song (Keynote Talk)
- 06:50 : Blind Drifting: Diffusion models with a linear SDE drift term for blind image restoration tasks (Spotlight)
- 07:05 : Break
- 08:05 : Rose Yu (Keynote Talk)
- 08:50 : A Universal Abstraction for Hierarchical Hopfield Networks (Spotlight)
- 09:05 : Poster Session 2
- 10:00 : Christopher Rackauckas (Keynote Talk)
- 10:45 : Closing remarks
Chris Rackauckas (confirmed) is the Co-PI of the Julia Lab at the Computer Science and AI Laboratory (CSAIL) of the Massachussets Institute of Science and Technology (MIT), as well as the director of Modeling and Simulation at Julia Computing and of Scientific Research at Pumas-AI. His research spans topics in numerical methods for differential equations, scientific machine learning and high-performance computing. He is the lead developer of the DifferentialEquations.jl solver suite along with over a hundred other Julia packages, earning him the inaugural Julia Community Prize, an outstanding paper award at the IEEE-HPEC conference on computational derivation for the efficient stochastic differential equation solvers, and front page features on many tech community sites. [Webpage]
Rose Yu (confirmed) is an assistant professor at UC San Diego department of Computer Science and Engineering and Halıcıo ̆glu Data Science Institute. She is a primary faculty with the AI Group and is affiliated with Contextual Robotics Institute, Bioinformatics and Systems Biology, and Center for Machine-Integrated Computing and Security. Her research interests lie primarily in machine learning, especially for large-scale spatiotemporal data. She is generally interested in deep learning, optimization, and spatio-temporal reasoning, and particularly excited about the interplay between physics and machine learning. Her work has been applied to learning dynamical systems in sustainability, health and physical sciences. [Webpage]
Yang Song (confirmed) is a final year Ph.D. student of Computer Science at Stanford University. His work focuses on the foundations of deep generative models, as well as their applications to AI safety, inverse problems, and their impacts on the broader field of machine learning. He has made several key contributions to the mathematical formulation and empirical performance of continuous-time score-based diffusion models, currently state--of--the--art methods in deep generative modeling. [Webpage]
The workshop will be broadcasted via Zoom and poster sessions will be on Gathertown. We will upload the recordings on YouTube.
University of Toronto, Vector Institute, NVIDIA
University of Toronto, Vector Institute
University of Toronto
University of Michigan,
University of Tokyo, RIKEN
University of Guelph
Université de Sherbrooke
University of Cambridge
University of California, Berkley
Université de Sherbrooke
Thanks to visualdialog.org for the webpage format. Thanks to whatsonmyblackboard for the cozy blackboard photo.
 D. Kochkov, J.A. Smith, A. Alieva, Q. Wang, M.P. Brenner, and S. Hoyer Machine learning–accelerated computational fluid dynamics. PNAS, 2021.
 E. Haber and L. Ruthotto Stable architectures for deep neural networks. IOPScience, 2017.
 R.T.Q. Chen, Y. Rubanova, J. Bettencourt, D. Duvenaud Neural Ordinary Differential Equations. NeurIPS, 2018.
 G. Carleo, M. Troyer Solving the quantum many-body problem with artificial neural networks. Science, 2017.
 P. Chaudhari, S. Soatto Stochastic gradient descent performs variational inference, converges to limit cycles for deep networks. ICLR, 2018.
 Q. Li, C. Tai, W. E Stochastic Modified Equations and Adaptive Stochastic Gradient Algorithms. ICML, 2017.
 W. E, C. Ma & L. Wu The Barron Space and the Flow-Induced Function Spaces for Neural Network Models. Constructive Approximation, 2021.
 Z. Li, N. Kovachki, K. Azizzadenesheli, B. Liu, K. Bhattacharya, A. Stuart, A. Anandkumar Fourier Neural Operator for Parametric Partial Differential Equations. ICLR, 2021.
 J. Sirignano, K. Spiliopoulos DGM: A deep learning algorithm for solving partial differential equations. Journal of Computational Physics, 2018.
 L. Lu, X. Meng, Z. Mao, G.E. Karniadakis DeepXDE: A Deep Learning Library for Solving Differential Equations. Society for Industrial and Applied Mathematics, 2021.
 S. Mishra, R. Molinaro Estimates on the generalization error of physics-informed neural networks for approximating a class of inverse problems for PDEs. IMA Journal of Numerical Analysis, 2021.
 S. Bai, J.Z. Kolter, V. Koltun Deep Equilibrium Models. NeurIPS, 2019.
 N. Yadav, K.S. McFall, M. Kumar, J.H. Kim A length factor artificial neural network method for the numerical solution of the advection dispersion equation characterizing the mass balance of fluid flow in a chemical reactor. Neural Computing and Applications, 2018.
 C. Beck, S. Becker, P. Grohs, N. Jaafari, A. Jentzen Solving the Kolmogorov PDE by Means of Deep Learning. Journal of Scientific Computing, 2021.