Seminar Archives

This the archive page for our ongoing Seminars in Numerical Analysis series. Please find the historical (<2021) archives in the left pane, or find a later archived event below.

17 november 2023 12:30 t/m 13:15

[NA] Alena Kopanińć√°kov√°: Enhancing Training of Deep Neural Networks Using Multilevel and Domain Decomposition Strategies

The training of deep neural networks (DNNs) is traditionally accomplished using stochastic gradient descent or its variants. While these methods have demonstrated certain robustness and accuracy, their convergence speed deteriorates for large-scale, highly ill-conditioned, and stiff problems, such as ones arising in scientific machine learning applications. Consequently, there is a growing interest in adopting more sophisticated training strategies that can not only accelerate convergence but may also enable parallelism, convergence control, and automatic selection of certain hyper-parameters.
In this talk, we propose to enhance the training of DNNs by leveraging nonlinear multilevel and domain decomposition strategies. We will discuss how to construct a multilevel hierarchy and how to decompose the parameters of the network by exploring the structure of the DNN architecture, properties of the loss function, and characteristics of the dataset. Furthermore, the dependency on a large number of hyper-parameters will be reduced by employing a trust-region globalization strategy. The effectiveness of the proposed training strategies will be demonstrated through a series of numerical experiments from the field of image classification and physics-informed neural networks.

References:
[1] A. Kopanińć√°kov√°, H. Kothari, G. Karniadakis and R. Krause. Enhancing training of physics-informed neural networks using domain-decomposition based preconditioning strategies. Under review, 2023.
[2] S. Gratton, A. Kopanińć√°kov√°, and Ph. Toint. Multilevel Objective-Function-Free Optimization with an Application to Neural Networks Training. SIAM, Journal on Optimization (Accepted), 2023.
[3] A. Kopanińć√°kov√°. On the use of hybrid coarse-level models in multilevel minimization methods. Domain Decomposition Methods in Science and Engineering XXVII (Accepted), 2023.
[4] A. Kopanińć√°kov√°, and R. Krause. Globally Convergent Multilevel Training of Deep Residual Networks. SIAM Journal on Scientific Computing, 2022.

16 juni 2023 12:30 t/m 13:15

[NA] Andrew Gibbs: Evaluating Oscillatory Integrals using Automatic Steepest Descent

Highly oscillatory integrals arise across physical and engineering applications, particularly when modelling wave phenomena. When using standard numerical quadrature rules to evaluate highly oscillatory integrals, one requires a fixed number of points per wavelength to maintain accuracy across all frequencies of interest. Several oscillatory quadrature methods exist, but in contrast to standard quadrature rules (such as Gauss and Clenshaw-Curtis), effective use requires a priori analysis of the integral and, thus, a strong understanding of the method. This makes highly oscillatory quadrature rules inaccessible to non-experts.

A popular approach for evaluating highly oscillatory integrals is &quot;Steepest Descent&quot;. The idea behind Steepest Descent methods is to deform the integration range onto complex contours where the integrand is non-oscillatory and exponentially decaying. By Cauchy's Theorem, the value of the integral is unchanged. Practically, this reformulation is advantageous, because exponential decay is far more amenable to asymptotic and numerical evaluation. As with other oscillatory quadrature rules, if naively applied, Steepest Descent methods can break down when the phase function contains coalescing stationary points.

In this talk, I will present a new algorithm based on Steepest Descent, which evaluates oscillatory integrals with a cost independent of frequency. The two main novelties are: (1) robustness - cost and accuracy are unaffected by coalescing stationary points, and (2) automation - no expertise or a priori analysis is required to use the algorithm.

17 februari 2023 12:30 t/m 13:30

[NA] Fernando José Henriquez Barraza: Shape Uncertainty Quantification in Acoustic and Electromagnetic Scattering

In this talk, we consider the propagation of acoustic and electromagnetic waves in domains of uncertain shape. We are particularly interested in quantifying the effect on these perturbations on the involved fields and possibly into other quantities of interest. After considering a domain or surface parametrization with countably-many parameters, one obtains a high-dimensional parametric map describing the problem's solution manifold. The design and analysis of a variety of methods commonly used in computational uncertainty quantification (UQ for short) affording provably dimension-independent convergence rates rely on the holomorphic dependence of the problem's solution upon the parametric input. When the parametric input encodes a family of domain or boundary transformations, the holomorphic dependence of the problem upon the parametric input is usually referred to as shape holomorphy. We present and discuss the key technicalities involved in the verification of this property for different models including: volume formulation of the Helmholtz problem, boundary integral formulation, volume integral equations, boundary integral formulations for multiple disjoint arcs, among others. We discuss the importance of this property in the implementation and analysis of different techniques used in forward and inverse computational shape UQ for the previously described models and the implications in the constructions efficient surrogates using neural networks.