Seminar Archives

17 november 2023 12:30 t/m 13:15

[NA] Alena Kopaničáková: Enhancing Training of Deep Neural Networks Using Multilevel and Domain Decomposition Strategies

The training of deep neural networks (DNNs) is traditionally accomplished using stochastic gradient descent or its variants. While these methods have demonstrated certain robustness and accuracy, their convergence speed deteriorates for large-scale, highly ill-conditioned, and stiff problems, such as ones arising in scientific machine learning applications. Consequently, there is a growing interest in adopting more sophisticated training strategies that can not only accelerate convergence but may also enable parallelism, convergence control, and automatic selection of certain hyper-parameters.
In this talk, we propose to enhance the training of DNNs by leveraging nonlinear multilevel and domain decomposition strategies. We will discuss how to construct a multilevel hierarchy and how to decompose the parameters of the network by exploring the structure of the DNN architecture, properties of the loss function, and characteristics of the dataset. Furthermore, the dependency on a large number of hyper-parameters will be reduced by employing a trust-region globalization strategy. The effectiveness of the proposed training strategies will be demonstrated through a series of numerical experiments from the field of image classification and physics-informed neural networks.

References:
[1] A. Kopaničáková, H. Kothari, G. Karniadakis and R. Krause. Enhancing training of physics-informed neural networks using domain-decomposition based preconditioning strategies. Under review, 2023.
[2] S. Gratton, A. Kopaničáková, and Ph. Toint. Multilevel Objective-Function-Free Optimization with an Application to Neural Networks Training. SIAM, Journal on Optimization (Accepted), 2023.
[3] A. Kopaničáková. On the use of hybrid coarse-level models in multilevel minimization methods. Domain Decomposition Methods in Science and Engineering XXVII (Accepted), 2023.
[4] A. Kopaničáková, and R. Krause. Globally Convergent Multilevel Training of Deep Residual Networks. SIAM Journal on Scientific Computing, 2022.

20 oktober 2023 12:30 t/m 13:15

[NA] Artur Palha: Dual-field formulation: Navier-Stokes

Mimetic structure-preserving discretizations fundamentally require a set of functions spaces that constitute a discrete de Rham sequence. Although a prerequisite, the proper choice of discrete function spaces is not sufficient to construct a discrete physical model that preserves the invariants of the original system. The choice of formulation used to represent the original system of PDEs, is often one among many equivalent choices. Different formulations present different discrete properties (even when using the same set of function spaces). The dual-field formulation is a particular approach that re-writes the original system of equations as a coupled system of equations: one representing the evolution of the primal fields and another representing the evolution of the dual fields. By choosing a staggered time integration approach it is possible to obtain two systems of equations that linearise the nonlinear terms. Several invariants are preserved with this formulation for both Navier-Stokes and MHD systems. In this talk, the dual-field formulation will be presented with focus on the Navier-Stokes.

16 juni 2023 12:30 t/m 13:15

[NA] Andrew Gibbs: Evaluating Oscillatory Integrals using Automatic Steepest Descent

Highly oscillatory integrals arise across physical and engineering applications, particularly when modelling wave phenomena. When using standard numerical quadrature rules to evaluate highly oscillatory integrals, one requires a fixed number of points per wavelength to maintain accuracy across all frequencies of interest. Several oscillatory quadrature methods exist, but in contrast to standard quadrature rules (such as Gauss and Clenshaw-Curtis), effective use requires a priori analysis of the integral and, thus, a strong understanding of the method. This makes highly oscillatory quadrature rules inaccessible to non-experts.

A popular approach for evaluating highly oscillatory integrals is "Steepest Descent". The idea behind Steepest Descent methods is to deform the integration range onto complex contours where the integrand is non-oscillatory and exponentially decaying. By Cauchy's Theorem, the value of the integral is unchanged. Practically, this reformulation is advantageous, because exponential decay is far more amenable to asymptotic and numerical evaluation. As with other oscillatory quadrature rules, if naively applied, Steepest Descent methods can break down when the phase function contains coalescing stationary points.

In this talk, I will present a new algorithm based on Steepest Descent, which evaluates oscillatory integrals with a cost independent of frequency. The two main novelties are: (1) robustness - cost and accuracy are unaffected by coalescing stationary points, and (2) automation - no expertise or a priori analysis is required to use the algorithm.

26 mei 2023 12:30 t/m 13:15

[NA] Olaf Steinbach: Space-time finite and boundary element methods for the wave equation

In this talk we will review some recent results on space-time finite and
boundary element methods for the wave equation. As a first model problem
we consider the inhomogeneous wave equation with zero Dirichlet boundary
and initial conditions. The related space-time variational formulation
follows the standard approach when applying integration by parts in space
and time simultaneously. Note that a space-time tensor-product based
finite element discretization requires a CFL condition to ensure stability.
While the ansatz and test spaces are both subspaces of functions whose
space-time gradient is square integrable, they differ in zero initial
and terminal conditions to be satisfied. When introducing a modified
Hilbert transformation we end up with a Galerkin variational formulation
which is unconditionally stable. This modified Hilbert transformation is
also an essential tool in the formulation of coercive boundary integral
equations for the wave equation. Finally we also consider distributed
optimal control problems subject to the wave equation, and related
space-time least-squares finite and boundary element methods.

The talk is based on joint work with Marco Zank (Vienna), Richard
Löscher (Graz), Carolina Urzua-Torres (Delft), and Daniel Hoonhout (Delft).

21 april 2023 12:30 t/m 13:15

[NA] Melven Röhrig-Zöllner: Performance of low-rank linear solvers in tensor-train format

In this talk we discuss the problem of efficiently computing a low-rank solution of high-dimensional linear systems. More specifically, we discuss several methods for linear systems in the tensor-train format, also known as matrix-product-states (MPS) in physics. In particular, we consider global approaches like TT-GMRES and local approaches like TT-(M)ALS and TT-AMEn and look at suitable preconditioners and some algorithmic variants for non-symmetric operators. Overall, we focus on the computational complexity and on the performance on today's multi-core CPUs: The considered algorithms are composed of tensor contractions and of dense linear algebra operations like QR-decompositions and singular value decompositions (SVDs). We show significant speedup by carefully choosing suitable combinations of building blocks (e.g. using a tall-skinny QR + SVD). In addition, we show how to exploit orthogonalities from previous steps to speed-up tensor-train truncations. We illustrate the different effects in numerical experiments for simple Laplace- and convection-diffusion equations.

17 maart 2023 12:30 t/m 13:30

[NA] Hendrik Speleers: Explicit error estimates for spline approximation in isogeometric analysis

Isogeometric analysis is a well established paradigm to improve interoperability between geometric modeling and numerical simulations. It is commonly based on B-splines and shows important advantages over classical finite element analysis. In particular, the inherently high smoothness of B-splines leads to a higher accuracy per degree of freedom. This has been numerically observed in a wide range of applications, and recently a mathematical explanation has been given thanks to error estimates in spline spaces that are explicit, not only in the mesh size, but also in the polynomial degree and the smoothness.

In this talk we review some recent results on explicit error estimates for approximation by splines. These estimates are sharp or very close to sharp in several interesting cases and allow us to explain that certain spline subspaces are able to approximate eigenvalue problems without spurious outliers.

17 februari 2023 12:30 t/m 13:30

[NA] Fernando José Henriquez Barraza: Shape Uncertainty Quantification in Acoustic and Electromagnetic Scattering

In this talk, we consider the propagation of acoustic and electromagnetic waves in domains of uncertain shape. We are particularly interested in quantifying the effect on these perturbations on the involved fields and possibly into other quantities of interest. After considering a domain or surface parametrization with countably-many parameters, one obtains a high-dimensional parametric map describing the problem's solution manifold. The design and analysis of a variety of methods commonly used in computational uncertainty quantification (UQ for short) affording provably dimension-independent convergence rates rely on the holomorphic dependence of the problem's solution upon the parametric input. When the parametric input encodes a family of domain or boundary transformations, the holomorphic dependence of the problem upon the parametric input is usually referred to as shape holomorphy. We present and discuss the key technicalities involved in the verification of this property for different models including: volume formulation of the Helmholtz problem, boundary integral formulation, volume integral equations, boundary integral formulations for multiple disjoint arcs, among others. We discuss the importance of this property in the implementation and analysis of different techniques used in forward and inverse computational shape UQ for the previously described models and the implications in the constructions efficient surrogates using neural networks.

20 januari 2023 12:30 t/m 13:30

[NA] Michiel Hochstenbach: 25+ years of nonnormality, a review

We will review several aspects of nonnormal matrices: well-established concepts, recent developments, and open questions. Aspects that will be mentioned include eigenvector condition numbers, pseudospectra, field of values, scaling/balancing, shift invariancy, Krylov spaces, and two-sided methods. We will mention connections to some key scientific problems such as solving linear systems and eigenvalue problems, as well as some practical implementation tips.

25 november 2022 12:30 t/m 13:30

Alessandro Reali: Some advances in isogeometric analysis of coupled and complex problems

21 oktober 2022 12:30 t/m 13:30

Vandana Dwarka: Scalable solvers for the Helmholtz problem

16 september 2022 12:30 t/m 13:30

Shobhit Jain: Dynamics-based model reduction for nonlinear finite element models

17 juni 2022 12:30 t/m 13:30

Eef van Dongen: Monitoring and numerical modelling of glacier dynamics on Greenland

20 mei 2022 12:30 t/m 13:30

Martin Kühn: Mathematical modeling of infectious diseases

22 april 2022 12:30 t/m 13:30

Thomas Takacs: Smooth isogeometric discretizations for fourth order PDEs

18 maart 2022 12:30 t/m 13:30

Espen Sande: Outlier-free isogeometric discretizations

21 januari 2022 12:30

Mengwu Guo: Options of Bayesian Methods for Data-Driven Model Reduction

17 december 2021 12:30

CANCELED: Iryna Rybak: Interface conditions for arbitrary flows in Stokes-Darcy systems

19 november 2021 12:30

Peter Bastian: Multilevel Spectral Domain Decomposition Methods

21 mei 2021 12:30

Victorita Dolean: Robust solvers for time harmonic wave propagation problems

16 april 2021 12:30