Seminar Archives

19 april 2024 12:30 t/m 13:15

[NA] Alena Kopaničáková : Enhancing Training of Deep Neural Networks Using Multilevel and Domain Decomposition Strategies

The training of deep neural networks (DNNs) is traditionally accomplished using stochastic gradient descent or its variants. While these methods have demonstrated certain robustness and accuracy, their convergence speed deteriorates for large-scale, highly ill-conditioned, and stiff problems, such as ones arising in scientific machine learning applications. Consequently, there is a growing interest in adopting more sophisticated training strategies that can not only accelerate convergence but may also enable parallelism, convergence control, and automatic selection of certain hyper-parameters.
In this talk, we propose to enhance the training of DNNs by leveraging nonlinear multilevel and domain decomposition strategies. We will discuss how to construct a multilevel hierarchy and how to decompose the parameters of the network by exploring the structure of the DNN architecture, properties of the loss function, and characteristics of the dataset. Furthermore, the dependency on a large number of hyper-parameters will be reduced by employing a trust-region globalization strategy. The effectiveness of the proposed training strategies will be demonstrated through a series of numerical experiments from the field of image classification and physics-informed neural networks.

References:
[1] A. Kopaničáková, H. Kothari, G. Karniadakis and R. Krause. Enhancing training of physics-informed neural networks using domain-decomposition based preconditioning strategies. Under review, 2023.
[2] S. Gratton, A. Kopaničáková, and Ph. Toint. Multilevel Objective-Function-Free Optimization with an Application to Neural Networks Training. SIAM, Journal on Optimization (Accepted), 2023.
[3] A. Kopaničáková. On the use of hybrid coarse-level models in multilevel minimization methods. Domain Decomposition Methods in Science and Engineering XXVII (Accepted), 2023.
[4] A. Kopaničáková, and R. Krause. Globally Convergent Multilevel Training of Deep Residual Networks. SIAM Journal on Scientific Computing, 2022.

16 maart 2024 12:30 t/m 13:15

[NA] Carlos Pérez Arancibia: Fast, high-order numerical evaluation of volume potentials via polynomial density interpolation

This talk outlines a novel class of high-order methods for the efficient numerical evaluation of volume potentials (VPs) defined by volume integrals over complex geometries. Inspired by the Density Interpolation Method (DIM) for boundary integral operators, the proposed methodology leverages Green’s third identity and a local polynomial interpolation of the density function to recast a given VP as a linear combination of surface-layer potentials and a volume integral with a regularized (bounded or smoother) integrand. The layer potentials can be accurately and efficiently evaluated inside and outside the integration domain using existing methods (e.g. DIM), while the regularized volume integral can be accurately evaluated by applying elementary quadrature rules to integrate over structured or unstructured domain decompositions without local numerical treatment at and around the kernel singularity. The proposed methodology is flexible, easy to implement, and fully compatible with well-established fast algorithms such as the Fast Multipole Method and H-matrices, enabling VP evaluations to achieve linearithmic computational complexity. To demonstrate the merits of the proposed methodology, we applied it to the Nyström discretization of the Lippmann-Schwinger volume integral equation for frequency-domain Helmholtz scattering problems in piecewise-smooth variable media.

16 februari 2024 12:30 t/m 13:15

[NA] Andrea Bressan: Dimension of piecewise polynomials on the Wang-Shi macroelement

In dimension one the construction of piecewise polynomials with given degree, number of continuous derivatives and the subdomains is a solved problem. Already in dimension two the rank of the continuity condition depends on the geometry of the polygonal subdomains and thus the space dimension can only be computed case by case. This changes if the subdomains are sufficiently structured for the required pair of degree and number of continuous derivatives. On cartesian meshes C^k splines are achieved with any degree d>k using a tensor product construction and both the space dimension and its properties are easily derived by the univariate case. On any triangulation C^0 splines can be constructed for all degree>0. More generally C^k functions on a general triangulation can be achieved for degree d>= 3k+2, e.g. for k=1, d=5 is the Argyris element. (a local basis requires d>=4k+1) An alternative is to add additional structure to the partition by replacing a the triangles of a triangulation with macro-elements, i.e. by “splitting” the triangles of a triangulation in subdomains. Examples are C^1-cubic on the Clough–Tocher split and C^2-quintics on the Powell-Sabin 12 split. Recently C^2-cubics (2022) and C^3-quartics (2024) spaces have been constructed using the Wang-Shi split where the macroelements consist of polygons with possibly many vertices. The talk will summarize an “elementary" proof that the dimension of C^(d-1) splines of degree d on the Wang-Shi macroelement can be expressed in purely combinatorial terms.

15 december 2023 12:30 t/m 13:15

[NA] Stefan Kurz: Observers in relativistic electrodynamics

"We introduce a relativistic splitting structure to map fields and equations of electromagnetism from four-dimensional spacetime to three-dimensional observer's space. We focus on a minimal set of mathematical structures that are directly motivated by the language of the physical theory. Space-time, world-lines, time translation, space platforms, and time synchronization all find their mathematical counterparts. The splitting structure is defined without recourse to coordinates or frames. This is noteworthy since, in much of the prevalent literature, observers are identified with adapted coordinates and frames. Among the benefits of the approach is a concise and insightful classification of observers. The application of the framework to Schiff's ""Question in General Relativity"" [1] further illustrates the advantages of the framework, enabling a compact, yet profound analysis of the problem at hand.  
[1] Schiff, L. I. ""A question in general relativity."" Proceedings of the National Academy of Sciences 25.7 (1939): 391-395.
Consider two concentric spheres with equal and opposite total charges uniformly distributed over their surfaces. When the spheres are at rest, the electric and magnetic fields outside the spheres vanish. [...] Then an observer traveling in a circular orbit around the spheres should find no field, for since all of the components of the electromagnetic field tensor vanish in one coordinate system, they must vanish in all coordinate systems. On the other hand, the spheres are rotating with respect to this observer, and so he should experience a magnetic field. [...] It is clear in the above arrangement that an observer A at rest with respect to the spheres does not obtain the same results from physical experiments as an observer B who is rotating about the spheres."

17 november 2023 12:30 t/m 13:15

[NA] Alena Kopaničáková: Enhancing Training of Deep Neural Networks Using Multilevel and Domain Decomposition Strategies

The training of deep neural networks (DNNs) is traditionally accomplished using stochastic gradient descent or its variants. While these methods have demonstrated certain robustness and accuracy, their convergence speed deteriorates for large-scale, highly ill-conditioned, and stiff problems, such as ones arising in scientific machine learning applications. Consequently, there is a growing interest in adopting more sophisticated training strategies that can not only accelerate convergence but may also enable parallelism, convergence control, and automatic selection of certain hyper-parameters.
In this talk, we propose to enhance the training of DNNs by leveraging nonlinear multilevel and domain decomposition strategies. We will discuss how to construct a multilevel hierarchy and how to decompose the parameters of the network by exploring the structure of the DNN architecture, properties of the loss function, and characteristics of the dataset. Furthermore, the dependency on a large number of hyper-parameters will be reduced by employing a trust-region globalization strategy. The effectiveness of the proposed training strategies will be demonstrated through a series of numerical experiments from the field of image classification and physics-informed neural networks.

References:
[1] A. Kopaničáková, H. Kothari, G. Karniadakis and R. Krause. Enhancing training of physics-informed neural networks using domain-decomposition based preconditioning strategies. Under review, 2023.
[2] S. Gratton, A. Kopaničáková, and Ph. Toint. Multilevel Objective-Function-Free Optimization with an Application to Neural Networks Training. SIAM, Journal on Optimization (Accepted), 2023.
[3] A. Kopaničáková. On the use of hybrid coarse-level models in multilevel minimization methods. Domain Decomposition Methods in Science and Engineering XXVII (Accepted), 2023.
[4] A. Kopaničáková, and R. Krause. Globally Convergent Multilevel Training of Deep Residual Networks. SIAM Journal on Scientific Computing, 2022.

20 oktober 2023 12:30 t/m 13:15

[NA] Artur Palha: Dual-field formulation: Navier-Stokes

Mimetic structure-preserving discretizations fundamentally require a set of functions spaces that constitute a discrete de Rham sequence. Although a prerequisite, the proper choice of discrete function spaces is not sufficient to construct a discrete physical model that preserves the invariants of the original system. The choice of formulation used to represent the original system of PDEs, is often one among many equivalent choices. Different formulations present different discrete properties (even when using the same set of function spaces). The dual-field formulation is a particular approach that re-writes the original system of equations as a coupled system of equations: one representing the evolution of the primal fields and another representing the evolution of the dual fields. By choosing a staggered time integration approach it is possible to obtain two systems of equations that linearise the nonlinear terms. Several invariants are preserved with this formulation for both Navier-Stokes and MHD systems. In this talk, the dual-field formulation will be presented with focus on the Navier-Stokes.

16 juni 2023 12:30 t/m 13:15

[NA] Andrew Gibbs: Evaluating Oscillatory Integrals using Automatic Steepest Descent

Highly oscillatory integrals arise across physical and engineering applications, particularly when modelling wave phenomena. When using standard numerical quadrature rules to evaluate highly oscillatory integrals, one requires a fixed number of points per wavelength to maintain accuracy across all frequencies of interest. Several oscillatory quadrature methods exist, but in contrast to standard quadrature rules (such as Gauss and Clenshaw-Curtis), effective use requires a priori analysis of the integral and, thus, a strong understanding of the method. This makes highly oscillatory quadrature rules inaccessible to non-experts.

A popular approach for evaluating highly oscillatory integrals is "Steepest Descent". The idea behind Steepest Descent methods is to deform the integration range onto complex contours where the integrand is non-oscillatory and exponentially decaying. By Cauchy's Theorem, the value of the integral is unchanged. Practically, this reformulation is advantageous, because exponential decay is far more amenable to asymptotic and numerical evaluation. As with other oscillatory quadrature rules, if naively applied, Steepest Descent methods can break down when the phase function contains coalescing stationary points.

In this talk, I will present a new algorithm based on Steepest Descent, which evaluates oscillatory integrals with a cost independent of frequency. The two main novelties are: (1) robustness - cost and accuracy are unaffected by coalescing stationary points, and (2) automation - no expertise or a priori analysis is required to use the algorithm.

26 mei 2023 12:30 t/m 13:15

[NA] Olaf Steinbach: Space-time finite and boundary element methods for the wave equation

In this talk we will review some recent results on space-time finite and
boundary element methods for the wave equation. As a first model problem
we consider the inhomogeneous wave equation with zero Dirichlet boundary
and initial conditions. The related space-time variational formulation
follows the standard approach when applying integration by parts in space
and time simultaneously. Note that a space-time tensor-product based
finite element discretization requires a CFL condition to ensure stability.
While the ansatz and test spaces are both subspaces of functions whose
space-time gradient is square integrable, they differ in zero initial
and terminal conditions to be satisfied. When introducing a modified
Hilbert transformation we end up with a Galerkin variational formulation
which is unconditionally stable. This modified Hilbert transformation is
also an essential tool in the formulation of coercive boundary integral
equations for the wave equation. Finally we also consider distributed
optimal control problems subject to the wave equation, and related
space-time least-squares finite and boundary element methods.

The talk is based on joint work with Marco Zank (Vienna), Richard
Löscher (Graz), Carolina Urzua-Torres (Delft), and Daniel Hoonhout (Delft).

21 april 2023 12:30 t/m 13:15

[NA] Melven Röhrig-Zöllner: Performance of low-rank linear solvers in tensor-train format

In this talk we discuss the problem of efficiently computing a low-rank solution of high-dimensional linear systems. More specifically, we discuss several methods for linear systems in the tensor-train format, also known as matrix-product-states (MPS) in physics. In particular, we consider global approaches like TT-GMRES and local approaches like TT-(M)ALS and TT-AMEn and look at suitable preconditioners and some algorithmic variants for non-symmetric operators. Overall, we focus on the computational complexity and on the performance on today's multi-core CPUs: The considered algorithms are composed of tensor contractions and of dense linear algebra operations like QR-decompositions and singular value decompositions (SVDs). We show significant speedup by carefully choosing suitable combinations of building blocks (e.g. using a tall-skinny QR + SVD). In addition, we show how to exploit orthogonalities from previous steps to speed-up tensor-train truncations. We illustrate the different effects in numerical experiments for simple Laplace- and convection-diffusion equations.

17 maart 2023 12:30 t/m 13:30

[NA] Hendrik Speleers: Explicit error estimates for spline approximation in isogeometric analysis

Isogeometric analysis is a well established paradigm to improve interoperability between geometric modeling and numerical simulations. It is commonly based on B-splines and shows important advantages over classical finite element analysis. In particular, the inherently high smoothness of B-splines leads to a higher accuracy per degree of freedom. This has been numerically observed in a wide range of applications, and recently a mathematical explanation has been given thanks to error estimates in spline spaces that are explicit, not only in the mesh size, but also in the polynomial degree and the smoothness.

In this talk we review some recent results on explicit error estimates for approximation by splines. These estimates are sharp or very close to sharp in several interesting cases and allow us to explain that certain spline subspaces are able to approximate eigenvalue problems without spurious outliers.

17 februari 2023 12:30 t/m 13:30

[NA] Fernando José Henriquez Barraza: Shape Uncertainty Quantification in Acoustic and Electromagnetic Scattering

In this talk, we consider the propagation of acoustic and electromagnetic waves in domains of uncertain shape. We are particularly interested in quantifying the effect on these perturbations on the involved fields and possibly into other quantities of interest. After considering a domain or surface parametrization with countably-many parameters, one obtains a high-dimensional parametric map describing the problem's solution manifold. The design and analysis of a variety of methods commonly used in computational uncertainty quantification (UQ for short) affording provably dimension-independent convergence rates rely on the holomorphic dependence of the problem's solution upon the parametric input. When the parametric input encodes a family of domain or boundary transformations, the holomorphic dependence of the problem upon the parametric input is usually referred to as shape holomorphy. We present and discuss the key technicalities involved in the verification of this property for different models including: volume formulation of the Helmholtz problem, boundary integral formulation, volume integral equations, boundary integral formulations for multiple disjoint arcs, among others. We discuss the importance of this property in the implementation and analysis of different techniques used in forward and inverse computational shape UQ for the previously described models and the implications in the constructions efficient surrogates using neural networks.

20 januari 2023 12:30 t/m 13:30

[NA] Michiel Hochstenbach: 25+ years of nonnormality, a review

We will review several aspects of nonnormal matrices: well-established concepts, recent developments, and open questions. Aspects that will be mentioned include eigenvector condition numbers, pseudospectra, field of values, scaling/balancing, shift invariancy, Krylov spaces, and two-sided methods. We will mention connections to some key scientific problems such as solving linear systems and eigenvalue problems, as well as some practical implementation tips.

25 november 2022 12:30 t/m 13:30

Alessandro Reali: Some advances in isogeometric analysis of coupled and complex problems

21 oktober 2022 12:30 t/m 13:30

Vandana Dwarka: Scalable solvers for the Helmholtz problem

16 september 2022 12:30 t/m 13:30

Shobhit Jain: Dynamics-based model reduction for nonlinear finite element models

17 juni 2022 12:30 t/m 13:30

Eef van Dongen: Monitoring and numerical modelling of glacier dynamics on Greenland

20 mei 2022 12:30 t/m 13:30

Martin Kühn: Mathematical modeling of infectious diseases

22 april 2022 12:30 t/m 13:30

Thomas Takacs: Smooth isogeometric discretizations for fourth order PDEs

18 maart 2022 12:30 t/m 13:30

Espen Sande: Outlier-free isogeometric discretizations

21 januari 2022 12:30