Anders Karlsson

Johan Helsing, Anders Karlsson

A high-order accurate, explicit kernel-split, panel-based, Fourier-Nyström discretization scheme is developed for integral equations associated with the Helmholtz equation in axially symmetric domains. Extensive incorporation of analytic information about singular integral kernels and on-the-fly computation of nearly singular quadrature rules allow for very high achievable accuracy, also in the evaluation of fields close to the boundary of the computational domain.

Johan Helsing, Anders Karlsson

In this paper we consider the classic problems of scattering of waves from perfectly conducting cylinders with piecewise smooth boundaries. The scattering problems are formulated as integral equations and solved using a Nyström scheme where the corners of the cylinders are efficiently handled by a method referred to as Recursively Compressed Inverse Preconditioning (RCIP). This method has been very successful in treating static problems in non-smooth domains and the present paper shows that it works equally well for the Helmholtz equation. In the numerical examples we specialize to scattering of E- and H-waves from a cylinder with one corner. Even at a size kd=1000, where k is the wavenumber and d the diameter, the scheme produces at least 13 digits of accuracy in the electric and magnetic fields everywhere outside the cylinder.

Benoit Dherin, Benny Avelin, Anders Karlsson et al.

Despite exceptional achievements, training neural networks remains computationally expensive and is often plagued by instabilities that can degrade convergence. While learning rate schedules can help mitigate these issues, finding optimal schedules is time-consuming and resource-intensive. This work explores theoretical issues concerning training stability in the constant-learning-rate (i.e., without schedule) and small-batch-size regime. Surprisingly, we show that the order of gradient updates affects stability and convergence in gradient-based optimizers. We illustrate this new line of thinking using backward-SGD, which processes batch gradient updates like SGD but in reverse order. Our theoretical analysis shows that in contractive regions (e.g., around minima) backward-SGD converges to a point while the standard forward-SGD generally only converges to a distribution. This leads to improved stability and convergence which we demonstrate experimentally. While full backward-SGD is computationally intensive in practice, it highlights opportunities to exploit reverse training dynamics (or more generally alternate iteration orders) to improve training. To our knowledge, this represents a new and unexplored avenue in deep learning optimization.

Johan Helsing, Anders Karlsson

An integral equation-based numerical method for scattering from multi-dielectric cylinders is presented. Electromagnetic fields are represented via layer potentials in terms of surface densities with physical interpretations. The existence of null-field representations then adds superior flexibility to the modeling. Local representations are used for fast field evaluation at points away from their sources. Partially global representations, constructed as to reduce the strength of kernel singularities, are used for near-evaluations. A mix of local- and partially global representations is also used to derive the system of integral equations from which the physical densities are solved. Unique solvability is proven for the special case of scattering from a homogeneous cylinder under rather general conditions. High achievable accuracy is demonstrated for several examples found in the literature.

Benny Avelin, Anders Karlsson

We consider dynamical and geometrical aspects of deep learning. For many standard choices of layer maps we display semi-invariant metrics which quantify differences between data or decision functions. This allows us, when considering random layer maps and using non-commutative ergodic theorems, to deduce that certain limits exist when letting the number of layers tend to infinity. We also examine the random initialization of standard networks where we observe a surprising cut-off phenomenon in terms of the number of layers, the depth of the network. This could be a relevant parameter when choosing an appropriate number of layers for a given learning task, or for selecting a good initialization procedure. More generally, we hope that the notions and results in this paper can provide a framework, in particular a geometric one, for a part of the theoretical understanding of deep neural networks.

Johan Helsing, Anders Karlsson

The magnetic field integral equation for axially symmetric cavities with perfectly conducting piecewise smooth surfaces is discretized according to a high-order convergent Fourier--Nyström scheme. The resulting solver is used to accurately determine eigenwavenumbers and normalized electric eigenfields in the entire computational domain.