Tomoya Takeuchi

Yusuke Sakemi, Tomoya Takeuchi, Takeo Hosomi et al.

Continuous-time spiking neural networks (SNNs) provide an event-driven framework for temporal computation, computational neuroscience, and neuromorphic hardware. However, training deep continuous-time SNNs is severely constrained by the memory required for exact spike-time computation, which evaluates and retains candidate firing times over intervals determined by presynaptic spike ordering. Here we introduce a memory-efficient training framework based on differentiable spike-time discretization (DSTD) for leaky integrate-and-fire neurons with general membrane and synaptic time constants. DSTD maps irregular presynaptic spikes onto differentiable weighted events at fixed time points, replacing the input-dependent candidate dimension with $M$ fixed time intervals while accurately approximating continuous-time membrane-potential dynamics. This reduces candidate-related activation memory from $O(N_{\mathrm{out}}N_{\mathrm{in}})$ to $O(N_{\mathrm{out}}M)$ in the case of time-to-first-spike (TTFS) coding, where $N_{\mathrm{in}}$ and $N_{\mathrm{out}}$ denote the numbers of presynaptic and postsynaptic neurons, respectively. We further introduce synfire-chain-inspired temporal regularization that organizes layer-wise firing windows, mitigates dead-neuron failures, and enables pipeline-like processing. In dense LIF layers, DSTD reduced peak memory consumption by up to approximately 100-fold and training time by up to approximately 20-fold compared with exact spike-time computation. Together, these methods allowed us to train 9-layer convolutional SNNs on CIFAR-10 and 20-layer convolutional SNNs on Fashion-MNIST on a single GPU.

Kazufumi Ito, Bangti Jin, Tomoya Takeuchi

We study multi-parameter Tikhonov regularization, i.e., with multiple penalties. Such models are useful when the sought-for solution exhibits several distinct features simultaneously. Two choice rules, i.e., discrepancy principle and balancing principle, are studied for choosing an appropriate (vector-valued) regularization parameter, and some theoretical results are presented. In particular, the consistency of the discrepancy principle as well as convergence rate are established, and an a posteriori error estimate for the balancing principle is established. Also two fixed point algorithms are proposed for computing the regularization parameter by the latter rule. Numerical results for several nonsmooth multi-parameter models are presented, which show clearly their superior performance over their single-parameter counterparts.

Kazufumi Ito, Tomoya Takeuchi

We propose a multi-moment method for one-dimensional hyperbolic equations with smooth coefficient and piecewise constant coefficient. The method is entirely based on the backward characteristic method and uses the solution and its derivative as unknowns and cubic Hermite interpolation for each computational cell. The exact update formula for solution and its derivative is derived and used for an efficient time integration. At points of discontinuity of wave speed we define a piecewise cubic Hermite interpolation based on immersed interface method. The method is extended to the one-dimensional Maxwell's equations with variable material properties.

Kazufumi Ito, Tomoya Takeuchi

We develop a numerical scheme for solving time-domain Maxwell's equation. The method is motivated by CIP method which uses function values and its derivatives as unknown variables. The proposed scheme is developed by using the Poisson formula for the wave equation. It is fully explicit space and time integration method with higher order accuracy and CFL number being one. The bi-cubic interpolation is used for the solution profile to attain the resolution. It preserves sharp profiles very accurately without any smearing and distortion due to the exact time integration and high resolution approximation. The stability and numerical accuracy are investigated.