David Levin

Anat Amir, David Levin

Approximations of non-smooth multivariate functions return low-order approximations in the vicinities of the singularities. Most prior works solve this problem for univariate functions. In this work we introduce a method for approximating non-smooth multivariate functions of the form $f = g + r_+$ where $g,r \in C^{M+1}(\mathbb{R}^n)$ and the function $r_+$ is defined by \[ r_+(y) = \left\{ \begin{array}{ll} r(y), & r(y) \geq 0 \\ 0, & r(y) < 0 \end{array} \right. \ , \ \forall y \in \mathbb{R}^n \ . \] Given scattered (or uniform) data points $X \subset \mathbb{R}^n$, we investigate approximation by quasi-interpolation. We design a correction term, such that the corrected approximation achieves full approximation order on the entire domain. We also show that the correction term is the solution to a Moving Least Squares (MLS) problem, and as such can both be easily computed and is smooth. Last, we prove that the suggested method includes a high-order approximation to the locations of the singularities.

José Kuruc, David Levin, Pep Mulet et al.

In this article we present a modification of classical Radial Basis Function (RBF) interpolation techniques aimed at reducing oscillations near discontinuities in one and two dimensions. Our approach introduces an adaptive mechanism by varying the shape parameter of the RBFs and making it data-dependent, forcing it to tend to infinity in the vicinity of discontinuities. This modification results in kernel functions that locally resemble %Kronecker delta functions, effectively minimizing spurious oscillations. To detect discontinuities, we employ smoothness indicators: for grid-based data, these are computed as undivided second-order differences squared. For scattered data, we use least squares approximations of the Laplacian multiplied by the square of the mean local separation of the stencil points, and then squared. These indicators guide the adaptive adjustment of the shape parameter. We prove the invertibility of the resulting interpolation matrix and propose a solution strategy that maintains the condition number comparable to that of a system where points near discontinuities are excluded. Numerical experiments in one and two dimensions demonstrate that the proposed method significantly reduces oscillations near discontinuities across various kernel types, whether locally or globally supported. At the same time, the interpolation accuracy and matrix conditioning in smooth regions remain essentially unchanged, as measured by the infinity norm of the error and the condition number.

David Levin, Gonen Singer

This paper introduces a novel graph-based filter method for automatic feature selection (abbreviated as GB-AFS) for multi-class classification tasks. The method determines the minimum combination of features required to sustain prediction performance while maintaining complementary discriminating abilities between different classes. It does not require any user-defined parameters such as the number of features to select. The methodology employs the Jeffries-Matusita (JM) distance in conjunction with t-distributed Stochastic Neighbor Embedding (t-SNE) to generate a low-dimensional space reflecting how effectively each feature can differentiate between each pair of classes. The minimum number of features is selected using our newly developed Mean Simplified Silhouette (abbreviated as MSS) index, designed to evaluate the clustering results for the feature selection task. Experimental results on public data sets demonstrate the superior performance of the proposed GB-AFS over other filter-based techniques and automatic feature selection approaches. Moreover, the proposed algorithm maintained the accuracy achieved when utilizing all features, while using only $7\%$ to $30\%$ of the features. Consequently, this resulted in a reduction of the time needed for classifications, from $15\%$ to $70\%$.

Qusay Muzaffar, David Levin, Michael Werman

Global optimization of black-box functions from noisy samples is a fundamental challenge in machine learning and scientific computing. Traditional methods such as Bayesian Optimization often converge to local minima on multi-modal functions, while gradient-free methods require many function evaluations. We present a novel neural approach that learns to find global minima through iterative refinement. Our model takes noisy function samples and their fitted spline representation as input, then iteratively refines an initial guess toward the true global minimum. Trained on randomly generated functions with ground truth global minima obtained via exhaustive search, our method achieves a mean error of 8.05 percent on challenging multi-modal test functions, compared to 36.24 percent for the spline initialization, a 28.18 percent improvement. The model successfully finds global minima in 72 percent of test cases with error below 10 percent, demonstrating learned optimization principles rather than mere curve fitting. Our architecture combines encoding of multiple modalities including function values, derivatives, and spline coefficients with iterative position updates, enabling robust global optimization without requiring derivative information or multiple restarts.

David Levin

We consider the problem of reconstructing missing data on a smooth manifold from incomplete and nonuniform samples. While classical methods for manifold approximation typically assume quasi-uniform data, their performance deteriorates significantly in the presence of large gaps or holes. We propose a unified framework for manifold data imputation that reduces the problem to function reconstruction on locally defined tangent spaces. The approach combines two complementary strategies. The first is a Fourier-based method that determines missing values by prescribing a decay rate of the discrete Fourier coefficients, thereby enforcing high-order smoothness through a global spectral criterion. The second is a local variational method based on minimizing high-order central differences, leading to sparse least-squares systems with favorable stability and conditioning properties. We establish a discrete inverse estimate linking decay of Fourier coefficients to uniform bounds on high-order divided differences, providing a theoretical foundation for the spectral approach. For the variational method, we analyze existence, uniqueness, and scaling behavior, showing that conditioning depends primarily on the geometry of the missing region. These functional reconstruction techniques are integrated with a moving least-squares projection framework to yield a practical algorithm for manifold completion. Numerical experiments, including reconstruction on surfaces with significant missing regions, demonstrate accurate and stable recovery without requiring a global parameterization. The proposed framework provides a flexible and effective approach to manifold data imputation in challenging settings with incomplete data.

David Levin, Gonen Singer

In this paper, we present Automatic Complementary Separation Pruning (ACSP), a novel and fully automated pruning method for convolutional neural networks. ACSP integrates the strengths of both structured pruning and activation-based pruning, enabling the efficient removal of entire components such as neurons and channels while leveraging activations to identify and retain the most relevant components. Our approach is designed specifically for supervised learning tasks, where we construct a graph space that encodes the separation capabilities of each component with respect to all class pairs. By employing complementary selection principles and utilizing a clustering algorithm, ACSP ensures that the selected components maintain diverse and complementary separation capabilities, reducing redundancy and maintaining high network performance. The method automatically determines the optimal subset of components in each layer, utilizing a knee-finding algorithm to select the minimal subset that preserves performance without requiring user-defined pruning volumes. Extensive experiments on multiple architectures, including VGG-16, ResNet-50, and MobileNet-V2, across datasets like CIFAR-10, CIFAR-100, and ImageNet-1K, demonstrate that ACSP achieves competitive accuracy compared to other methods while significantly reducing computational costs. This fully automated approach not only enhances scalability but also makes ACSP especially practical for real-world deployment by eliminating the need for manually defining the pruning volume.

Barak Sober, Yariv Aizenbud, David Levin

We present an algorithm for approximating a function defined over a $d$-dimensional manifold utilizing only noisy function values at locations sampled from the manifold with noise. To produce the approximation we do not require any knowledge regarding the manifold other than its dimension $d$. We use the Manifold Moving Least-Squares approach of (Sober and Levin 2016) to reconstruct the atlas of charts and the approximation is built on-top of those charts. The resulting approximant is shown to be a function defined over a neighborhood of a manifold, approximating the originally sampled manifold. In other words, given a new point, located near the manifold, the approximation can be evaluated directly on that point. We prove that our construction yields a smooth function, and in case of noiseless samples the approximation order is $\mathcal{O}(h^{m+1})$, where $h$ is a local density of sample parameter (i.e., the fill distance) and $m$ is the degree of a local polynomial approximation, used in our algorithm. In addition, the proposed algorithm has linear time complexity with respect to the ambient-space's dimension. Thus, we are able to avoid the computational complexity, commonly encountered in high dimensional approximations, without having to perform non-linear dimension reduction, which inevitably introduces distortions to the geometry of the data. Additionaly, we show numerical experiments that the proposed approach compares favorably to statistical approaches for regression over manifolds and show its potential.

Barak Sober, David Levin

In order to avoid the curse of dimensionality, frequently encountered in Big Data analysis, there was a vast development in the field of linear and nonlinear dimension reduction techniques in recent years. These techniques (sometimes referred to as manifold learning) assume that the scattered input data is lying on a lower dimensional manifold, thus the high dimensionality problem can be overcome by learning the lower dimensionality behavior. However, in real life applications, data is often very noisy. In this work, we propose a method to approximate $\mathcal{M}$ a $d$-dimensional $C^{m+1}$ smooth submanifold of $\mathbb{R}^n$ ($d \ll n$) based upon noisy scattered data points (i.e., a data cloud). We assume that the data points are located "near" the lower dimensional manifold and suggest a non-linear moving least-squares projection on an approximating $d$-dimensional manifold. Under some mild assumptions, the resulting approximant is shown to be infinitely smooth and of high approximation order (i.e., $O(h^{m+1})$, where $h$ is the fill distance and $m$ is the degree of the local polynomial approximation). The method presented here assumes no analytic knowledge of the approximated manifold and the approximation algorithm is linear in the large dimension $n$. Furthermore, the approximating manifold can serve as a framework to perform operations directly on the high dimensional data in a computationally efficient manner. This way, the preparatory step of dimension reduction, which induces distortions to the data, can be avoided altogether.

Barak Sober, David Levin

This work suggests a new variational approach to the task of computer aided restoration of incomplete characters, residing in a highly noisy document. We model character strokes as the movement of a pen with a varying radius. Following this model, a cubic spline representation is being utilized to perform gradient descent steps, while maintaining interpolation at some initial (manually sampled) points. The proposed algorithm was utilized in the process of restoring approximately 1000 ancient Hebrew characters (dating to ca. 8th-7th century BCE), some of which are presented herein and show that the algorithm yields plausible results when applied on deteriorated documents.