Priyam Das

Priyam Das

In this paper, a novel derivative-free pattern search based algorithm for Black-box optimization is proposed over a simplex constrained parameter space. At each iteration, starting from the current solution, new possible set of solutions are found by adding a set of derived step-size vectors to the initial starting point. While deriving these step-size vectors, precautions and adjustments are considered so that the set of new possible solution points still remain within the simplex constrained space. Thus, no extra time is spent in evaluating the (possibly expensive) objective function at infeasible points (points outside the unit-simplex space). While minimizing any objective function of m parameters, within each iteration, the objective function is evaluated at 2m new possible solution points. So, upto 2m parallel threads can be incorporated which makes the computation even faster while optimizing expensive objective functions over high-dimensional parameter space. Once a local minimum is discovered, in order to find a better solution, a novel `re-start' strategy is considered to increase the likelihood of finding a better solution. Unlike existing pattern search based methods, a sparsity control parameter is introduced which can be used to induce sparsity in the solution in case the solution is expected to be sparse in prior. A comparative study of the performances of the proposed algorithm and other existing algorithms are shown for a few low, moderate and high-dimensional optimization problems. Upto 338 folds improvement in computation time is achieved using the proposed algorithm over Genetic algorithm along with better solution. The proposed algorithm is used to estimate the simultaneous quantiles of North Atlantic Hurricane velocities during 1981-2006 by maximizing a non-closed form likelihood function with (possibly) multiple maximums.

Beomchang Kim, Subhrajyoty Roy, Priyam Das

Optimization over the Stiefel manifold $\mathrm{St}(p,d)$, the set of $p \times d$ column-orthonormal matrices, is fundamental in statistics, machine learning, and scientific computing, yet remains challenging in the presence of non-convex, non-smooth, or black-box objectives. Existing methods largely rely on either convex relaxations or gradient-based Riemannian optimization, limiting applicability in derivative-free and highly multimodal settings. We propose \textsc{BOOOM} (Black-box Optimization Over Orthonormal Manifolds), a general-purpose framework for loss-function-agnostic optimization on $\mathrm{St}(p,d)$. The key idea is a global Givens rotation-based parametrization that maps the manifold to an unconstrained Euclidean angle space while preserving feasibility exactly. Building on this representation, BOOOM employs a structured, parallelizable, derivative-free search based on Recursive Modified Pattern Search, enabling systematic exploration through plane-wise rotations without requiring gradient information and facilitating escape from poor local optima. We establish a unified theoretical framework showing equivalence between angle-space and manifold optimization, transfer of stationarity, and global convergence in probability under mild conditions. Empirical results across diverse problems, including heterogeneous quadratic optimization, low-rank and sparse matrix decomposition, independent component analysis, and orthogonal joint diagonalization, among other widely studied settings, demonstrate strong performance relative to state-of-the-art methods, particularly in non-smooth and highly multimodal regimes. We further illustrate its practical utility through a novel supervised PCA formulation applied to metabolomics data in colorectal cancer.