Gurpreet Singh

Elham Peimani, Gurpreet Singh, Nisarg Mahyavanshi et al.

Retrieving semantically relevant documents in niche domains poses significant challenges for traditional TF-IDF-based systems, often resulting in low similarity scores and suboptimal retrieval performance. This paper addresses these challenges by introducing an iterative and semi-automated query refinement methodology tailored to Humber College's career services webpages. Initially, generic queries related to interview preparation yield low top-document similarities (approximately 0.2--0.3). To enhance retrieval effectiveness, we implement a two-fold approach: first, domain-aware query refinement by incorporating specialized terms such as resources-online-learning, student-online-services, and career-advising; second, the integration of structured educational descriptors like "online resume and interview improvement tools." Additionally, we automate the extraction of domain-specific keywords from top-ranked documents to suggest relevant terms for query expansion. Through experiments conducted on five baseline queries, our semi-automated iterative refinement process elevates the average top similarity score from approximately 0.18 to 0.42, marking a substantial improvement in retrieval performance. The implementation details, including reproducible code and experimental setups, are made available in our GitHub repositories \url{https://github.com/Elipei88/HumberChatbotBackend} and \url{https://github.com/Nisarg851/HumberChatbot}. We also discuss the limitations of our approach and propose future directions, including the integration of advanced neural retrieval models.

Gurpreet Singh, Soumyajit Gupta, Clint N. Dawson

Dense nets are an integral part of any classification and regression problem. Recently, these networks have found a new application as solvers for known representations in various domains. However, one crucial issue with dense nets is it's feature interpretation and lack of reproducibility over multiple training runs. In this work, we identify a basis collapse issue as a primary cause and propose a modified loss function that circumvents this problem. We also provide a few general guidelines relating the choice of activations to loss surface roughness and appropriate scaling for designing low-weight dense nets. We demonstrate through carefully chosen numerical experiments that the basis collapse issue leads to the design of massively redundant networks. Our approach results in substantially concise nets, having $100 \times$ fewer parameters, while achieving a much lower $(10\times)$ MSE loss at scale than reported in prior works. Further, we show that the width of a dense net is acutely dependent on the feature complexity. This is in contrast to the dimension dependent width choice reported in prior theoretical works. To the best of our knowledge, this is the first time these issues and contradictions have been reported and experimentally verified. With our design guidelines we render transparency in terms of a low-weight network design. We share our codes for full reproducibility available at https://github.com/smjtgupta/Dense_Net_Regress.

Gurpreet Singh, Soumyajit Gupta

SVD serves as an exploratory tool in identifying the dominant features in the form of top rank-r singular factors corresponding to the largest singular values. For Big Data applications it is well known that Singular Value Decomposition (SVD) is restrictive due to main memory requirements. However, a number of applications such as community detection, clustering, or bottleneck identification in large scale graph data-sets rely upon identifying the lowest singular values and the singular corresponding vectors. For example, the lowest singular values of a graph Laplacian reveal the number of isolated clusters (zero singular values) or bottlenecks (lowest non-zero singular values) for undirected, acyclic graphs. A naive approach here would be to perform a full SVD however, this quickly becomes infeasible for practical big data applications due to the enormous memory requirements. Furthermore, for such applications only a few lowest singular factors are desired making a full decomposition computationally exorbitant. In this work, we trivially extend the previously proposed Range-Net to \textbf{Tail-Net} for a memory and compute efficient extraction of lowest singular factors of a given big dataset and a specified rank-r. We present a number of numerical experiments on both synthetic and practical data-sets for verification and bench-marking using conventional SVD as the baseline.

Gurpreet Singh, Soumyajit Gupta, Clint Dawson

Hyperspectral unmixing involves separating a pixel as a weighted combination of its constituent endmembers and corresponding fractional abundances, with the current state of the art results achieved by neural models on benchmark datasets. However, these networks are severely over-parameterized and consequently, the invariant endmember spectra extracted as decoder weights have a high variance over multiple runs. These approaches perform substantial post-processing while requiring an exact specification of the number of endmembers and specialized initialization of weights from other algorithms like VCA. We show for the first time that a two-layer autoencoder (SCA), with $2FK$ parameters ($F$ features, $K$ endmembers), achieves error metrics that are scales apart ($10^{-5})$ from previously reported values $(10^{-2})$. SCA converges to this low error solution starting from a random initialization of weights. We also show that SCA, based upon a bi-orthogonal representation, performs a self-correction when the number of endmembers are over-specified. Numerical experiments on Samson, Jasper, and Urban datasets demonstrate that SCA outperforms previously reported error metrics for all the cases while being robust to noise and outliers.

Gurpreet Singh, Soumyajit Gupta, Matthew Lease et al.

In a Big Data setting computing the dominant SVD factors is restrictive due to the main memory requirements. Recently introduced streaming Randomized SVD schemes work under the restrictive assumption that the singular value spectrum of the data has exponential decay. This is seldom true for any practical data. Although these methods are claimed to be applicable to scientific computations due to associated tail-energy error bounds, the approximation errors in the singular vectors and values are high when the aforementioned assumption does not hold. Furthermore from a practical perspective, oversampling can still be memory intensive or worse can exceed the feature dimension of the data. To address these issues, we present Range-Net as an alternative to randomized SVD that satisfies the tail-energy lower bound given by Eckart-Young-Mirsky (EYM) theorem. Range-Net is a deterministic two-stage neural optimization approach with random initialization, where the main memory requirement depends explicitly on the feature dimension and desired rank, independent of the sample dimension. The data samples are read in a streaming setting with the network minimization problem converging to the desired rank-r approximation. Range-Net is fully interpretable where all the network outputs and weights have a specific meaning. We provide theoretical guarantees that Range-Net extracted SVD factors satisfy EYM tail-energy lower bound at machine precision. Our numerical experiments on real data at various scales confirms this bound. A comparison against the state of the art streaming Randomized SVD shows that Range-Net accuracy is better by six orders of magnitude while being memory efficient.

Gurpreet Singh, Mary F. Wheeler

We present an adaptive space-time mesh refinement approach based a domain decomposition approach (Singh and Wheeler, 2018) that allows different time-step sizes and mesh refinements in different subdomains. Our numerical experiments indicate that non-linear solvers fail to converge, to the desired tolerance, due to large non-linear residuals in a smaller subdomain. We exploit this feature to identify subdomains where smaller time-step sizes are necessary while using large time-step sizes in the rest of the reservoir domain. The three key components of our approach are: (1) a space-time, enhanced velocity, domain decomposition approach that allows different mesh refinements and time-step sizes in different subdomains while preserving local mass conservation, (2) a residual based error estimator to identify or mark regions (or subdomains) that pose non-linear convergence issues, and (3) a fully coupled monolithic solver is also presented that solves the coarse and fine subdomain problems, both in space and time, simultaneously. This solution scheme is fully implicit and is therefore unconditionally stable. The proposed space-time domain decomposition approach, with smaller time-step sizes in a subdomain and large time-step sizes everywhere else, circumvents the non-linear convergence issue without adding computational costs. Additionally, a space-time monolithic solver renders a massively parallel, time concurrent framework for solving flow and transport problems in subsurface porous media. Since the proposed approach is similar to the widely used finite difference scheme, it can be easily integrated in any existing legacy reservoir simulator.