Jingzhi Li

Jiawei Liang, Siyuan Liang, Aishan Liu et al.

Knowledge Distillation (KD) for object detection aims to train a compact detector by transferring knowledge from a teacher model. Since the teacher model perceives data in a way different from humans, existing KD methods only distill knowledge that is consistent with labels annotated by human expert while neglecting knowledge that is not consistent with human perception, which results in insufficient distillation and sub-optimal performance. In this paper, we propose inconsistent knowledge distillation (IKD), which aims to distill knowledge inherent in the teacher model's counter-intuitive perceptions. We start by considering the teacher model's counter-intuitive perceptions of frequency and non-robust features. Unlike previous works that exploit fine-grained features or introduce additional regularizations, we extract inconsistent knowledge by providing diverse input using data augmentation. Specifically, we propose a sample-specific data augmentation to transfer the teacher model's ability in capturing distinct frequency components and suggest an adversarial feature augmentation to extract the teacher model's perceptions of non-robust features in the data. Extensive experiments demonstrate the effectiveness of our method which outperforms state-of-the-art KD baselines on one-stage, two-stage and anchor-free object detectors (at most +1.0 mAP). Our codes will be made available at \url{https://github.com/JWLiang007/IKD.git}.

Binyi Su, Hua Zhang, Jingzhi Li et al.

Open-set object detection (OSOD) aims to detect the known categories and reject unknown objects in a dynamic world, which has achieved significant attention. However, previous approaches only consider this problem in data-abundant conditions, while neglecting the few-shot scenes. In this paper, we seek a solution for the generalized few-shot open-set object detection (G-FOOD), which aims to avoid detecting unknown classes as known classes with a high confidence score while maintaining the performance of few-shot detection. The main challenge for this task is that few training samples induce the model to overfit on the known classes, resulting in a poor open-set performance. We propose a new G-FOOD algorithm to tackle this issue, named \underline{F}ew-sh\underline{O}t \underline{O}pen-set \underline{D}etector (FOOD), which contains a novel class weight sparsification classifier (CWSC) and a novel unknown decoupling learner (UDL). To prevent over-fitting, CWSC randomly sparses parts of the normalized weights for the logit prediction of all classes, and then decreases the co-adaptability between the class and its neighbors. Alongside, UDL decouples training the unknown class and enables the model to form a compact unknown decision boundary. Thus, the unknown objects can be identified with a confidence probability without any threshold, prototype, or generation. We compare our method with several state-of-the-art OSOD methods in few-shot scenes and observe that our method improves the F-score of unknown classes by 4.80\%-9.08\% across all shots in VOC-COCO dataset settings \footnote[1]{The source code is available at \url{https://github.com/binyisu/food}}.

Siyuan Liang, Longkang Li, Yanbo Fan et al.

Recent studies have shown that detectors based on deep models are vulnerable to adversarial examples, even in the black-box scenario where the attacker cannot access the model information. Most existing attack methods aim to minimize the true positive rate, which often shows poor attack performance, as another sub-optimal bounding box may be detected around the attacked bounding box to be the new true positive one. To settle this challenge, we propose to minimize the true positive rate and maximize the false positive rate, which can encourage more false positive objects to block the generation of new true positive bounding boxes. It is modeled as a multi-objective optimization (MOP) problem, of which the generic algorithm can search the Pareto-optimal. However, our task has more than two million decision variables, leading to low searching efficiency. Thus, we extend the standard Genetic Algorithm with Random Subset selection and Divide-and-Conquer, called GARSDC, which significantly improves the efficiency. Moreover, to alleviate the sensitivity to population quality in generic algorithms, we generate a gradient-prior initial population, utilizing the transferability between different detectors with similar backbones. Compared with the state-of-art attack methods, GARSDC decreases by an average 12.0 in the mAP and queries by about 1000 times in extensive experiments. Our codes can be found at https://github.com/LiangSiyuan21/ GARSDC.

Jingzhi Li, Hongyu Liu, Zaijiu Shang et al.

We develop two inverse scattering schemes for locating multiple electromagnetic (EM) scatterers by the electric far-field measurement corresponding to a single incident/detecting plane wave. The first scheme is for locating scatterers of small size compared to the wavelength of the detecting plane wave. The multiple scatterers could be extremely general with an unknown number of components, and each scatterer component could be either an impenetrable perfectly conducting obstacle or a penetrable inhomogeneous medium with an unknown content. The second scheme is for locating multiple perfectly conducting obstacles of regular size compared to the detecting EM wavelength. The number of the obstacle components is not required to be known in advance, but the shape of each component must be from a certain known admissible class. The admissible class may consist of multiple different reference obstacles. The second scheme could also be extended to include the medium components if a certain generic condition is satisfied. Both schemes are based on some novel indicator functions whose indicating behaviors could be used to locate the scatterers. No inversion will be involved in calculating the indicator functions, and the proposed methods are every efficient and robust to noise. Rigorous mathematical justifications are provided and extensive numerical experiments are conducted to illustrate the effectiveness of the imaging schemes.

Michael V. Klibanov, Jingzhi Li, Wenlong Zhang

We propose a new numerical method to reconstruct the isotropic electrical conductivity from measured restricted Dirichlet-to-Neumann map data in electrical impedance tomography (EIT) model. "Restricted Dirichlet-to-Neumann (DtN) map data" means that the Dirichlet and Neumann boundary data for EIT are generated by a point source running either along an interval of a straight line or along a curve located outside of the domain of interest. We "convexify" the problem via constructing a globally strictly convex Tikhonov-like functional using a Carleman Weight Function. In particular, two new Carleman estimates are established. Global convergenceto the correct solution of the gradient projection method for this functional is proven. Numerical examples demonstrate a good performance of this numerical procedure.

Deyue Zhang, Yukun Guo, Jingzhi Li et al.

This work is concerned with the inverse source problem of locating multiple multipolar sources from boundary measurements for the Helmholtz equation. We develop simple and effective sampling schemes for location acquisition of the sources with a single wavenumber. Our algorithms are based on some novel indicator functions whose indicating behaviors could be used to locate multiple multipolar sources. The inversion schemes are totally "direct" in the sense that only simple integral calculations are involved in evaluating the indicator functions. Rigorous mathematical justifications are provided and extensive numerical examples are presented to demonstrate the effectiveness, robustness and efficiency of the proposed methods.

Michael V. Klibanov, Jingzhi Li, Tian Niu et al.

The first globally convergent numerical method is developed for a coefficient inverse problem (CIP) for the $n-$d, $n\geq 2$ wave equation with the unknown potential in the most challenging case when the $δ-$ function is present in the initial condition with a single location of the point source. In fact, an approximate mathematical model for that CIP is derived. That globally convergent numerical method is developed for this model. This is a new version of the so-called convexification numerical method. Uniqueness theorem is proven as well within the framework of that approximate mathematical model. The question about uniqueness of this CIP was first posed by a famous mathematician I. M. Gelfand in 1954 as an $n-$d ($n=2,3$) extension of the fundamental theorem of V.A. Marchenko in the 1-d case (1950). Based on a Carleman estimate, convergence analysis is carried out. This analysis ensures the global convergence of the proposed numerical method, i.e. it is not necessary to have a good first guess for the solution. Exhaustive computational experiments with noisy data demonstrate a high reconstruction accuracy of complicated structures. In particular, this accuracy points towards a high adequacy of that approximate mathematical model.

Huipeng Gu, Mingchao Cai, Jingzhi Li et al.

This paper introduces a unified model for thermo-poroelasticity and multiple-network poroelasticity, reformulated into a total-pressure-based system. We first establish the well-posedness of the problem via a Galerkin-based argument and subsequently introduce a robust space-time finite element approximation. To efficiently solve the fully coupled system, we propose a global-in-time iterative algorithm that sequentially decouples the mechanics from the transport equations, while incorporating necessary stabilization terms. We explicitly analyze the convergence rate and provide a rigorous proof that the proposed scheme constitutes a contraction mapping under physically relevant conditions, thereby ensuring its unconditional convergence. Numerical experiments confirm the theoretical stability bounds and demonstrate optimal convergence rates in both space and time, yielding solutions free of non-physical pressure oscillations.

Jingzhi Li, Zhiyong Cheng, Richang Hong et al.

Provider exposure fairness is crucial for sustaining a healthy content ecosystem and preventing monopolization in recommender systems. Yet, most existing methods either incorporate fairness constraints during model training, requiring expensive retraining when fairness objectives change, or rely on post-hoc reranking with fixed criteria, which lacks adaptability to diverse fairness requirements. To overcome these limitations, we propose Post-hoc Fairness Adaptation (PFA), a lightweight framework that equips a frozen recommender with a fairness adapter, enabling flexible fairness control without retraining the backbone model. Specifically, the fairness adapter learns personalized additive score adjustments from user-item embeddings, which are injected into the original ranking scores to steer provider exposure toward fairness. To train the adapter, we minimize the KL divergence between the actual and the target fair exposure distributions. However, this global objective implicitly treats all providers equally, ignoring structural disparities such as imbalanced provider group sizes and heterogeneous exposure within groups. Consequently, fairness may appear satisfied at an aggregate level while severe inter-group and intra-group exposure imbalances persist, undermining practical fairness. To address this, we design Hierarchical Exposure Fairness Alignment (HEFA), which explicitly balances inter- and intra-group provider exposure disparities, enabling flexible adaptation to diverse fairness requirements. To mitigate potential accuracy degradation, PFA jointly optimizes HEFA with a differentiable NDCG loss, enabling end-to-end fairness optimization while preserving ranking quality. Extensive experiments on three public datasets demonstrate that PFA achieves substantial fairness gains with negligible accuracy loss, consistently outperforming strong baselines.

Ruoyu Chen, Siyuan Liang, Jingzhi Li et al.

Advances in multimodal pre-training have propelled object-level foundation models, such as Grounding DINO and Florence-2, in tasks like visual grounding and object detection. However, interpreting these models' decisions has grown increasingly challenging. Existing interpretable attribution methods for object-level task interpretation have notable limitations: (1) gradient-based methods lack precise localization due to visual-textual fusion in foundation models, and (2) perturbation-based methods produce noisy saliency maps, limiting fine-grained interpretability. To address these, we propose a Visual Precision Search method that generates accurate attribution maps with fewer regions. Our method bypasses internal model parameters to overcome attribution issues from multimodal fusion, dividing inputs into sparse sub-regions and using consistency and collaboration scores to accurately identify critical decision-making regions. We also conducted a theoretical analysis of the boundary guarantees and scope of applicability of our method. Experiments on RefCOCO, MS COCO, and LVIS show our approach enhances object-level task interpretability over SOTA for Grounding DINO and Florence-2 across various evaluation metrics, with faithfulness gains of 23.7%, 31.6%, and 20.1% on MS COCO, LVIS, and RefCOCO for Grounding DINO, and 102.9% and 66.9% on MS COCO and RefCOCO for Florence-2. Additionally, our method can interpret failures in visual grounding and object detection tasks, surpassing existing methods across multiple evaluation metrics. The code will be released at https://github.com/RuoyuChen10/VPS.

Shangquan Sun, Wenqi Ren, Jingzhi Li et al.

Knowledge distillation involves transferring soft labels from a teacher to a student using a shared temperature-based softmax function. However, the assumption of a shared temperature between teacher and student implies a mandatory exact match between their logits in terms of logit range and variance. This side-effect limits the performance of student, considering the capacity discrepancy between them and the finding that the innate logit relations of teacher are sufficient for student to learn. To address this issue, we propose setting the temperature as the weighted standard deviation of logit and performing a plug-and-play Z-score pre-process of logit standardization before applying softmax and Kullback-Leibler divergence. Our pre-process enables student to focus on essential logit relations from teacher rather than requiring a magnitude match, and can improve the performance of existing logit-based distillation methods. We also show a typical case where the conventional setting of sharing temperature between teacher and student cannot reliably yield the authentic distillation evaluation; nonetheless, this challenge is successfully alleviated by our Z-score. We extensively evaluate our method for various student and teacher models on CIFAR-100 and ImageNet, showing its significant superiority. The vanilla knowledge distillation powered by our pre-process can achieve favorable performance against state-of-the-art methods, and other distillation variants can obtain considerable gain with the assistance of our pre-process.

Ruoyu Chen, Siyuan Liang, Jingzhi Li et al.

To develop a trustworthy AI system, which aim to identify the input regions that most influence the models decisions. The primary task of existing attribution methods lies in efficiently and accurately identifying the relationships among input-prediction interactions. Particularly when the input data is discrete, such as images, analyzing the relationship between inputs and outputs poses a significant challenge due to the combinatorial explosion. In this paper, we propose a novel and efficient black-box attribution mechanism, LiMA (Less input is More faithful for Attribution), which reformulates the attribution of important regions as an optimization problem for submodular subset selection. First, to accurately assess interactions, we design a submodular function that quantifies subset importance and effectively captures their impact on decision outcomes. Then, efficiently ranking input sub-regions by their importance for attribution, we improve optimization efficiency through a novel bidirectional greedy search algorithm. LiMA identifies both the most and least important samples while ensuring an optimal attribution boundary that minimizes errors. Extensive experiments on eight foundation models demonstrate that our method provides faithful interpretations with fewer regions and exhibits strong generalization, shows an average improvement of 36.3% in Insertion and 39.6% in Deletion. Our method also outperforms the naive greedy search in attribution efficiency, being 1.6 times faster. Furthermore, when explaining the reasons behind model prediction errors, the average highest confidence achieved by our method is, on average, 86.1% higher than that of state-of-the-art attribution algorithms. The code is available at https://github.com/RuoyuChen10/LIMA.

Ruoyu Chen, Hua Zhang, Siyuan Liang et al.

Image attribution algorithms aim to identify important regions that are highly relevant to model decisions. Although existing attribution solutions can effectively assign importance to target elements, they still face the following challenges: 1) existing attribution methods generate inaccurate small regions thus misleading the direction of correct attribution, and 2) the model cannot produce good attribution results for samples with wrong predictions. To address the above challenges, this paper re-models the above image attribution problem as a submodular subset selection problem, aiming to enhance model interpretability using fewer regions. To address the lack of attention to local regions, we construct a novel submodular function to discover more accurate small interpretation regions. To enhance the attribution effect for all samples, we also impose four different constraints on the selection of sub-regions, i.e., confidence, effectiveness, consistency, and collaboration scores, to assess the importance of various subsets. Moreover, our theoretical analysis substantiates that the proposed function is in fact submodular. Extensive experiments show that the proposed method outperforms SOTA methods on two face datasets (Celeb-A and VGG-Face2) and one fine-grained dataset (CUB-200-2011). For correctly predicted samples, the proposed method improves the Deletion and Insertion scores with an average of 4.9% and 2.5% gain relative to HSIC-Attribution. For incorrectly predicted samples, our method achieves gains of 81.0% and 18.4% compared to the HSIC-Attribution algorithm in the average highest confidence and Insertion score respectively. The code is released at https://github.com/RuoyuChen10/SMDL-Attribution.

Jingzhi Li, Zongwei Wu, Eduard Zamfir et al.

Accurate 3D objects relighting in diverse unseen environments is crucial for realistic virtual object placement. Due to the albedo-lighting ambiguity, existing methods often fall short in producing faithful relights. Without proper constraints, observed training views can be explained by numerous combinations of lighting and material attributes, lacking physical correspondence with the actual environment maps used for relighting. In this work, we present ReCap, treating cross-environment captures as multi-task target to provide the missing supervision that cuts through the entanglement. Specifically, ReCap jointly optimizes multiple lighting representations that share a common set of material attributes. This naturally harmonizes a coherent set of lighting representations around the mutual material attributes, exploiting commonalities and differences across varied object appearances. Such coherence enables physically sound lighting reconstruction and robust material estimation - both essential for accurate relighting. Together with a streamlined shading function and effective post-processing, ReCap outperforms all leading competitors on an expanded relighting benchmark.

Jingzhi Li, Changjiang Luo, Ruoyu Chen et al.

Recent advances in multimodal large language models (MLLMs) have demonstrated strong capabilities in understanding general visual content. However, these general-domain MLLMs perform poorly in face perception tasks, often producing inaccurate or misleading responses to face-specific queries. To address this gap, we propose FaceInsight, the versatile face perception MLLM that provides fine-grained facial information. Our approach introduces visual-textual alignment of facial knowledge to model both uncertain dependencies and deterministic relationships among facial information, mitigating the limitations of language-driven reasoning. Additionally, we incorporate face segmentation maps as an auxiliary perceptual modality, enriching the visual input with localized structural cues to enhance semantic understanding. Comprehensive experiments and analyses across three face perception tasks demonstrate that FaceInsight consistently outperforms nine compared MLLMs under both training-free and fine-tuned settings.

Ruoyu Chen, Hua Zhang, Jingzhi Li et al.

The objective of few-shot object detection (FSOD) is to detect novel objects with few training samples. The core challenge of this task is how to construct a generalized feature space for novel categories with limited data on the basis of the base category space, which could adapt the learned detection model to unknown scenarios. However, limited by insufficient samples for novel categories, two issues still exist: (1) the features of the novel category are easily implicitly represented by the features of the base category, leading to inseparable classifier boundaries, (2) novel categories with fewer data are not enough to fully represent the distribution, where the model fine-tuning is prone to overfitting. To address these issues, we introduce the side information to alleviate the negative influences derived from the feature space and sample viewpoints and formulate a novel generalized feature representation learning method for FSOD. Specifically, we first utilize embedding side information to construct a knowledge matrix to quantify the semantic relationship between the base and novel categories. Then, to strengthen the discrimination between semantically similar categories, we further develop contextual semantic supervised contrastive learning which embeds side information. Furthermore, to prevent overfitting problems caused by sparse samples, a side-information guided region-aware masked module is introduced to augment the diversity of samples, which finds and abandons biased information that discriminates between similar categories via counterfactual explanation, and refines the discriminative representation space further. Extensive experiments using ResNet and ViT backbones on PASCAL VOC, MS COCO, LVIS V1, FSOD-1K, and FSVOD-500 benchmarks demonstrate that our model outperforms the previous state-of-the-art methods, significantly improving the ability of FSOD in most shots/splits.

Shixuan Sun, Siyuan Liang, Ruoyu Chen et al.

Retrieval-Augmented Generation (RAG) and its Multimodal Retrieval-Augmented Generation (MRAG) significantly improve the knowledge coverage and contextual understanding of Large Language Models (LLMs) by introducing external knowledge sources. However, retrieval and multimodal fusion obscure content provenance, rendering existing membership inference methods unable to reliably attribute generated outputs to pre-training, external retrieval, or user input, thus undermining privacy leakage accountability To address these challenges, we propose the first Source-aware Membership Audit (SMA) that enables fine-grained source attribution of generated content in a semi-black-box setting with retrieval control capabilities. To address the environmental constraints of semi-black-box auditing, we further design an attribution estimation mechanism based on zero-order optimization, which robustly approximates the true influence of input tokens on the output through large-scale perturbation sampling and ridge regression modeling. In addition, SMA introduces a cross-modal attribution technique that projects image inputs into textual descriptions via MLLMs, enabling token-level attribution in the text modality, which for the first time facilitates membership inference on image retrieval traces in MRAG systems. This work shifts the focus of membership inference from 'whether the data has been memorized' to 'where the content is sourced from', offering a novel perspective for auditing data provenance in complex generative systems.

Guan Wang, Fuming Ma, Yukun Guo et al.

This work is concerned with an inverse problem of identifying the current source distribution of the time-harmonic Maxwell's equations from multi-frequency measurements. Motivated by the Fourier method for the scalar Helmholtz equation and the polarization vector decomposition, we propose a novel method for determining the source function in the full vector Maxwell's system. Rigorous mathematical justifications of the method are given and numerical examples are provided to demonstrate the feasibility and effectiveness of the method.

Jingzhi Li, Xiaofei Li, Hongyu Liu

We are concerned with the inverse scattering problem of extracting the geometric structures of an unknown/inaccessible inhomogeneous medium by using the corresponding acoustic far-field measurement. Using the intrinsic geometric properties of the so-called interior transmission eigenfunctions, we develop a novel inverse scattering scheme. The proposed method can efficiently capture the cusp singularities of the support of the inhomogeneous medium. If further a priori information is available on the support of the medium, say, it is a convex polyhedron, then one can actually recover its shape. Both theoretical analysis and numerical experiments are provided. Our reconstruction method is new to the literature and opens up a new direction in the study of inverse scattering problems.

Yukun Guo, Jingzhi Li, Hongyu Liu et al.

This paper is concerned with the mathematical design of a novel input/instruction device using a moving emitter. The emitter generates a point source and can be installed on a digit pen or worn on the finger of the human being who wants to interact/communicate with the computer. The input/instruction can be recognized by identifying the motion trajectory of the emitter performed by the human being from the collected wave field data. The identification process is modelled as an in- verse source problem where one intends to identify the trajectory of a moving point source. There are several salient features of our study which distinguish our result from the existing ones in the literature. First, the point source is moving in an inhomogeneous background medium, which models the human body. Second, the dynamical wave field data are collected in a limited aperture. Third, the recon- struction method is independent of the background medium, and it is totally direct without any matrix inversion. Hence, it is efficient and robust with respect to the measurement noise. Both theoretical justifications and computational experiments are presented to verify our novel findings.

Huangxin Chen, Jingzhi Li, Weifeng Qiu

We propose a robust a posteriori error estimator for the hybridizable discontinuous Galerkin (HDG) method for convection-diffusion equations with dominant convection. The reliability and efficiency of the estimator are established for the error measured in an energy norm. The energy norm is uniformly bounded even when the diffusion coefficient tends to zero. The estimators are robust in the sense that the upper and lower bounds of error are uniformly bounded with respect to the diffusion coefficient. A weighted test function technique and the Oswald interpolation are key ingredients in the analysis. Numerical results verify the robustness of the proposed a posteriori error estimator. In numerical experiments, optimal convergence is observed.

Huangxin Chen, Guosheng Fu, Jingzhi Li et al.

We present and analyze a first order least squares method for convection dominated diffusion problems, which provides robust L2 a priori error estimate for the scalar variable even if the given data f in L2 space. The novel theoretical approach is to rewrite the method in the framework of discontinuous Petrov - Galerkin (DPG) method, and then show numerical stability by using a key equation discovered by J. Gopalakrishnan and W. Qiu [Math. Comp. 83(2014), pp. 537-552]. This new approach gives an alternative way to do numerical analysis for least squares methods for a large class of differential equations. We also show that the condition number of the global matrix is independent of the diffusion coefficient. A key feature of the method is that there is no stabilization parameter chosen empirically. In addition, Dirichlet boundary condition is weakly imposed. Numerical experiments verify our theoretical results and, in particular, show our way of weakly imposing Dirichlet boundary condition is essential to the design of least squares methods - numerical solutions on subdomains away from interior layers or boundary layers have remarkable accuracy even on coarse meshes, which are unstructured quasi-uniform.