Huiwen Dong

Jiaming Zhang, Jitao Sang, Qi Yi et al.

ImageNet pre-training has enabled state-of-the-art results on many tasks. In spite of its recognized contribution to generalization, we observed in this study that ImageNet pre-training also transfers adversarial non-robustness from pre-trained model into fine-tuned model in the downstream classification tasks. We first conducted experiments on various datasets and network backbones to uncover the adversarial non-robustness in fine-tuned model. Further analysis was conducted on examining the learned knowledge of fine-tuned model and standard model, and revealed that the reason leading to the non-robustness is the non-robust features transferred from ImageNet pre-trained model. Finally, we analyzed the preference for feature learning of the pre-trained model, explored the factors influencing robustness, and introduced a simple robust ImageNet pre-training solution. Our code is available at \url{https://github.com/jiamingzhang94/ImageNet-Pretraining-transfers-non-robustness}.

Meiliang Liu, Huiwen Dong, Xiaoxiao Yang et al.

With the advancement of deep learning technologies, various neural network-based Granger causality models have been proposed. Although these models have demonstrated notable improvements, several limitations remain. Most existing approaches adopt the component-wise architecture, necessitating the construction of a separate model for each time series, which results in substantial computational costs. In addition, imposing the sparsity-inducing penalty on the first-layer weights of the neural network to extract causal relationships weakens the model's ability to capture complex interactions. To address these limitations, we propose Gradient Regularization-based Neural Granger Causality (GRNGC), which requires only one time series prediction model and applies $L_{1}$ regularization to the gradient between model's input and output to infer Granger causality. Moreover, GRNGC is not tied to a specific time series forecasting model and can be implemented with diverse architectures such as KAN, MLP, and LSTM, offering enhanced flexibility. Numerical simulations on DREAM, Lorenz-96, fMRI BOLD, and CausalTime show that GRNGC outperforms existing baselines and significantly reduces computational overhead. Meanwhile, experiments on real-world DNA, Yeast, HeLa, and bladder urothelial carcinoma datasets further validate the model's effectiveness in reconstructing gene regulatory networks.

Huiwen Dong, Jing Yu, Chuangbai Xiao

The robust principal component analysis (RPCA), which aims to estimate underlying low-rank and sparse structures from the degraded observation data, has found wide applications in computer vision. It is usually replaced by the principal component pursuit (PCP) model in order to pursue the convex property, leading to the undesirable overshrink problem. In this paper, we propose a dual weighted lp-norm (DWLP) model with a more reasonable weighting rule and weaker powers, which greatly generalizes the previous work and provides a better approximation to the rank minimization problem for original matrix as well as the l0-norm minimization problem for sparse data. Moreover, an approximate closed-form solution is introduced to solve the lp-norm minimization, which has more stability in the nonconvex optimization and provides a more accurate estimation for the low-rank and sparse matrix recovery problem. We then apply the DWLP model to remove salt-and-pepper noise by exploiting the image nonlocal self-similarity. Both qualitative and quantitative experiments demonstrate that the proposed method outperforms other state-of-the-art methods. In terms of PSNR evaluation, our DWLP achieves about 7.188dB, 5.078dB, 3.854dB, 2.536dB and 0.158dB improvements over the current WSNM-RPCA under 10\% to 50\% salt-and-pepper noise with an interval 10\% respectively.