6.9CVMar 16
PAKAN: Pixel Adaptive Kolmogorov-Arnold Network Modules for PansharpeningHaoyu Zhang, Haojing Chen, Zhen Zhong et al.
Pansharpening aims to fuse high-resolution spatial details from panchromatic images with the rich spectral information of multispectral images. Existing deep neural networks for this task typically rely on static activation functions, which limit their ability to dynamically model the complex, non-linear mappings required for optimal spatial-spectral fusion. While the recently introduced Kolmogorov-Arnold Network (KAN) utilizes learnable activation functions, traditional KANs lack dynamic adaptability during inference. To address this limitation, we propose a Pixel Adaptive Kolmogorov-Arnold Network framework. Starting from KAN, we design two adaptive variants: a 2D Adaptive KAN that generates spline summation weights across spatial dimensions and a 1D Adaptive KAN that generates them across spectral channels. These two components are then assembled into PAKAN 2to1 for feature fusion and PAKAN 1to1 for feature refinement. Extensive experiments demonstrate that our proposed modules significantly enhance network performance, proving the effectiveness and superiority of pixel-adaptive activation in pansharpening tasks.
31.3CVApr 25
Toward Real-World Adoption of Portrait Relighting via Hybrid Domain Knowledge FusionQian Huang, Mayoore Selvarasa Jaiswal, Zhen Zhong et al.
The real-world adoption of portrait relighting is hindered by dataset domain gaps, camera sensitivity, and computational costs. We address these challenges with Hybrid Domain Knowledge Fusion, a paradigm that fuses the specialized strengths of synthetic, One-Light-at-A-Time (OLAT), and real-world datasets into a compact model. Our approach features specialized prior models hardened by domain-aware adaptation, followed by augmented knowledge distillation into a lightweight student model with multi-domain expertise. Our method demonstrates a 6x to 240x inference speedup while maintaining state-of-the-art (SOTA) visual quality in the experiments. Additionally, we construct a massive, high-fidelity synthetic dataset with diverse ground-truth intrinsics to support our training pipeline.
CVMar 29, 2024
Separate, Dynamic and Differentiable (SMART) Pruner for Block/Output Channel Pruning on Computer Vision TasksGuanhua Ding, Zexi Ye, Zhen Zhong et al.
Block pruning, which eliminates contiguous blocks of weights, is a structural pruning method that can significantly enhance the performance of neural processing units (NPUs). In industrial applications, an ideal block pruning algorithm should meet three key requirements: (1) maintain high accuracy across diverse models and tasks, as machine learning deployments on edge devices are typically accuracy-critical; (2) offer precise control over resource constraints to facilitate user adoption; and (3) provide convergence guarantees to prevent performance instability. However, to the best of our knowledge, no existing block pruning algorithm satisfies all three requirements simultaneously. In this paper, we introduce SMART (Separate, Dynamic, and Differentiable) pruning, a novel algorithm designed to address this gap. SMART leverages both weight and activation information to enhance accuracy, employs a differentiable top-k operator for precise control of resource constraints, and offers convergence guarantees under mild conditions. Extensive experiments involving seven models, four datasets, three different block types, and three computer vision tasks demonstrate that SMART pruning achieves state-of-the-art performance in block pruning.