Yan He

Xiaru Meng, Yulan Ju, Yan He et al.

Live cultural experiences like concerts generate shared physiological arousal among audience members, a collective resonance that contributes to their emotional power. Recreating such experiences in virtual reality therefore requires not just audiovisual fidelity, but reproduction of this physiological dimension. Yet current VR evaluation methods rely on post-hoc self-reports that interrupt immersion and cannot capture moment-to-moment arousal dynamics. We propose cross-temporal physiological synchrony as an unobtrusive methodology for evaluating VR cultural recreations: measuring how closely a VR participant's arousal patterns align with those of the original live audience. In a two-phase study, we recorded electrodermal activity from 40 live concert attendees, then created three VR recreations with varying abstraction levels (realistic 360-degree video, mixed video-plus-visualization, and fully abstract physiological representations) and measured synchrony with 22 laboratory participants using Dynamic Time Warping. Contrary to assumptions favoring realism, abstract visualizations achieved the strongest synchrony with live audiences. During musical climaxes, the abstract condition maintained correlation while realistic video showed none. These findings suggest that abstract physiological representations may be more effective than realistic footage for evoking authentic collective engagement in VR cultural recreations.

Yuanlong Li, Gaopan Huang, Min Zhou et al.

Learning an explainable classifier often results in low accuracy model or ends up with a huge rule set, while learning a deep model is usually more capable of handling noisy data at scale, but with the cost of hard to explain the result and weak at generalization. To mitigate this gap, we propose an end-to-end deep explainable learning approach that combines the advantage of deep model in noise handling and expert rule-based interpretability. Specifically, we propose to learn a deep data assessing model which models the data as a graph to represent the correlations among different observations, whose output will be used to extract key data features. The key features are then fed into a rule network constructed following predefined noisy expert rules with trainable parameters. As these models are correlated, we propose an end-to-end training framework, utilizing the rule classification loss to optimize the rule learning model and data assessing model at the same time. As the rule-based computation is none-differentiable, we propose a gradient linking search module to carry the gradient information from the rule learning model to the data assessing model. The proposed method is tested in an industry production system, showing comparable prediction accuracy, much higher generalization stability and better interpretability when compared with a decent deep ensemble baseline, and shows much better fitting power than pure rule-based approach.

Siwang Zhou, Yan He, Yonghe Liu et al.

Incorporating deep neural networks in image compressive sensing (CS) receives intensive attentions in multimedia technology and applications recently. As deep network approaches learn the inverse mapping directly from the CS measurements, the reconstruction speed is significantly faster than the conventional CS algorithms. However, for existing network based approaches, a CS sampling procedure has to map a separate network model. This may potentially degrade the performance of image CS with block-wise sampling because of blocking artifacts, especially when multiple sampling rates are assigned to different blocks within an image. In this paper, we develop a multichannel deep network for block-based image CS by exploiting inter-block correlation with performance significantly exceeding the current state-of-the-art methods. The significant performance improvement is attributed to block-wise approximation but full image removal of blocking artifacts. Specifically, with our multichannel structure, the image blocks with a variety of sampling rates can be reconstructed in a single model. The initially reconstructed blocks are then capable of being reassembled into a full image to improve the recovered images by unrolling a hand-designed block based CS recovery algorithm. Experimental results demonstrate that the proposed method outperforms the state-of-the-art CS methods by a large margin in terms of objective metrics and subjective visual image quality. Our source codes are available at https://github.com/siwangzhou/DeepBCS.

Yan He, Bing Tu, Bo Liu et al.

Hyperspectral image (HSI) classification constitutes the fundamental research in remote sensing fields. Convolutional Neural Networks (CNNs) and Transformers have demonstrated impressive capability in capturing spectral-spatial contextual dependencies. However, these architectures suffer from limited receptive fields and quadratic computational complexity, respectively. Fortunately, recent Mamba architectures built upon the State Space Model integrate the advantages of long-range sequence modeling and linear computational efficiency, exhibiting substantial potential in low-dimensional scenarios. Motivated by this, we propose a novel 3D-Spectral-Spatial Mamba (3DSS-Mamba) framework for HSI classification, allowing for global spectral-spatial relationship modeling with greater computational efficiency. Technically, a spectral-spatial token generation (SSTG) module is designed to convert the HSI cube into a set of 3D spectral-spatial tokens. To overcome the limitations of traditional Mamba, which is confined to modeling causal sequences and inadaptable to high-dimensional scenarios, a 3D-Spectral-Spatial Selective Scanning (3DSS) mechanism is introduced, which performs pixel-wise selective scanning on 3D hyperspectral tokens along the spectral and spatial dimensions. Five scanning routes are constructed to investigate the impact of dimension prioritization. The 3DSS scanning mechanism combined with conventional mapping operations forms the 3D-spectral-spatial mamba block (3DMB), enabling the extraction of global spectral-spatial semantic representations. Experimental results and analysis demonstrate that the proposed method outperforms the state-of-the-art methods on HSI classification benchmarks.

Bingjian Yao, Weiping Lin, Yan He et al.

The fine-grained annotations in whole slide images (WSIs) show the boundaries of various pathological regions. However, generating such detailed annotation is often costly, whereas the coarse annotations are relatively simpler to produce. Existing methods for refining coarse annotations often rely on extensive training samples or clean datasets, and fail to capture both intra-slide and inter-slide latent sematic patterns, limiting their precision. In this paper, we propose a prototype-guided approach. Specifically, we introduce a local-to-global approach to construct non-redundant representative prototypes by jointly modeling intra-slide local semantics and inter-slide contextual relationships. Then a prototype-guided pseudo-labeling module is proposed for refining coarse annotations. Finally, we employ dynamic data sampling and re-finetuning strategy to train a patch classifier. Extensive experiments on three publicly available WSI datasets, covering lymph, liver, and colorectal cancers, demonstrate that our method significantly outperforms existing state-of-the-art (SOTA) methods. The code will be available.

Rui Zhang, Xiaoyang Hou, Zhihua Tian et al.

Graph clustering is an unsupervised machine learning method that partitions the nodes in a graph into different groups. Despite achieving significant progress in exploiting both attributed and structured data information, graph clustering methods often face practical challenges related to data isolation. Moreover, the absence of collaborative methods for graph clustering limits their effectiveness. In this paper, we propose a collaborative graph clustering framework for attributed graphs, supporting attributed graph clustering over vertically partitioned data with different participants holding distinct features of the same data. Our method leverages a novel technique that reduces the sample space, improving the efficiency of the attributed graph clustering method. Furthermore, we compare our method to its centralized counterpart under a proximity condition, demonstrating that the successful local results of each participant contribute to the overall success of the collaboration. We fully implement our approach and evaluate its utility and efficiency by conducting experiments on four public datasets. The results demonstrate that our method achieves comparable accuracy levels to centralized attributed graph clustering methods. Our collaborative graph clustering framework provides an efficient and effective solution for graph clustering challenges related to data isolation.

Zhenyu Zhao, Yan He, Miao Chen

Online controlled experimentation is widely adopted for evaluating new features in the rapid development cycle for web products and mobile applications. Measurement of the overall experiment sample is a common practice to quantify the overall treatment effect. In order to understand why the treatment effect occurs in a certain way, segmentation becomes a valuable approach to a finer analysis of experiment results. This paper introduces a framework for creating and utilizing user behavioral segments in online experimentation. By using the data of user engagement with individual product components as input, this method defines segments that are closely related to the features being evaluated in the product development cycle. With a real-world example, we demonstrate that the analysis with such behavioral segments offered deep, actionable insights that successfully informed product decision-making.

Zhiyu Chen, Zhanghao Yu, Qing Jin et al.

A compact, accurate, and bitwidth-programmable in-memory computing (IMC) static random-access memory (SRAM) macro, named CAP-RAM, is presented for energy-efficient convolutional neural network (CNN) inference. It leverages a novel charge-domain multiply-and-accumulate (MAC) mechanism and circuitry to achieve superior linearity under process variations compared to conventional IMC designs. The adopted semi-parallel architecture efficiently stores filters from multiple CNN layers by sharing eight standard 6T SRAM cells with one charge-domain MAC circuit. Moreover, up to six levels of bit-width of weights with two encoding schemes and eight levels of input activations are supported. A 7-bit charge-injection SAR (ciSAR) analog-to-digital converter (ADC) getting rid of sample and hold (S&H) and input/reference buffers further improves the overall energy efficiency and throughput. A 65-nm prototype validates the excellent linearity and computing accuracy of CAP-RAM. A single 512x128 macro stores a complete pruned and quantized CNN model to achieve 98.8% inference accuracy on the MNIST data set and 89.0% on the CIFAR-10 data set, with a 573.4-giga operations per second (GOPS) peak throughput and a 49.4-tera operations per second (TOPS)/W energy efficiency.

Yan He, Liang Feng Zhang

The problem of securely outsourcing computation to cloud servers has attracted a large amount of attention in recent years. The verifiable computation of Gennaro, Gentry, Parno (Crypto'10) allows a client to verify the server's computation of a function with substantially less time than performing the outsourced computation from scratch. In a multi-function model (Parno, Raykova, Vaikuntanathan; TCC'12) of verifiable computation, the process of encoding function and the process of preparing input are decoupled such that any client can freely submit a computation request on its input, without having to generate an encoding of the function in advance. In this paper, we propose a multi-matrix verifiable computation scheme that allows the secure outsourcing of the matrix functions over a finite field. Our scheme is outsourceable. When it is used to outsource $m$ linear functions, the scheme is roughly $m$ times faster and has less communication cost than the previously best known scheme by Fiore and Gennaro (CCS'12), both in the client-side computation and in the server-side computation. We also show the cost saving with detailed implementations.

Sheng Ren, Yan He, Neal N. Xiong et al.

Incremental learning is a form of online learning. Incremental learning can modify the parameters and structure of the deep learning model so that the model does not forget the old knowledge while learning new knowledge. Preventing catastrophic forgetting is the most important task of incremental learning. However, the current incremental learning is often only for one type of input. For example, if the input images are of the same type, the current incremental model can learn new knowledge while not forgetting old knowledge. However, if several categories are added to the input graphics, the current model will not be able to deal with it correctly, and the accuracy will drop significantly. Therefore, this paper proposes a kind of incremental method, which adjusts the parameters of the model by identifying the prototype vector and increasing the distance of the vector, so that the model can learn new knowledge without catastrophic forgetting. Experiments show the effectiveness of our proposed method.