Xilin Zhang

Jinhui Han, Ming Hu, Xilin Zhang

Transformer-based agentic AI is rapidly being deployed on major platforms to help users shop, watch, and navigate content with less effort. While these systems can deliver impressive performance, a key concern is whether they may be less reliable than they appear. We ask a simple but fundamental question: whether the mechanisms that make transformer-based agents effective can also induce systematic biases or distortions? We study this question through a theoretical analysis of transformer-based generative recommenders, in which the next user interaction is generated sequentially from the user history. Focusing on how the model allocates attention across historical evidence, we identify four bias channels: (i) Positional bias: stronger positional encoding shifts influence toward recent history, improving responsiveness but potentially reducing stability and long-term diversity; (ii) Popularity amplification: small frequency differences in data can be magnified into disproportionate exposure, contributing to Matthew effects and echo chambers; (iii) Latent driver bias: when important drivers of user choices are not directly observed, the model can place overly concentrated weight on a small subset of past events, creating overconfident attributions. (iv) Synthetic data bias: when users increasingly follow AI suggestions and platforms retrain on model-shaped synthetic logs, outputs can concentrate over time, and long-tail alternatives can disappear first. Our analysis highlights mechanism-level reliability risks that may not be visible in offline performance metrics. The four bias channels indicate that large-scale deployment may systematically distort exposure and choice. For managers, the immediate implication is to treat these as operational risk factors and to monitor concentration and drift over time, rather than assuming that performance gains alone guarantee reliability.

Cheng Chen, Xilin Zhang, Yizhou Wang et al.

In this study, we propose a novel method to measure bottom-up saliency maps of natural images. In order to eliminate the influence of top-down signals, backward masking is used to make stimuli (natural images) subjectively invisible to subjects, however, the bottom-up saliency can still orient the subjects attention. To measure this orientation/attention effect, we adopt the cueing effect paradigm by deploying discrimination tasks at each location of an image, and measure the discrimination performance variation across the image as the attentional effect of the bottom-up saliency. Such attentional effects are combined to construct a final bottomup saliency map. Based on the proposed method, we introduce a new bottom-up saliency map dataset of natural images to benchmark computational models. We compare several state-of-the-art saliency models on the dataset. Moreover, the proposed paradigm is applied to investigate the neural basis of the bottom-up visual saliency map by analyzing psychophysical and fMRI experimental results. Our findings suggest that the bottom-up saliency maps of natural images are constructed in V1. It provides a strong scientific evidence to resolve the long standing dispute in neuroscience about where the bottom-up saliency map is constructed in human brain.