Zehui Jiang

Xin Zhao, Zehui Jiang, Naoki Yoshinaga

While feed-forward neurons in pre-trained language models (PLMs) can encode knowledge, past research targeted a small subset of neurons that heavily influence outputs. This leaves the broader role of neuron activations unclear, limiting progress in areas like knowledge editing. We uncover a global linear relationship between neuron activations and outputs using neuron interventions on a knowledge probing dataset. The gradient of this linear relationship, which we call the neuron empirical gradient (NEG), captures how changes in activations affect predictions. To compute NEG efficiently, we propose NeurGrad, enabling large-scale analysis of neuron behavior in PLMs. We also show that NEG effectively captures language skills across diverse prompts through skill neuron probing. Experiments on MCEval8k, a multi-genre multiple-choice knowledge benchmark, support NEG's ability to represent model knowledge. Further analysis highlights the key properties of NEG-based skill representation: efficiency, robustness, flexibility, and interdependency. The code and data are released.

Zehui Jiang, Chao Liu, Nathan P. Hendricks et al.

Corn yield prediction is beneficial as it provides valuable information about production and prices prior the harvest. Publicly available high-quality corn yield prediction can help address emergent information asymmetry problems and in doing so improve price efficiency in futures markets. This paper is the first to employ Long Short-Term Memory (LSTM), a special form of Recurrent Neural Network (RNN) method to predict corn yields. A cross sectional time series of county-level corn yield and hourly weather data made the sample space large enough to use deep learning technics. LSTM is efficient in time series prediction with complex inner relations, which makes it suitable for this task. The empirical results from county level data in Iowa show promising predictive power relative to existing survey based methods.