2.0SOC-PHJun 23
When Networks Substitute for Outcome Surveillance? A Substitution-Complementarity Framework for Behavioral Signals in Predictive MonitoringSepehr Ilami, Qingtao Cao, Babak Heydari
Monitoring systems increasingly fuse dynamic behavioral data with outcome-based surveillance, raising a basic question: when does behavioral data carry predictive information that outcome history lacks? We study this using epidemic forecasting on mobility networks, asking whether mobility networks provide independent predictive signal beyond local outcome-based surveillance. We formalize this as a substitution-complementarity problem over directed, weighted mobility networks. Using a Frisch-Waugh-Lovell variance decomposition, our analytical framework derives domain-agnostic conditions under which network-topology features retain incremental explanatory power beyond autoregressive outcome histories. We instantiate the framework using town-level COVID-19 forecasting in Massachusetts (April 2020-April 2021), constructing mobility networks among 300+ towns from smartphone-derived origin-destination aggregates to extract centrality metrics. An agent-based model on synthetic networks confirms that the regime boundary arises from a generic interaction between macro-scale epidemic state and network topology, rather than dataset-specific artifacts. Prevalence-gated interactions between statewide incidence and network features yield large out-of-sample gains when primary surveillance is degraded (Predict-R2 increases from about 0.60 to 0.83-0.89) but only marginal lift when granular local histories are available (+0.5 percentage points). Gains concentrate during epidemic waves when behavioral responses shift network connectivity rapidly. Framed as a value-of-information problem, the substitution gain reflects the marginal value of behavioral data relative to primary-channel quality. This yields a transferable, cost-aware design rule: integrate topology-aware behavioral signals when primary surveillance is degraded or the network changes rapidly; otherwise, rely on autoregressive baselines.
4.8CLAug 5, 2024
Large Model Strategic Thinking, Small Model Efficiency: Transferring Theory of Mind in Large Language ModelsNunzio Lore, Sepehr Ilami, Babak Heydari
As the performance of larger, newer Large Language Models continues to improve for strategic Theory of Mind (ToM) tasks, the demand for these state-of-the-art models increases commensurately. However, their deployment is costly both in terms of processing power and time. In this paper, we investigate the feasibility of creating smaller, highly-performing specialized algorithms by way of fine-tuning. To do this, we first present a large pre-trained model with 20 unique scenarios that combine different social contexts with games of varying social dilemmas, record its answers, and use them for Q&A fine-tuning on a smaller model of the same family. Our focus is on in-context game-theoretic decision-making, the same domain within which human interaction occurs and that requires both a theory of mind (or a semblance thereof) and an understanding of social dynamics. The smaller model is therefore trained not just on the answers provided, but also on the motivations provided by the larger model, which should contain advice and guidelines to navigate both strategic dilemmas and social cues. We find that the fine-tuned smaller language model consistently bridged the gap in performance between the smaller pre-trained version of the model and its larger relative and that its improvements extended in areas and contexts beyond the ones provided in the training examples, including on out-of-sample scenarios that include completely different game structures. On average for all games, through fine-tuning, the smaller model showed a 46% improvement measured as alignment towards the behavior of the larger model, with 100% representing indistinguishable behavior. When presented with out-of-sample social contexts and games, the fine-tuned model still displays remarkable levels of alignment, reaching an improvement of 18% and 28% respectively.
7.3AISep 16, 2024
Integrated Design and Governance of Agentic AI Systems through Adaptive Information ModulationQiliang Chen, Sepehr Ilami, Nunzio Lore et al.
Modern engineered systems increasingly involve complex sociotechnical environments where multiple agents, including humans and the emerging paradigm of agentic AI powered by large language models, must navigate social dilemmas that pit individual interests against collective welfare. As engineered systems evolve toward multi-agent architectures with autonomous LLM-based agents, traditional governance approaches using static rules or fixed network structures fail to address the dynamic uncertainties inherent in real-world operations. This paper presents a novel framework that integrates adaptive governance mechanisms directly into the design of sociotechnical systems through a unique separation of agent interaction networks from information flow networks. We introduce a system comprising strategic LLM-based system agents that engage in repeated interactions and a reinforcement learning-based governing agent that dynamically modulates information transparency. Unlike conventional approaches that require direct structural interventions or payoff modifications, our framework preserves agent autonomy while promoting cooperation through adaptive information governance. The governing agent learns to strategically adjust information disclosure at each timestep, determining what contextual or historical information each system agent can access. Experimental results demonstrate that this RL-based governance significantly enhances cooperation compared to static information-sharing baselines.