Bo Wang

Jiahao Ying, Yixin Cao, Yushi Bai et al.

Large language models (LLMs) have achieved impressive performance across various natural language benchmarks, prompting a continual need to curate more difficult datasets for larger LLMs, which is costly and time-consuming. In this paper, we propose to automate dataset updating and provide systematic analysis regarding its effectiveness in dealing with benchmark leakage issue, difficulty control, and stability. Thus, once the current benchmark has been mastered or leaked, we can update it for timely and reliable evaluation. There are two updating strategies: 1) mimicking strategy to generate similar samples based on original data, preserving stylistic and contextual essence, and 2) extending strategy that further expands existing samples at varying cognitive levels by adapting Bloom's taxonomy of educational objectives. Extensive experiments on updated MMLU and BIG-Bench demonstrate the stability of the proposed strategies and find that the mimicking strategy can effectively alleviate issues of overestimation from benchmark leakage. In cases where the efficient mimicking strategy fails, our extending strategy still shows promising results. Additionally, by controlling the difficulty, we can better discern the models' performance and enable fine-grained analysis neither too difficult nor too easy an exam can fairly judge students' learning status. To the best of our knowledge, we are the first to automate updating benchmarks for reliable and timely evaluation. Our demo leaderboard can be found at https://yingjiahao14.github.io/Automating-DatasetUpdates/.

Ka Kin Lam, Bo Wang

There have been significant efforts devoted to solving the longevity risk given that a continuous growth in population ageing has become a severe issue for many developed countries over the past few decades. The Cairns-Blake-Dowd (CBD) model, which incorporates cohort effects parameters in its parsimonious design, is one of the most well-known approaches for mortality modelling at higher ages and longevity risk. This article proposes a novel mixed-effects time-series approach for mortality modelling and forecasting with considerations of age groups dependence and random cohort effects parameters. The proposed model can disclose more mortality data information and provide a natural quantification of the model parameters uncertainties with no pre-specified constraint required for estimating the cohort effects parameters. The abilities of the proposed approach are demonstrated through two applications with empirical male and female mortality data. The proposed approach shows remarkable improvements in terms of forecast accuracy compared to the CBD model in the short-, mid-and long-term forecasting using mortality data of several developed countries in the numerical examples.

Arkaitz Zubiaga, Bo Wang, Maria Liakata et al.

Social media and data mining are increasingly being used to analyse political and societal issues. Here we undertake the classification of social media users as supporting or opposing ongoing independence movements in their territories. Independence movements occur in territories whose citizens have conflicting national identities; users with opposing national identities will then support or oppose the sense of being part of an independent nation that differs from the officially recognised country. We describe a methodology that relies on users' self-reported location to build large-scale datasets for three territories -- Catalonia, the Basque Country and Scotland. An analysis of these datasets shows that homophily plays an important role in determining who people connect with, as users predominantly choose to follow and interact with others from the same national identity. We show that a classifier relying on users' follow networks can achieve accurate, language-independent classification performances ranging from 85% to 97% for the three territories.