Jennifer Chan

Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri et al. · allen-ai, berkeley

Modern artificial intelligence (AI) systems are powered by foundation models. This paper presents a new set of foundation models, called Llama 3. It is a herd of language models that natively support multilinguality, coding, reasoning, and tool usage. Our largest model is a dense Transformer with 405B parameters and a context window of up to 128K tokens. This paper presents an extensive empirical evaluation of Llama 3. We find that Llama 3 delivers comparable quality to leading language models such as GPT-4 on a plethora of tasks. We publicly release Llama 3, including pre-trained and post-trained versions of the 405B parameter language model and our Llama Guard 3 model for input and output safety. The paper also presents the results of experiments in which we integrate image, video, and speech capabilities into Llama 3 via a compositional approach. We observe this approach performs competitively with the state-of-the-art on image, video, and speech recognition tasks. The resulting models are not yet being broadly released as they are still under development.

Steven Y. K. Wong, Jennifer Chan, Lamiae Azizi et al.

We consider the problem of neural network training in a time-varying context. Machine learning algorithms have excelled in problems that do not change over time. However, problems encountered in financial markets are often time-varying. We propose the online early stopping algorithm and show that a neural network trained using this algorithm can track a function changing with unknown dynamics. We compare the proposed algorithm to current approaches on predicting monthly U.S. stock returns and show its superiority. We also show that prominent factors (such as the size and momentum effects) and industry indicators, exhibit time varying stock return predictiveness. We find that during market distress, industry indicators experience an increase in importance at the expense of firm level features. This indicates that industries play a role in explaining stock returns during periods of heightened risk.

Nick James, Max Menzies, Jennifer Chan

This paper introduces new methods for analysing the extreme and erratic behaviour of time series to evaluate the impact of COVID-19 on cryptocurrency market dynamics. Across 51 cryptocurrencies, we examine extreme behaviour through a study of distribution extremities, and erratic behaviour through structural breaks. First, we analyse the structure of the market as a whole and observe a reduction in self-similarity as a result of COVID-19, particularly with respect to structural breaks in variance. Second, we compare and contrast these two behaviours, and identify individual anomalous cryptocurrencies. Tether (USDT) and TrueUSD (TUSD) are consistent outliers with respect to their returns, while Holo (HOT), NEXO (NEXO), Maker (MKR) and NEM (XEM) are frequently observed as anomalous with respect to both behaviours and time. Even among a market known as consistently volatile, this identifies individual cryptocurrencies that behave most irregularly in their extreme and erratic behaviour and shows these were more affected during the COVID-19 market crisis.

Nick James, Max Menzies, Lamiae Azizi et al.

This paper proposes a new method for determining similarity and anomalies between time series, most practically effective in large collections of (likely related) time series, by measuring distances between structural breaks within such a collection. We introduce a class of \emph{semi-metric} distance measures, which we term \emph{MJ distances}. These semi-metrics provide an advantage over existing options such as the Hausdorff and Wasserstein metrics. We prove they have desirable properties, including better sensitivity to outliers, while experiments on simulated data demonstrate that they uncover similarity within collections of time series more effectively. Semi-metrics carry a potential disadvantage: without the triangle inequality, they may not satisfy a "transitivity property of closeness." We analyse this failure with proof and introduce an computational method to investigate, in which we demonstrate that our semi-metrics violate transitivity infrequently and mildly. Finally, we apply our methods to cryptocurrency and measles data, introducing a judicious application of eigenvalue analysis.