Jeffrey Quesnelle

Bowen Peng, Jeffrey Quesnelle, Honglu Fan et al.

Rotary Position Embeddings (RoPE) have been shown to effectively encode positional information in transformer-based language models. However, these models fail to generalize past the sequence length they were trained on. We present YaRN (Yet another RoPE extensioN method), a compute-efficient method to extend the context window of such models, requiring 10x less tokens and 2.5x less training steps than previous methods. Using YaRN, we show that LLaMA models can effectively utilize and extrapolate to context lengths much longer than their original pre-training would allow, while also surpassing previous the state-of-the-art at context window extension. In addition, we demonstrate that YaRN exhibits the capability to extrapolate beyond the limited context of a fine-tuning dataset. Code is available at https://github.com/jquesnelle/yarn

Jeffrey Quesnelle

Zcash is a fork of Bitcoin with optional anonymity features. While transparent transactions are fully linkable, shielded transactions use zero-knowledge proofs to obscure the parties and amounts of the transactions. First, we observe various metrics regarding the usage of shielded addresses. Moreover, we show that most coins sent to shielded addresses are later sent back to transparent addresses. We then search for round-trip transactions, where the same, or nearly the same number of coins are sent from a transparent address, to a shielded address, and back again to a transparent address. We argue that such behavior exhibits high linkability, especially when they occur nearby temporally. Using this heuristic our analysis matched 31.5% of all coins sent to shielded addresses.