Patent Citation Dynamics Modeling via Multi-Attention Recurrent Networks
This work addresses the problem of predicting patent citations for technology discovery and innovation measurement, representing an incremental improvement with a novel method for a known bottleneck.
The paper tackles forecasting forward citations for patents by modeling multiple time sequences, including assignee and inventor histories, using a novel multi-attention recurrent network. It demonstrates superior performance over state-of-the-art models on a USPTO dataset.
Modeling and forecasting forward citations to a patent is a central task for the discovery of emerging technologies and for measuring the pulse of inventive progress. Conventional methods for forecasting these forward citations cast the problem as analysis of temporal point processes which rely on the conditional intensity of previously received citations. Recent approaches model the conditional intensity as a chain of recurrent neural networks to capture memory dependency in hopes of reducing the restrictions of the parametric form of the intensity function. For the problem of patent citations, we observe that forecasting a patent's chain of citations benefits from not only the patent's history itself but also from the historical citations of assignees and inventors associated with that patent. In this paper, we propose a sequence-to-sequence model which employs an attention-of-attention mechanism to capture the dependencies of these multiple time sequences. Furthermore, the proposed model is able to forecast both the timestamp and the category of a patent's next citation. Extensive experiments on a large patent citation dataset collected from USPTO demonstrate that the proposed model outperforms state-of-the-art models at forward citation forecasting.