From Scattered Sources to Comprehensive Technology Landscape: A Recommendation-based Retrieval Approach
This addresses the information overload problem for market actors needing to make informed investment decisions, representing an incremental improvement over existing methods.
The paper tackles the problem of automatically mapping technology landscapes from scattered web data by proposing an end-to-end recommendation-based retrieval approach, which returns 4 times more relevant companies and outperforms traditional baselines in technology retrieval.
Mapping the technology landscape is crucial for market actors to take informed investment decisions. However, given the large amount of data on the Web and its subsequent information overload, manually retrieving information is a seemingly ineffective and incomplete approach. In this work, we propose an end-to-end recommendation based retrieval approach to support automatic retrieval of technologies and their associated companies from raw Web data. This is a two-task setup involving (i) technology classification of entities extracted from company corpus, and (ii) technology and company retrieval based on classified technologies. Our proposed framework approaches the first task by leveraging DistilBERT which is a state-of-the-art language model. For the retrieval task, we introduce a recommendation-based retrieval technique to simultaneously support retrieving related companies, technologies related to a specific company and companies relevant to a technology. To evaluate these tasks, we also construct a data set that includes company documents and entities extracted from these documents together with company categories and technology labels. Experiments show that our approach is able to return 4 times more relevant companies while outperforming traditional retrieval baseline in retrieving technologies.