The ROOTS Search Tool: Data Transparency for LLMs
This provides a tool for researchers and developers to improve data governance and transparency in LLM training, though it is incremental as it builds on existing corpus efforts.
The researchers tackled the problem of data transparency for large language models by developing the ROOTS Search Tool, a search engine over the 1.6TB ROOTS corpus, which is the largest corpus to date that can be investigated this way, and they made it open-sourced and available on Hugging Face Spaces.
ROOTS is a 1.6TB multilingual text corpus developed for the training of BLOOM, currently the largest language model explicitly accompanied by commensurate data governance efforts. In continuation of these efforts, we present the ROOTS Search Tool: a search engine over the entire ROOTS corpus offering both fuzzy and exact search capabilities. ROOTS is the largest corpus to date that can be investigated this way. The ROOTS Search Tool is open-sourced and available on Hugging Face Spaces. We describe our implementation and the possible use cases of our tool.