Improving the Capabilities of Large Language Model Based Marketing Analytics Copilots With Semantic Search And Fine-Tuning
This work addresses the problem of making AI models more accessible and effective for marketing professionals by reducing the need for extensive implementation teams, though it is incremental as it builds on existing LLM techniques.
The paper tackled the challenge of reliably using large language models for marketing analytics by improving domain-specific question-answering, SQL generation, and tabular analysis through semantic search, prompt engineering, and fine-tuning, resulting in dramatic accuracy improvements in tasks like marketing mix modeling and attribution.
Artificial intelligence (AI) is widely deployed to solve problems related to marketing attribution and budget optimization. However, AI models can be quite complex, and it can be difficult to understand model workings and insights without extensive implementation teams. In principle, recently developed large language models (LLMs), like GPT-4, can be deployed to provide marketing insights, reducing the time and effort required to make critical decisions. In practice, there are substantial challenges that need to be overcome to reliably use such models. We focus on domain-specific question-answering, SQL generation needed for data retrieval, and tabular analysis and show how a combination of semantic search, prompt engineering, and fine-tuning can be applied to dramatically improve the ability of LLMs to execute these tasks accurately. We compare both proprietary models, like GPT-4, and open-source models, like Llama-2-70b, as well as various embedding methods. These models are tested on sample use cases specific to marketing mix modeling and attribution.