Machine Generated Product Advertisements: Benchmarking LLMs Against Human Performance
This research addresses the problem of evaluating AI-generated content for e-commerce, providing insights into current capabilities and limitations, though it is incremental as it benchmarks existing models.
The study compared AI-generated and human-written product descriptions across multiple evaluation metrics, finding that ChatGPT 4 performed best while other models produced incoherent and illogical output.
This study compares the performance of AI-generated and human-written product descriptions using a multifaceted evaluation model. We analyze descriptions for 100 products generated by four AI models (Gemma 2B, LLAMA, GPT2, and ChatGPT 4) with and without sample descriptions, against human-written descriptions. Our evaluation metrics include sentiment, readability, persuasiveness, Search Engine Optimization(SEO), clarity, emotional appeal, and call-to-action effectiveness. The results indicate that ChatGPT 4 performs the best. In contrast, other models demonstrate significant shortcomings, producing incoherent and illogical output that lacks logical structure and contextual relevance. These models struggle to maintain focus on the product being described, resulting in disjointed sentences that do not convey meaningful information. This research provides insights into the current capabilities and limitations of AI in the creation of content for e-Commerce.