Generative AI Systems: A Systems-based Perspective on Generative AI
This work addresses foundational issues in AI system design for researchers and developers, but it is incremental as it builds on existing GenAI advancements without presenting new experimental results.
The paper tackles the challenge of designing and understanding Generative AI systems (GenAISys) that integrate multimodal processing, content creation, and decision-making using natural language and tools, proposing new research directions for their development and analysis.
Large Language Models (LLMs) have revolutionized AI systems by enabling communication with machines using natural language. Recent developments in Generative AI (GenAI) like Vision-Language Models (GPT-4V) and Gemini have shown great promise in using LLMs as multimodal systems. This new research line results in building Generative AI systems, GenAISys for short, that are capable of multimodal processing and content creation, as well as decision-making. GenAISys use natural language as a communication means and modality encoders as I/O interfaces for processing various data sources. They are also equipped with databases and external specialized tools, communicating with the system through a module for information retrieval and storage. This paper aims to explore and state new research directions in Generative AI Systems, including how to design GenAISys (compositionality, reliability, verifiability), build and train them, and what can be learned from the system-based perspective. Cross-disciplinary approaches are needed to answer open questions about the inner workings of GenAI systems.