Hang Zou

Lina Bariah, Qiyang Zhao, Hang Zou et al.

The evolution of generative artificial intelligence (GenAI) constitutes a turning point in reshaping the future of technology in different aspects. Wireless networks in particular, with the blooming of self-evolving networks, represent a rich field for exploiting GenAI and reaping several benefits that can fundamentally change the way how wireless networks are designed and operated nowadays. To be specific, large GenAI models are envisioned to open up a new era of autonomous wireless networks, in which multi-modal GenAI models trained over various Telecom data, can be fine-tuned to perform several downstream tasks, eliminating the need for building and training dedicated AI models for each specific task and paving the way for the realization of artificial general intelligence (AGI)-empowered wireless networks. In this article, we aim to unfold the opportunities that can be reaped from integrating large GenAI models into the Telecom domain. In particular, we first highlight the applications of large GenAI models in future wireless networks, defining potential use-cases and revealing insights on the associated theoretical and practical challenges. Furthermore, we unveil how 6G can open up new opportunities through connecting multiple on-device large GenAI models, and hence, paves the way to the collective intelligence paradigm. Finally, we put a forward-looking vision on how large GenAI models will be the key to realize self-evolving networks.

Lina Bariah, Hang Zou, Qiyang Zhao et al.

The recent progress of artificial intelligence (AI) opens up new frontiers in the possibility of automating many tasks involved in Telecom networks design, implementation, and deployment. This has been further pushed forward with the evolution of generative artificial intelligence (AI), including the emergence of large language models (LLMs), which is believed to be the cornerstone toward realizing self-governed, interactive AI agents. Motivated by this, in this paper, we aim to adapt the paradigm of LLMs to the Telecom domain. In particular, we fine-tune several LLMs including BERT, distilled BERT, RoBERTa and GPT-2, to the Telecom domain languages, and demonstrate a use case for identifying the 3rd Generation Partnership Project (3GPP) standard working groups. We consider training the selected models on 3GPP technical documents (Tdoc) pertinent to years 2009-2019 and predict the Tdoc categories in years 2020-2023. The results demonstrate that fine-tuning BERT and RoBERTa model achieves 84.6% accuracy, while GPT-2 model achieves 83% in identifying 3GPP working groups. The distilled BERT model with around 50% less parameters achieves similar performance as others. This corroborates that fine-tuning pretrained LLM can effectively identify the categories of Telecom language. The developed framework shows a stepping stone towards realizing intent-driven and self-evolving wireless networks from Telecom languages, and paves the way for the implementation of generative AI in the Telecom domain.

Hang Zou, Qiyang Zhao, Yu Tian et al.

Large Language Models (LLMs) have the potential to revolutionize the Sixth Generation (6G) communication networks. However, current mainstream LLMs generally lack the specialized knowledge in telecom domain. In this paper, for the first time, we propose a pipeline to adapt any general purpose LLMs to a telecom-specific LLMs. We collect and build telecom-specific pre-train dataset, instruction dataset, preference dataset to perform continual pre-training, instruct tuning and alignment tuning respectively. Besides, due to the lack of widely accepted evaluation benchmarks in telecom domain, we extend existing evaluation benchmarks and proposed three new benchmarks, namely, Telecom Math Modeling, Telecom Open QnA and Telecom Code Tasks. These new benchmarks provide a holistic evaluation of the capabilities of LLMs including math modeling, Open-Ended question answering, code generation, infilling, summarization and analysis in telecom domain. Our fine-tuned LLM TelecomGPT outperforms state of the art (SOTA) LLMs including GPT-4, Llama-3 and Mistral in Telecom Math Modeling benchmark significantly and achieve comparable performance in various evaluation benchmarks such as TeleQnA, 3GPP technical documents classification, telecom code summary and generation and infilling.

Qiyang Zhao, Hang Zou, Mehdi Bennis et al.

In this work, we study the problem of semantic communication and inference, in which a student agent (i.e. mobile device) queries a teacher agent (i.e. cloud sever) to generate higher-order data semantics living in a simplicial complex. Specifically, the teacher first maps its data into a k-order simplicial complex and learns its high-order correlations. For effective communication and inference, the teacher seeks minimally sufficient and invariant semantic structures prior to conveying information. These minimal simplicial structures are found via judiciously removing simplices selected by the Hodge Laplacians without compromising the inference query accuracy. Subsequently, the student locally runs its own set of queries based on a masked simplicial convolutional autoencoder (SCAE) leveraging both local and remote teacher's knowledge. Numerical results corroborate the effectiveness of the proposed approach in terms of improving inference query accuracy under different channel conditions and simplicial structures. Experiments on a coauthorship dataset show that removing simplices by ranking the Laplacian values yields a 85% reduction in payload size without sacrificing accuracy. Joint semantic communication and inference by masked SCAE improves query accuracy by 25% compared to local student based query and 15% compared to remote teacher based query. Finally, incorporating channel semantics is shown to effectively improve inference accuracy, notably at low SNR values.

Thomas Mongaillard, Samson Lasaulce, Othman Hicheur et al.

While traditional optimization and scheduling schemes are designed to meet fixed, predefined system requirements, future systems are moving toward user-driven approaches and personalized services, aiming to achieve high quality-of-experience (QoE) and flexibility. This challenge is particularly pronounced in wireless and digitalized energy networks, where users' requirements have largely not been taken into consideration due to the lack of a common language between users and machines. The emergence of powerful large language models (LLMs) marks a radical departure from traditional system-centric methods into more advanced user-centric approaches by providing a natural communication interface between users and devices. In this paper, for the first time, we introduce a novel architecture for resource scheduling problems by constructing three LLM agents to convert an arbitrary user's voice request (VRQ) into a resource allocation vector. Specifically, we design an LLM intent recognition agent to translate the request into an optimization problem (OP), an LLM OP parameter identification agent, and an LLM OP solving agent. To evaluate system performance, we construct a database of typical VRQs in the context of electric vehicle (EV) charging. As a proof of concept, we primarily use Llama 3 8B. Through testing with different prompt engineering scenarios, the obtained results demonstrate the efficiency of the proposed architecture. The conducted performance analysis allows key insights to be extracted. For instance, having a larger set of candidate OPs to model the real-world problem might degrade the final performance because of a higher recognition/OP classification noise level. All results and codes are open source.

Hang Zou, Qiyang Zhao, Samson Lasaulce et al.

Generative Artificial Intelligence (GenAI) and communication networks are expected to have groundbreaking synergies for 6G. Connecting GenAI agents via a wireless network can potentially unleash the power of Collective Intelligence (CI) and pave the way for Artificial General Intelligence (AGI). However, current wireless networks are designed as a "data pipe" and are not suited to accommodate and leverage the power of GenAI. In this paper, we propose the GenAINet framework in which distributed GenAI agents communicate knowledge (facts, experiences, and methods) to accomplish arbitrary tasks. We first propose an architecture for a single GenAI agent and then provide a network architecture integrating GenAI capabilities to manage both network protocols and applications. Building on this, we investigate effective communication and reasoning problems by proposing a semantic-native GenAINet. Specifically, GenAI agents extract semantics from heterogeneous raw data, build and maintain a knowledge model representing the semantic relationships among pieces of knowledge, which is retrieved by GenAI models for planning and reasoning. Under this paradigm, different levels of collaboration can be achieved flexibly depending on the complexity of targeted tasks. Furthermore, we conduct two case studies in which, through wireless device queries, we demonstrate that extracting, compressing and transferring common knowledge can improve query accuracy while reducing communication costs; and in the wireless power control problem, we show that distributed agents can complete general tasks independently through collaborative reasoning without predefined communication protocols. Finally, we discuss challenges and future research directions in applying Large Language Models (LLMs) in 6G networks.

Hang Zou, Chao Zhang, Samson Lasaulce et al.

Assuming that the number of possible decisions for a transmitter (e.g., the number of possible beamforming vectors) has to be finite and is given, this paper investigates for the first time the problem of determining the best decision set when energy-efficiency maximization is pursued. We propose a framework to find a good (finite) decision set which induces a minimal performance loss w.r.t. to the continuous case. We exploit this framework for a scenario of energy-efficient MIMO communications in which transmit power and beamforming vectors have to be adapted jointly to the channel given under finite-rate feedback. To determine a good decision set we propose an algorithm which combines the approach of Invasive Weed Optimization (IWO) and an Evolutionary Algorithm (EA). We provide a numerical analysis which illustrates the benefits of our point of view. In particular, given a performance loss level, the feedback rate can by reduced by 2 when the transmit decision set has been designed properly by using our algorithm. The impact on energy-efficiency is also seen to be significant.

Hang Zou, Chao Zhang, Samson Lasaulce et al.

In this paper, we introduce the problem of decision-oriented communications, that is, the goal of the source is to send the right amount of information in order for the intended destination to execute a task. More specifically, we restrict our attention to how the source should quantize information so that the destination can maximize a utility function which represents the task to be executed only knowing the quantized information. For example, for utility functions under the form $u\left(\boldsymbol{x};\ \boldsymbol{g}\right)$, $\boldsymbol{x}$ might represent a decision in terms of using some radio resources and $\boldsymbol{g}$ the system state which is only observed through its quantized version $Q(\boldsymbol{g})$. Both in the case where the utility function is known and the case where it is only observed through its realizations, we provide solutions to determine such a quantizer. We show how this approach applies to energy-efficient power allocation. In particular, it is seen that quantizing the state very roughly is perfectly suited to sum-rate-type function maximization, whereas energy-efficiency metrics are more sensitive to imperfections.

Hang Zou, Chao Zhang, Samson Lasaulce

In this paper, we propose a new perspective for quantizing a signal and more specifically the channel state information (CSI). The proposed point of view is fully relevant for a receiver which has to send a quantized version of the channel state to the transmitter. Roughly, the key idea is that the receiver sends the right amount of information to the transmitter so that the latter be able to take its (resource allocation) decision. More formally, the decision task of the transmitter is to maximize an utility function u(x;g) with respect to x (e.g., a power allocation vector) given the knowledge of a quantized version of the function parameters g. We exhibit a special case of an energy-efficient power control (PC) problem for which the optimal task oriented CSI quantizer (TOCQ) can be found analytically. For more general utility functions, we propose to use neural networks (NN) based learning. Simulations show that the compression rate obtained by adapting the feedback information rate to the function to be optimized may be significantly increased.