Shaohan Feng

Nguyen Cong Luong, Shaohan Feng, Nguyen Duc Hai et al.

Reinforcement Learning (RL) has long been a powerful solution to various problems in communication networks. However, traditional RL models still face with several limitations. Not only do they rely on large numbers of interactions with the environment, but they are also limited in terms of modeling long-term relationships and tackling partial observability. In recent years, the Transformer model has demonstrated the ability to enhance RL models, allowing them to overcome these issues. Particularly, the self-attention mechanism within the Transformer enables efficient modeling of long-range dependencies and global correlations, as well as accelerates training processes and handles heterogeneous data modalities. In this paper, we present a comprehensive survey of Transformer-based RL algorithms and their applications in communication networks. Specifically, the paper provides the mathematical background of RL and Transformer architectures, along with insights into key issues such as resource allocation, computation offloading, routing, and trajectory control, and network security. We conclude the paper by discussing challenges, open issues, and notable future research directions, including Transformer-enhanced DRL algorithms for semantic communication and network optimization.

Shaohan Feng, Dusit Niyato, Ping Wang et al.

For the sake of protecting data privacy and due to the rapid development of mobile devices, e.g., powerful central processing unit (CPU) and nascent neural processing unit (NPU), collaborative machine learning on mobile devices, e.g., federated learning, has been envisioned as a new AI approach with broad application prospects. However, the learning process of the existing federated learning platforms rely on the direct communication between the model owner, e.g., central cloud or edge server, and the mobile devices for transferring the model update. Such a direct communication may be energy inefficient or even unavailable in mobile environments. In this paper, we consider adopting the relay network to construct a cooperative communication platform for supporting model update transfer and trading. In the system, the mobile devices generate model updates based on their training data. The model updates are then forwarded to the model owner through the cooperative relay network. The model owner enjoys the learning service provided by the mobile devices. In return, the mobile devices charge the model owner certain prices. Due to the coupled interference of wireless transmission among the mobile devices that use the same relay node, the rational mobile devices have to choose their relay nodes as well as deciding on their transmission powers. Thus, we formulate a Stackelberg game model to investigate the interaction among the mobile devices and that between the mobile devices and the model owner. The Stackelberg equilibrium is investigated by capitalizing on the exterior point method. Moreover, we provide a series of insightful analytical and numerical results on the equilibrium of the Stackelberg game.

Shaohan Feng, Wenbo Wang, Zehui Xiong et al.

Open-access blockchains based on proof-of-work protocols have gained tremendous popularity for their capabilities of providing decentralized tamper-proof ledgers and platforms for data-driven autonomous organization. Nevertheless, the proof-of-work based consensus protocols are vulnerable to cyber-attacks such as double-spending. In this paper, we propose a novel approach of cyber risk management for blockchain-based service. In particular, we adopt the cyber-insurance as an economic tool for neutralizing cyber risks due to attacks in blockchain networks. We consider a blockchain service market, which is composed of the infrastructure provider, the blockchain provider, the cyber-insurer, and the users. The blockchain provider purchases from the infrastructure provider, e.g., a cloud, the computing resources to maintain the blockchain consensus, and then offers blockchain services to the users. The blockchain provider strategizes its investment in the infrastructure and the service price charged to the users, in order to improve the security of the blockchain and thus optimize its profit. Meanwhile, the blockchain provider also purchases a cyber-insurance from the cyber-insurer to protect itself from the potential damage due to the attacks. In return, the cyber-insurer adjusts the insurance premium according to the perceived risk level of the blockchain service. Based on the assumption of rationality for the market entities, we model the interaction among the blockchain provider, the users, and the cyber-insurer as a two-level Stackelberg game. Namely, the blockchain provider and the cyber-insurer lead to set their pricing/investment strategies, and then the users follow to determine their demand of the blockchain service. Specifically, we consider the scenario of double-spending attacks and provide a series of analytical results about the Stackelberg equilibrium in the market game.

Zehui Xiong, Shaohan Feng, Dusit Niyato et al.

As the core issue of blockchain, the mining requires solving a proof-of-work puzzle, which is resource expensive to implement in mobile devices due to high computing power needed. Thus, the development of blockchain in mobile applications is restricted. In this paper, we consider the edge computing as the network enabler for mobile blockchain. In particular, we study optimal pricing-based edge computing resource management to support mobile blockchain applications where the mining process can be offloaded to an Edge computing Service Provider (ESP). We adopt a two-stage Stackelberg game to jointly maximize the profit of the ESP and the individual utilities of different miners. In Stage I, the ESP sets the price of edge computing services. In Stage II, the miners decide on the service demand to purchase based on the observed prices. We apply the backward induction to analyze the sub-game perfect equilibrium in each stage for uniform and discriminatory pricing schemes. Further, the existence and uniqueness of Stackelberg game are validated for both pricing schemes. At last, the performance evaluation shows that the ESP intends to set the maximum possible value as the optimal price for profit maximization under uniform pricing. In addition, the discriminatory pricing helps the ESP encourage higher total service demand from miners and achieve greater profit correspondingly.

Zehui Xiong, Shaohan Feng, Wenbo Wang et al.

The mining process in blockchain requires solving a proof-of-work puzzle, which is resource expensive to implement in mobile devices due to the high computing power and energy needed. In this paper, we, for the first time, consider edge computing as an enabler for mobile blockchain. In particular, we study edge computing resource management and pricing to support mobile blockchain applications in which the mining process of miners can be offloaded to an edge computing service provider. We formulate a two-stage Stackelberg game to jointly maximize the profit of the edge computing service provider and the individual utilities of the miners. In the first stage, the service provider sets the price of edge computing nodes. In the second stage, the miners decide on the service demand to purchase based on the observed prices. We apply the backward induction to analyze the sub-game perfect equilibrium in each stage for both uniform and discriminatory pricing schemes. For the uniform pricing where the same price is applied to all miners, the existence and uniqueness of Stackelberg equilibrium are validated by identifying the best response strategies of the miners. For the discriminatory pricing where the different prices are applied to different miners, the Stackelberg equilibrium is proved to exist and be unique by capitalizing on the Variational Inequality theory. Further, the real experimental results are employed to justify our proposed model.