Zhi Wang

h-index29
2papers
3,148citations

2 Papers

11.3CVJul 6, 2024Code
PRANCE: Joint Token-Optimization and Structural Channel-Pruning for Adaptive ViT Inference

Ye Li, Chen Tang, Yuan Meng et al.

We introduce PRANCE, a Vision Transformer compression framework that jointly optimizes the activated channels and reduces tokens, based on the characteristics of inputs. Specifically, PRANCE~ leverages adaptive token optimization strategies for a certain computational budget, aiming to accelerate ViTs' inference from a unified data and architectural perspective. However, the joint framework poses challenges to both architectural and decision-making aspects. Firstly, while ViTs inherently support variable-token inference, they do not facilitate dynamic computations for variable channels. To overcome this limitation, we propose a meta-network using weight-sharing techniques to support arbitrary channels of the Multi-head Self-Attention and Multi-layer Perceptron layers, serving as a foundational model for architectural decision-making. Second, simultaneously optimizing the structure of the meta-network and input data constitutes a combinatorial optimization problem with an extremely large decision space, reaching up to around $10^{14}$, making supervised learning infeasible. To this end, we design a lightweight selector employing Proximal Policy Optimization for efficient decision-making. Furthermore, we introduce a novel "Result-to-Go" training mechanism that models ViTs' inference process as a Markov decision process, significantly reducing action space and mitigating delayed-reward issues during training. Extensive experiments demonstrate the effectiveness of PRANCE~ in reducing FLOPs by approximately 50\%, retaining only about 10\% of tokens while achieving lossless Top-1 accuracy. Additionally, our framework is shown to be compatible with various token optimization techniques such as pruning, merging, and sequential pruning-merging strategies. The code is available at \href{https://github.com/ChildTang/PRANCE}{https://github.com/ChildTang/PRANCE}.

12.6CRMay 17, 2020
Time-Travel Investigation: Towards Building A Scalable Attack Detection Framework on Ethereum

Lei Wu, Siwei Wu, Yajin Zhou et al.

As one of the representative blockchain platforms, Ethereum has attracted lots of attacks. Due to the existed financial loss, there is a pressing need to perform timely investigation and detect more attack instances. Though multiple systems have been proposed, they suffer from the scalability issue due to the following reasons. First, the tight coupling between malicious contract detection and blockchain data importing makes them infeasible to repeatedly detect different attacks. Second, the coarse-grained archive data makes them inefficient to replay transactions. Third, the separation between malicious contract detection and runtime state recovery consumes lots of storage. In this paper, we present the design of a scalable attack detection framework on Ethereum. It overcomes the scalability issue by saving the Ethereum state into a database and providing an efficient way to locate suspicious transactions. The saved state is fine-grained to support the replay of arbitrary transactions. The state is well-designed to avoid saving unnecessary state to optimize the storage consumption. We implement a prototype named EthScope and solve three technical challenges, i.e., incomplete Ethereum state, scalability, and extensibility. The performance evaluation shows that our system can solve the scalability issue, i.e., efficiently performing a large-scale analysis on billions of transactions, and a speedup of around 2,300x when replaying transactions. It also has lower storage consumption compared with existing systems. The result with three different types of information as inputs shows that our system can help an analyst understand attack behaviors and further detect more attacks. To engage the community, we will release our system and the dataset of detected attacks.