Hongyu Lu

Bowen Zheng, Hongyu Lu, Yu Chen et al.

Recently, generative recommendation has emerged as a promising paradigm, attracting significant research attention. The basic framework involves an item tokenizer, which represents each item as a sequence of codes serving as its identifier, and a generative recommender that predicts the next item by autoregressively generating the target item identifier. However, in existing methods, both the tokenizer and the recommender are typically domain-specific, limiting their ability for effective transfer or adaptation to new domains. To this end, we propose UTGRec, a Universal item Tokenization approach for transferable Generative Recommendation. Specifically, we design a universal item tokenizer for encoding rich item semantics by adapting a multimodal large language model (MLLM). By devising tree-structured codebooks, we discretize content representations into corresponding codes for item tokenization. To effectively learn the universal item tokenizer on multiple domains, we introduce two key techniques in our approach. For raw content reconstruction, we employ dual lightweight decoders to reconstruct item text and images from discrete representations to capture general knowledge embedded in the content. For collaborative knowledge integration, we assume that co-occurring items are similar and integrate collaborative signals through co-occurrence alignment and reconstruction. Finally, we present a joint learning framework to pre-train and adapt the transferable generative recommender across multiple domains. Extensive experiments on four public datasets demonstrate the superiority of UTGRec compared to both traditional and generative recommendation baselines.

Rohan Mahadev, Hongyu Lu

Our brains combine vision and hearing to create a more elaborate interpretation of the world. When the visual input is insufficient, a rich panoply of sounds can be used to describe our surroundings. Since more than 1,000 hours of videos are uploaded to the internet everyday, it is arduous, if not impossible, to manually annotate these videos. Therefore, incorporating audio along with visual data without annotations is crucial for leveraging this explosion of data for recognizing and understanding objects and scenes. Owens,et.al suggest that a rich representation of the physical world can be learned by using a convolutional neural network to predict sound textures associated with a given video frame. We attempt to reproduce the claims from their experiments, of which the code is not publicly available. In addition, we propose improvements in the pretext task that result in better performance in other downstream computer vision tasks.

Hongyu Lu, Chongguang Wu, Shanglian Bao

A new method based on Shannon-Happ formula to calculate transfer function from Signal Flow Graph (SFG) is presented. The algorithm provides an explicit approach to get the transfer function in a format with both numerical and symbolic expressions. The adoption of the symbolic variable in SFG, which could represent the nonlinear item or the independent sub-system, is achieved by variable separation approach. An investigation is given for the solutions of several special conditions of SFG. To improve the efficiency of the algorithm, a new technique combined with Johnson method for generating the combinations of the non-touching loops is developed. It uses the previous combinations in lower order to get the ones in higher order. There is an introduction about the visualization of SFG and the subroutines for system performance analysis in the software, AVANT.