Tao Zeng

Bo Liu, Chaowei Tan, Jiazhou Wang et al.

In this paper, we present Fedlearn-Algo, an open-source privacy preserving machine learning platform. We use this platform to demonstrate our research and development results on privacy preserving machine learning algorithms. As the first batch of novel FL algorithm examples, we release vertical federated kernel binary classification model and vertical federated random forest model. They have been tested to be more efficient than existing vertical federated learning models in our practice. Besides the novel FL algorithm examples, we also release a machine communication module. The uniform data transfer interface supports transferring widely used data formats between machines. We will maintain this platform by adding more functional modules and algorithm examples. The code is available at https://github.com/fedlearnAI/fedlearn-algo.

Le Fang, Tao Zeng, Chaochun Liu et al.

We investigate large-scale latent variable models (LVMs) for neural story generation -- an under-explored application for open-domain long text -- with objectives in two threads: generation effectiveness and controllability. LVMs, especially the variational autoencoder (VAE), have achieved both effective and controllable generation through exploiting flexible distributional latent representations. Recently, Transformers and its variants have achieved remarkable effectiveness without explicit latent representation learning, thus lack satisfying controllability in generation. In this paper, we advocate to revive latent variable modeling, essentially the power of representation learning, in the era of Transformers to enhance controllability without hurting state-of-the-art generation effectiveness. Specifically, we integrate latent representation vectors with a Transformer-based pre-trained architecture to build conditional variational autoencoder (CVAE). Model components such as encoder, decoder and the variational posterior are all built on top of pre-trained language models -- GPT2 specifically in this paper. Experiments demonstrate state-of-the-art conditional generation ability of our model, as well as its excellent representation learning capability and controllability.

Le Fang, Tao Zeng, Chaochun Liu et al.

Large-scale pretrained language models have shown thrilling generation capabilities, especially when they generate consistent long text in thousands of words with ease. However, users of these models can only control the prefix of sentences or certain global aspects of generated text. It is challenging to simultaneously achieve fine-grained controllability and preserve the state-of-the-art unconditional text generation capability. In this paper, we first propose a new task named "Outline to Story" (O2S) as a test bed for fine-grained controllable generation of long text, which generates a multi-paragraph story from cascaded events, i.e. a sequence of outline events that guide subsequent paragraph generation. We then create dedicate datasets for future benchmarks, built by state-of-the-art keyword extraction techniques. Finally, we propose an extremely simple yet strong baseline method for the O2S task, which fine tunes pre-trained language models on augmented sequences of outline-story pairs with simple language modeling objective. Our method does not introduce any new parameters or perform any architecture modification, except several special tokens as delimiters to build augmented sequences. Extensive experiments on various datasets demonstrate state-of-the-art conditional story generation performance with our model, achieving better fine-grained controllability and user flexibility. Our paper is among the first ones by our knowledge to propose a model and to create datasets for the task of "outline to story". Our work also instantiates research interest of fine-grained controllable generation of open-domain long text, where controlling inputs are represented by short text.

Yu Li, Hu Wang, Kangjia Mo et al.

A variety of modeling techniques have been developed in the past decade to reduce the computational expense and improve the accuracy of modeling. In this study, a new framework of modeling is suggested. Compared with other popular methods, a distinctive characteristic is "from image based model to analysis based model (e.g. stress, strain, and deformation)". In such a framework, a reconstruction neural network (ReConNN) model designed for simulation-based physical field's reconstruction is proposed. The ReConNN contains two submodels that are convolutional neural network (CNN) and generative adversarial net-work (GAN). The CNN is employed to construct the mapping between contour images of physical field and objective function. Subsequently, the GAN is utilized to generate more images which are similar to the existing contour images. Finally, Lagrange polynomial is applied to complete the reconstruction. However, the existing CNN models are commonly applied to the classification tasks, which seem to be difficult to handle with regression tasks of images. Meanwhile, the existing GAN architectures are insufficient to generate high-accuracy "pseudo contour images". Therefore, a ReConNN model based on a Convolution in Convolution (CIC) and a Convolutional AutoEncoder based on Wasserstein Generative Adversarial Network (WGAN-CAE) is suggested. To evaluate the performance of the proposed model representatively, a classical topology optimization procedure is considered. Then the ReConNN is utilized to the reconstruction of heat transfer process of a pin fin heat sink. It demonstrates that the proposed ReConNN model is proved to be a potential capability to reconstruct physical field for multidisciplinary, such as structural optimization.