Jun Yang

Jiawei Liu, Jia Le Tian, Vijay Daita et al.

Recent advances have been improving the context windows of Large Language Models (LLMs). To quantify the real long-context capabilities of LLMs, evaluators such as the popular Needle in a Haystack have been developed to test LLMs over a large chunk of raw texts. While effective, current evaluations overlook the insight of how LLMs work with long-context code, i.e., repositories. To this end, we initiate the RepoQA benchmark to evaluate LLMs on long-context code understanding. Traditional needle testers ask LLMs to directly retrieve the answer from the context without necessary deep understanding. In RepoQA, we built our initial task, namely Searching Needle Function (SNF), which exercises LLMs to search functions given their natural-language description, i.e., LLMs cannot find the desired function if they cannot understand the description and code. RepoQA is multilingual and comprehensive: it includes 500 code search tasks gathered from 50 popular repositories across 5 modern programming languages. By evaluating 26 general and code-specific LLMs on RepoQA, we show (i) there is still a small gap between the best open and proprietary models; (ii) different models are good at different languages; and (iii) models may understand code better without comments.

Jun Yang, Nico Brailovsky

This paper proposes a dual-stage, low complexity, and reconfigurable technique to enhance the speech contaminated by various types of noise sources. Driven by input data and audio contents, the proposed dual-stage speech enhancement approach performs a coarse and fine processing in the first-stage and second-stage, respectively. In this paper, we demonstrate that the proposed speech enhancement solution significantly enhances the metrics of 3-fold QUality Evaluation of Speech in Telecommunication (3QUEST) consisting of speech mean-opinion-score (SMOS) and noise MOS (NMOS) for near-field and far-field applications. Moreover, the proposed speech enhancement approach greatly improves both the signal-to-noise ratio (SNR) and subjective listening experience. For comparisons, the traditional speech enhancement methods reduce the SMOS although they increase NMOS and SNR. In addition, the proposed speech enhancement scheme can be easily adopted in both capture path and speech render path for speech communication and conferencing systems, and voice-trigger applications.

Jun Yang, Joshua Bingham

The paper proposes an efficient, robust, and reconfigurable technique to suppress various types of noises for any sampling rate. The theoretical analyses, subjective and objective test results show that the proposed noise suppression (NS) solution significantly enhances the speech transmission index (STI), speech intelligibility (SI), signal-to-noise ratio (SNR), and subjective listening experience. The STI and SI consists of 5 levels, i.e., bad, poor, fair, good, and excellent. The most common noisy condition is of SNR ranging from -5 to 8 dB. For the input SNR between -5 and 2.5 dB, the proposed NS improves the STI and SI from "fair" to "good". For the input SNR between 2.5 to 8 dB, the STI and SI are improved from "good" to "excellent" by the proposed NS. The proposed NS can be adopted in both capture and playback paths for voice over internet protocol, voice-trigger, and automatic speech recognition applications.

Zhe Zhang, Ming Wu, Xinyu Han et al.

Many sound source localization (SSL) algorithms based on circular microphone array (CMA), including uniform circular array (UCA) and uniform concentric circular array (UCCA), have been well developed and verified via computer simulations and offline processing. On the other hand, beamforming in the harmonic domain has been shown to be a very efficient tool for broadband sound source localization due to its frequency-invariant properties. In this paper, design, implementation, and performance of a real-time sound source localization system are discussed. Specifically, we analyze the effect of parameter settings and compare the performance between UCA and UCCA through real-time experiments in a real room. The proposed method shows significant improvement by using UCCA instead of UCA.

Xuecong Sun, Han Jia, Zhe Zhang et al.

Conventional approaches to sound localization and separation are based on microphone arrays in artificial systems. Inspired by the selective perception of human auditory system, we design a multi-source listening system which can separate simultaneous overlapping sounds and localize the sound sources in three-dimensional space, using only a single microphone with a metamaterial enclosure. The enclosure modifies the frequency response of the microphone in a direction-dependent way by giving each direction a signature. Thus, the information about the location and audio content of sound sources can be experimentally reconstructed from the modulated mixed signals using compressive sensing algorithm. Owing to the low computational complexity of the proposed reconstruction algorithm, the designed system can also be applied in source identification and tracking. The effectiveness of the system in multiple real scenarios has been proved through multiple random listening tests. The proposed metamaterial-based single-sensor listening system opens a new way of sound localization and separation, which can be applied to intelligent scene monitoring and robot audition.

Donghui Sun, Qing Zhang, Jun Yang

In this paper, we investigate the Chinese font synthesis problem and propose a Pyramid Embedded Generative Adversarial Network (PEGAN) to automatically generate Chinese character images. The PEGAN consists of one generator and one discriminator. The generator is built using one encoder-decoder structure with cascaded refinement connections and mirror skip connections. The cascaded refinement connections embed a multiscale pyramid of downsampled original input into the encoder feature maps of different layers, and multi-scale feature maps from the encoder are connected to the corresponding feature maps in the decoder to make the mirror skip connections. Through combining the generative adversarial loss, pixel-wise loss, category loss and perceptual loss, the generator and discriminator can be trained alternately to synthesize character images. In order to verify the effectiveness of our proposed PEGAN, we first build one evaluation set, in which the characters are selected according to their stroke number and frequency of use, and then use both qualitative and quantitative metrics to measure the performance of our model comparing with the baseline method. The experimental results demonstrate the effectiveness of our proposed model, it shows the potential to automatically extend small font banks into complete ones.