Duc Minh Le

Suyoun Kim, Akshat Shrivastava, Duc Le et al. · meta-ai

End-to-end (E2E) spoken language understanding (SLU) systems that generate a semantic parse from speech have become more promising recently. This approach uses a single model that utilizes audio and text representations from pre-trained speech recognition models (ASR), and outperforms traditional pipeline SLU systems in on-device streaming scenarios. However, E2E SLU systems still show weakness when text representation quality is low due to ASR transcription errors. To overcome this issue, we propose a novel E2E SLU system that enhances robustness to ASR errors by fusing audio and text representations based on the estimated modality confidence of ASR hypotheses. We introduce two novel techniques: 1) an effective method to encode the quality of ASR hypotheses and 2) an effective approach to integrate them into E2E SLU models. We show accuracy improvements on STOP dataset and share the analysis to demonstrate the effectiveness of our approach.

Duc Minh Le, Suhrid Karthik, Marcelo Schmitt Laser et al.

Architectural decay imposes real costs in terms of developer effort, system correctness, and performance. Over time, those problems are likely to be revealed as explicit implementation issues (defects, feature changes, etc.). Recent empirical studies have demonstrated that there is a significant correlation between architectural "smells" -- manifestations of architectural decay -- and implementation issues. In this paper, we take a step further in exploring this phenomenon. We analyze the available development data from 10 open-source software systems and show that information regarding current architectural decay in these systems can be used to build models that accurately predict future issue-proneness and change-proneness of the systems' implementations. As a less intuitive result, we also show that, in cases where historical data for a system is unavailable, such data from other, unrelated systems can provide reasonably accurate issue- and change-proneness prediction capabilities.

Trang Le, Daniel Lazar, Suyoun Kim et al.

Spoken Language Understanding (SLU) is a critical component of voice assistants; it consists of converting speech to semantic parses for task execution. Previous works have explored end-to-end models to improve the quality and robustness of SLU models with Deliberation, however these models have remained autoregressive, resulting in higher latencies. In this work we introduce PRoDeliberation, a novel method leveraging a Connectionist Temporal Classification-based decoding strategy as well as a denoising objective to train robust non-autoregressive deliberation models. We show that PRoDeliberation achieves the latency reduction of parallel decoding (2-10x improvement over autoregressive models) while retaining the ability to correct Automatic Speech Recognition (ASR) mistranscriptions of autoregressive deliberation systems. We further show that the design of the denoising training allows PRoDeliberation to overcome the limitations of small ASR devices, and we provide analysis on the necessity of each component of the system.

Arman Shahbazian, Youn Kyu Lee, Duc Le et al.

Over the past three decades, considerable effort has been devoted to the study of software architecture. A major portion of this effort has focused on the originally proposed view of four "C"s---components, connectors, configurations, and constraints---that are the building blocks of a system's architecture. Despite being simple and appealing, this view has proven to be incomplete and has required further elaboration. To that end, researchers have more recently tried to approach architectures from another important perspective---that of design decisions that yield a system's architecture. These more recent efforts have lacked a precise understanding of several key questions, however: (1) What is an architectural design decision (definition)? (2) How can architectural design decisions be found in existing systems (identification)? (3) What system decisions are and are not architectural (classification)? (4) How are architectural design decisions manifested in the code (reification)? (5) How can important architectural decisions be preserved and/or changed as desired (evolution)? This paper presents a technique targeted at answering these questions by analyzing information that is readily available about software systems. We applied our technique on over 100 different versions of two widely adopted open- source systems, and found that it can accurately uncover the architectural design decisions embodied in the systems.