Thomas Riedl

Jens Kohl, Luisa Gloger, Rui Costa et al.

As LLM-based applications reach millions of customers, ensuring their scalability and continuous quality improvement is critical for success. However, the current workflows for developing, maintaining, and operating (DevOps) these applications are predominantly manual, slow, and based on trial-and-error. With this paper we introduce the Generative AI Toolkit, which automates essential workflows over the whole life cycle of LLM-based applications. The toolkit helps to configure, test, continuously monitor and optimize Generative AI applications such as agents, thus significantly improving quality while shortening release cycles. We showcase the effectiveness of our toolkit on representative use cases, share best practices, and outline future enhancements. Since we are convinced that our Generative AI Toolkit is helpful for other teams, we are open sourcing it on and hope that others will use, forward, adapt and improve

Amir Houmansadr, Thomas Riedl, Nikita Borisov et al.

Open communication over the Internet poses a serious threat to countries with repressive regimes, leading them to develop and deploy network-based censorship mechanisms within their networks. Existing censorship circumvention systems face different difficulties in providing unobservable communication with their clients; this limits their availability and poses threats to their users. To provide the required unobservability, several recent circumvention systems suggest modifying Internet routers running outside the censored region to intercept and redirect packets to censored destinations. However, these approaches require modifications to ISP networks, and hence requires cooperation from ISP operators and/or network equipment vendors, presenting a substantial deployment challenge. In this report we propose a deployable and unobservable censorship-resistant infrastructure, called FreeWave. FreeWave works by modulating a client's Internet connections into acoustic signals that are carried over VoIP connections. Such VoIP connections are targeted to a server, FreeWave server, that extracts the tunneled traffic of clients and proxies them to the uncensored Internet. The use of actual VoIP connections, as opposed to traffic morphing, allows FreeWave to relay its VoIP connections through oblivious VoIP nodes, hence keeping itself unblockable from censors that perform IP address blocking. Also, the use of end-to-end encryption prevents censors from identifying FreeWave's VoIP connections using packet content filtering technologies, like deep-packet inspection. We prototype the designed FreeWave system over the popular VoIP system of Skype. We show that FreeWave is able to reliably achieve communication bandwidths that are sufficient for web browsing, even when clients are far distanced from the FreeWave server.