Alexey Khoroshilov

Dmitry Stanishevskii, Nini Kamkia, Alexey Khoroshilov et al.

Large language models (LLMs) are increasingly being applied to financial analysis, reporting, investment decision support, risk management, compliance, and professional training. However, robust evaluation of their domain competence in finance remains incomplete. Widely used open benchmarks such as FinQA, ConvFinQA, and TAT-QA have played an important role in advancing financial question answering and numerical reasoning, but they focus primarily on question answering over financial reports and do not provide an explicit hierarchy of professional difficulty. Broader resources, including FinanceBench, PIXIU, FinBen, and FLaME, expand the coverage of financial tasks, yet the problem of evaluating the transition from foundational knowledge to expert-level financial reasoning remains open. In this work, we present FINESSE-Bench, a suite of eight specialized benchmarks comprising 3,993 questions for hierarchical evaluation of financial competencies in LLMs. FINESSE-Bench combines exam-oriented datasets inspired by professional certifications (CFA-like Levels 1-3, CMT-like Level 2, and CFTe-like Level 1), applied trading task collections, and a Russian-language olympiad benchmark. This design enables evaluation of domain breadth, performance degradation as difficulty increases, the ability to solve computational tasks, and model behavior in specialized financial domains. We also describe a unified evaluation protocol covering multiple-choice questions, numerical answers, and short open-ended responses, together with an automated scoring scheme for freeform answers based on the LLM-as-judge paradigm. FINESSE-Bench is intended both as a complement to existing open financial benchmarks and as a tool for more substantive evaluation of professionally relevant financial competencies in large language models.

Alexey Khoroshilov, Alexey Chernysh, Orkhan Ekhtibarov et al.

Large language models have demonstrated strong performance on general-purpose programming tasks, yet their ability to generate executable algorithmic trading strategies remains underexplored. Unlike standard code benchmarks, trading-strategy generation requires simultaneous mastery of domain-specific financial logic, knowledge of a specialized API, and the ability to produce code that is not only syntactically correct but also leads to actual trades on historical data. In this work, we present QuantCode-Bench, a benchmark for the systematic evaluation of modern LLMs in generating strategies for the Backtrader framework from textual descriptions in English. The benchmark contains 400 tasks of varying difficulty collected from Reddit, TradingView, StackExchange, GitHub, and synthetic sources. Evaluation is conducted through a multi-stage pipeline that checks syntactic correctness, successful backtest execution, the presence of trades, and semantic alignment with the task description using an LLM judge. We compare state-of-the-art models in two settings: single-turn, where the strategy must be generated correctly on the first attempt, and agentic multi-turn, where the model receives iterative feedback and may repair its errors. We analyze the failure modes across different stages of the pipeline and show that the main limitations of current models are not related to syntax, but rather to the correct operationalization of trading logic, proper API usage, and adherence to task semantics. These findings suggest that trading strategy generation constitutes a distinct class of domain-specific code generation tasks in which success requires not only technical correctness, but also alignment between natural-language descriptions, financial logic, and the observable behavior of the strategy on data.

Maksim Raenchuk, Vitaly Cheptsov, Alexey Khoroshilov

In the development of modern cockpits, there is a trend towards the use of large displays that combine information about air navigation and the status of aircraft equipment. Flight and equipment performance information generated by multiple flight control systems should be graphically displayed in an easy-to-read form on widescreen multifunction displays. It is usually generated by independent systems whose output must not interfere with each other in accordance with the requirements of the ARINC 653 standard. This paper presents a solution to the problem of displaying ARINC 653 applications, which further improves security and portability, when running multiple applications on a single screen of one physical device.

Vitaly Cheptsov, Alexey Khoroshilov

Existing standards for airborne-embedded software systems impose a number of requirements applicable to the software development cycle of hard real-time operating systems found in modern aircraft. The measures taken are meant to reduce the risks of undesired consequences, but have strongly varying costs. Dynamic instrumentation and static analysis are common practices used to automatically find software defects, from strictly non-conforming code constructions to memory corruptions or invalid control flow. LLVM analyser and sanitizer infrastructure, while regularly applied to general-purpose software, originally was not thought to be introduced to heavily restricted environments. In this paper we discuss the specifics of airborne systems with regards to dynamic instrumentation and provide practical considerations to be taken into account for the effective use of general-purpose instrumentation tools. We bring a complete LLVM stack support to JetOS, a prospective onboard real-time operating system currently being developed at ISP RAS in collaboration with GosNIIAS. As an example, we port AddressSanitizer, MemorySanitizer, and UndefinedBehaviorSanitizer and provide the details against the caveats on all relevant sides: a sanitizer, a compiler, and an operating system. In addition we suggest uninvolved optimisations and enhancements to the runtimes to maximise the effects of the tools.

Denis Efremov, Mikhail Mandrykin, Alexey Khoroshilov

This paper presents results from the development and evaluation of a deductive verification benchmark consisting of 26 unmodified Linux kernel library functions implementing conventional memory and string operations. The formal contract of the functions was extracted from their source code and was represented in the form of preconditions and postconditions. The correctness of 23 functions was completely proved using AstraVer toolset, although success for 11 functions was achieved using 2 new specification language constructs. Another 2 functions were proved after a minor modification of their source code, while the final one cannot be completely proved using the existing memory model. The benchmark can be used for the testing and evaluation of deductive verification tools and as a starting point for verifying other parts of the Linux kernel.

Yevgeny Gerlits, Alexey Khoroshilov

The paper presents the experience of the authors in model based testing of safety critical real-time control logic software. It describes specifics of the corresponding industrial settings and discusses technical details of usage of UniTESK model based testing technology in these settings. Finally, we discuss possible future directions of safety critical software development processes and a place of model based testing techniques in it.