2.6CVApr 12, 2022
EVOPS Benchmark: Evaluation of Plane Segmentation from RGBD and LiDAR DataAnastasiia Kornilova, Dmitrii Iarosh, Denis Kukushkin et al.
This paper provides the EVOPS dataset for plane segmentation from 3D data, both from RGBD images and LiDAR point clouds. We have designed two annotation methodologies (RGBD and LiDAR) running on well-known and widely-used datasets for SLAM evaluation and we have provided a complete set of benchmarking tools including point, planes and segmentation metrics. The data includes a total number of 10k RGBD and 7K LiDAR frames over different selected scenes which consist of high quality segmented planes. The experiments report quality of SOTA methods for RGBD plane segmentation on our annotated data. We also have provided learnable baseline for plane segmentation in LiDAR point clouds. All labeled data and benchmark tools used have been made publicly available at https://evops.netlify.app/.
The Chronicles of RiDiC: Generating Datasets with Controlled Popularity Distribution for Long-form Factuality EvaluationPavel Braslavski, Dmitrii Iarosh, Nikita Sushko et al.
We present a configurable pipeline for generating multilingual sets of entities with specified characteristics, such as domain, geographical location and popularity, using data from Wikipedia and Wikidata. These datasets are intended for evaluating the factuality of LLMs' long-form generation, thereby complementing evaluation based on short-form QA datasets. We present the RiDiC dataset as an example of this approach. RiDiC contains 3,000 entities from three domains -- rivers, natural disasters, and car models -- spanning different popularity tiers. Each entity is accompanied by its geographical location, English and Chinese names (if available) and relevant English and Chinese Wikipedia content, which is used to evaluate LLMs' responses. Generations about RiDiC entities were obtained from three LLMs in English and Chinese. These were then evaluated using a third-party factuality checker, which showed that entities from our dataset caused even frontier models to hallucinate. To facilitate the evaluation of LLMs' long-form factuality in multiple languages, the code, data, and generation/evaluation scripts have been released.
2.4IVMay 24, 2021
Smart mobile microscopy: towards fully-automated digitizationA. Kornilova, I. Kirilenko, D. Iarosh et al.
Mobile microscopy is a newly formed field that emerged from a combination of optical microscopy capabilities and spread, functionality, and ever-increasing computing resources of mobile devices. Despite the idea of creating a system that would successfully merge a microscope, numerous computer vision methods, and a mobile device is regularly examined, the resulting implementations still require the presence of a qualified operator to control specimen digitization. In this paper, we address the task of surpassing this constraint and present a ``smart'' mobile microscope concept aimed at automatic digitization of the most valuable visual information about the specimen. We perform this through combining automated microscope setup control and classic techniques such as auto-focusing, in-focus filtering, and focus-stacking -- adapted and optimized as parts of a mobile cross-platform library.