Alejandro Sánchez Guinea

Thomas Kreutz, Max Mühlhäuser, Alejandro Sanchez Guinea

In this work, we address the problem of unsupervised moving object segmentation (MOS) in 4D LiDAR data recorded from a stationary sensor, where no ground truth annotations are involved. Deep learning-based state-of-the-art methods for LiDAR MOS strongly depend on annotated ground truth data, which is expensive to obtain and scarce in existence. To close this gap in the stationary setting, we propose a novel 4D LiDAR representation based on multivariate time series that relaxes the problem of unsupervised MOS to a time series clustering problem. More specifically, we propose modeling the change in occupancy of a voxel by a multivariate occupancy time series (MOTS), which captures spatio-temporal occupancy changes on the voxel level and its surrounding neighborhood. To perform unsupervised MOS, we train a neural network in a self-supervised manner to encode MOTS into voxel-level feature representations, which can be partitioned by a clustering algorithm into moving or stationary. Experiments on stationary scenes from the Raw KITTI dataset show that our fully unsupervised approach achieves performance that is comparable to that of supervised state-of-the-art approaches.

Thomas Kreutz, Ousama Esbel, Max Mühlhäuser et al.

The data collected from a vehicle's Controller Area Network (CAN) can quickly exceed human analysis or annotation capabilities when considering fleets of vehicles, which stresses the importance of unsupervised machine learning methods. This work presents a simultaneous clustering and segmentation approach for vehicle CAN-data that identifies common driving events in an unsupervised manner. The approach builds on self-supervised learning (SSL) for multivariate time series to distinguish different driving events in the learned latent space. We evaluate our approach with a dataset of real Tesla Model 3 vehicle CAN-data and a two-hour driving session that we annotated with different driving events. With our approach, we evaluate the applicability of recent time series-related contrastive and generative SSL techniques to learn representations that distinguish driving events. Compared to state-of-the-art (SOTA) generative SSL methods for driving event discovery, we find that contrastive learning approaches reach similar performance.

Thomas Kreutz, Jens Lemke, Max Mühlhäuser et al.

In this paper, we propose LiOn-XA, an unsupervised domain adaptation (UDA) approach that combines LiDAR-Only Cross-Modal (X) learning with Adversarial training for 3D LiDAR point cloud semantic segmentation to bridge the domain gap arising from environmental and sensor setup changes. Unlike existing works that exploit multiple data modalities like point clouds and RGB image data, we address UDA in scenarios where RGB images might not be available and show that two distinct LiDAR data representations can learn from each other for UDA. More specifically, we leverage 3D voxelized point clouds to preserve important geometric structure in combination with 2D projection-based range images that provide information such as object orientations or surfaces. To further align the feature space between both domains, we apply adversarial training using both features and predictions of both 2D and 3D neural networks. Our experiments on 3 real-to-real adaptation scenarios demonstrate the effectiveness of our approach, achieving new state-of-the-art performance when compared to previous uni- and multi-model UDA methods. Our source code is publicly available at https://github.com/JensLe97/lion-xa.

Fabian Fagerholm, Patrik Johnson, Alejandro Sánchez Guinea et al.

Nowadays, many software projects are partially or completely open-source based. There is an increasing need for companies to participate in open-source software (OSS) projects, e.g., in order to benefit from open source ecosystems. OSS projects introduce particular challenges that have to be understood in order to gain the benefits. One such challenge is getting newcomers onboard into the projects effectively. Similar challenges may be present in other self-organised, virtual team environments. In this paper we present preliminary observations and results of in-progress research that studies the process of onboarding into virtual OSS teams. The study is based on a program created and conceived at Stanford University in conjunction with Facebook's Education Modernization program. It involves the collaboration of more than a dozen international universities and nine open source projects. More than 120 students participated in 2013. The students have been introduced to and supported by mentors experienced in the participating OSS projects. Our findings indicate that mentoring is an important factor for effective onboarding in OSS projects, promoting cohesion within distributed teams and maintaining an appropriate pace.

Thomas Kreutz, Max Mühlhäuser, Alejandro Sanchez Guinea

Despite LiDAR (Light Detection and Ranging) being an effective privacy-preserving alternative to RGB cameras to perceive human activities, it remains largely underexplored in the context of multi-modal contrastive pre-training for human activity understanding (e.g., human activity recognition (HAR), retrieval, or person re-identification (RE-ID)). To close this gap, our work explores learning the correspondence between LiDAR point clouds, human skeleton poses, IMU data, and text in a joint embedding space. More specifically, we present DeSPITE, a Deep Skeleton-Pointcloud-IMU-Text Embedding model, which effectively learns a joint embedding space across these four modalities. At the heart of our empirical exploration, we have combined the existing LIPD and Babel datasets, which enabled us to synchronize data of all four modalities, allowing us to explore the learning of a new joint embedding space. Our experiments demonstrate novel human activity understanding tasks for point cloud sequences enabled through DeSPITE, including Skeleton<->Pointcloud<->IMU matching, retrieval, and temporal moment retrieval. Furthermore, we show that DeSPITE is an effective pre-training strategy for point cloud HAR through experiments in MSR-Action3D and HMPEAR.

Thomas Kreutz, Max Mühlhäuser, Alejandro Sanchez Guinea

Realistic crowd simulations are essential for immersive virtual environments, relying on both individual behaviors (microscopic dynamics) and overall crowd patterns (macroscopic characteristics). While recent data-driven methods like deep reinforcement learning improve microscopic realism, they often overlook critical macroscopic features such as crowd density and flow, which are governed by spatio-temporal spawn dynamics, namely, when and where agents enter a scene. Traditional methods, like random spawn rates, stochastic processes, or fixed schedules, are not guaranteed to capture the underlying complexity or lack diversity and realism. To address this issue, we propose a novel approach called nTPP-GMM that models spatio-temporal spawn dynamics using Neural Temporal Point Processes (nTPPs) that are coupled with a spawn-conditional Gaussian Mixture Model (GMM) for agent spawn and goal positions. We evaluate our approach by orchestrating crowd simulations of three diverse real-world datasets with nTPP-GMM. Our experiments demonstrate the orchestration with nTPP-GMM leads to realistic simulations that reflect real-world crowd scenarios and allow crowd analysis.

Aidmar Wainakh, Ephraim Zimmer, Sandeep Subedi et al.

Federated learning (FL) enables a set of entities to collaboratively train a machine learning model without sharing their sensitive data, thus, mitigating some privacy concerns. However, an increasing number of works in the literature propose attacks that can manipulate the model and disclose information about the training data in FL. As a result, there has been a growing belief in the research community that FL is highly vulnerable to a variety of severe attacks. Although these attacks do indeed highlight security and privacy risks in FL, some of them may not be as effective in production deployment because they are feasible only under special -- sometimes impractical -- assumptions. Furthermore, some attacks are evaluated under limited setups that may not match real-world scenarios. In this paper, we investigate this issue by conducting a systematic mapping study of attacks against FL, covering 48 relevant papers from 2016 to the third quarter of 2021. On the basis of this study, we provide a quantitative analysis of the proposed attacks and their evaluation settings. This analysis reveals several research gaps with regard to the type of target ML models and their architectures. Additionally, we highlight unrealistic assumptions in the problem settings of some attacks, related to the hyper-parameters of the ML model and data distribution among clients. Furthermore, we identify and discuss several fallacies in the evaluation of attacks, which open up questions on the generalizability of the conclusions. As a remedy, we propose a set of recommendations to avoid these fallacies and to promote adequate evaluations.

Aidmar Wainakh, Alejandro Sanchez Guinea, Tim Grube et al.

Federated learning suffers from several privacy-related issues that expose the participants to various threats. A number of these issues are aggravated by the centralized architecture of federated learning. In this paper, we discuss applying federated learning on a hierarchical architecture as a potential solution. We introduce the opportunities for more flexible decentralized control over the training process and its impact on the participants' privacy. Furthermore, we investigate possibilities to enhance the efficiency and effectiveness of defense and verification methods.

Alejandro Sanchez Guinea

This paper introduces abstractions that are meaningful for computers and that can be built and used according to computers' own criteria, i.e., computable abstractions. It is analyzed how abstractions can be seen to serve as the building blocks for the creation of one own's understanding of things, which is essential in performing intellectual tasks. Thus, abstractional machines are defined, which following a mechanical process can, based on computable abstractions, build and use their own understanding of things. Abstractional machines are illustrated through an example that outlines their application to the task of natural language processing.

Alejandro Sanchez Guinea

Software measurement programs have emerged as compounds of several measurement activities that are pursued as part of a combined effort of several parties within a software organization, based on interests that the organization has regarding the assessment of the different elements that intervene in the development of software. This paper recognizes design of measurement programs as an essential activity that, up until now, has been studied extensively, however, only in what respects to the content of the programs. In addition, proper specification for this kind of programs, accounting for preciseness and unambiguity, to facilitate maintenance, evolution, and execution has not been thoroughly considered. A methodology for designing programs that embody these and some other desirable features is presented. The methodology is built in solid ground. From software measurement literature, a goal-oriented approach is considered for building the content of the program. On the other hand, a successful technique from software development as modularization is utilized to give coherent structure to the measurement program.