Martin Steinert

Kim Alexander Christensen, Andreas Gudahl Tufte, Alexey Gusev et al.

The draft IMO MASS Code requires autonomous and remotely supervised maritime vessels to detect departures from their operational design domain, enter a predefined fallback that notifies the operator, permit immediate human override, and avoid changing the voyage plan without approval. Meeting these obligations in the alert-to-takeover gap calls for a short-horizon, human-overridable fallback maneuver. Classical maritime autonomy stacks struggle when the correct action depends on meaning (e.g., diver-down flag means people in the water, fire close by means hazard). We argue (i) that vision-language models (VLMs) provide semantic awareness for such out-of-distribution situations, and (ii) that a fast-slow anomaly pipeline with a short-horizon, human-overridable fallback maneuver makes this practical in the handover window. We introduce Semantic Lookout, a camera-only, candidate-constrained VLM fallback maneuver selector that selects one cautious action (or station-keeping) from water-valid, world-anchored trajectories under continuous human authority. On 40 harbor scenes we measure per-call scene understanding and latency, alignment with human consensus (model majority-of-three voting), short-horizon risk-relief on fire hazard scenes, and an on-water alert->fallback maneuver->operator handover. Sub-10 s models retain most of the awareness of slower state-of-the-art models. The fallback maneuver selector outperforms geometry-only baselines and increases standoff distance on fire scenes. A field run verifies end-to-end operation. These results support VLMs as semantic fallback maneuver selectors compatible with the draft IMO MASS Code, within practical latency budgets, and motivate future work on domain-adapted, hybrid autonomy that pairs foundation-model semantics with multi-sensor bird's-eye-view perception and short-horizon replanning. Website: kimachristensen.github.io/bridge_policy

Jorgen F. Erichsen, Sampsa Kohtala, Martin Steinert et al.

While computer vision has received increasing attention in computer science over the last decade, there are few efforts in applying this to leverage engineering design research. Existing datasets and technologies allow researchers to capture and access more observations and video files, hence analysis is becoming a limiting factor. Therefore, this paper is investigating the application of machine learning, namely object detection methods to aid in the analysis of physical porotypes. With access to a large dataset of digitally captured physical prototypes from early-stage development projects (5950 images from 850 prototypes), the authors investigate applications that can be used for analysing this dataset. The authors retrained two pre-trained object detection models from two known frameworks, the TensorFlow Object Detection API and Darknet, using custom image sets of images of physical prototypes. As a result, a proof-of-concept of four trained models are presented; two models for detecting samples of wood-based sheet materials and two models for detecting samples containing microcontrollers. All models are evaluated using standard metrics for object detection model performance and the applicability of using object detection models in engineering design research is discussed. Results indicate that the models can successfully classify the type of material and type of pre-made component, respectively. However, more work is needed to fully integrate object detection models in the engineering design analysis workflow. The authors also extrapolate that the use of object detection for analysing images of physical prototypes will substantially reduce the effort required for analysing large datasets in engineering design research.

Jorgen F. Erichsen, Heikki Sjöman, Martin Steinert et al.

Aiming to help researchers capture output from the early stages of engineering design projects, this article presents a new research tool for digitally capturing physical prototypes. The motivation for this work is to collect observations that can aid in understanding prototyping in the early stages of engineering design projects, and this article investigates if and how digital capture of physical prototypes can be used for this purpose. Early-stage prototypes are usually rough and of low-fidelity and are thus often discarded or substantially modified through the projects. Hence, retrospective access to prototypes is a challenge when trying to gather accurate empirical data. To capture the prototypes developed through the early stages of a project, a new research tool has been developed for capturing prototypes through multi-view images, along with metadata describing by whom, why, when and where the prototypes were captured. Over the course of 17 months, this research tool has been used to capture more than 800 physical prototypes from 76 individual users across many projects. In this article, one project is shown in detail to demonstrate how this capturing system can gather empirical data for enriching engineering design project cases that focus on prototyping for concept generation. The authors also analyse the metadata provided by the system to give understanding into prototyping patterns in the projects. Lastly, through enabling digital capture of large quantities of data, the research tool presents the foundations for training artificial intelligence-based predictors and classifiers that can be used for analysis in engineering design research.

Achim Gerstenberg, Martin Steinert

Quantitatively evaluating and comparing the performance of robotic solutions that are designed to work under a variety of conditions is inherently challenging because they need to be evaluated under numerous precisely repeatable conditions Manually acquiring this data is time consuming and imprecise. A deterministic simulation can reproduce the conditions and can evaluate the solutions autonomously, faster and statistically significantly. We developed such a simulation designated to leverage data from a human-subject experiment post-experimentally. We present the development of the simulation and the verification that it actually reproduces the results obtained with the physical robot. The aim of this publication is to provide insight into the development details such that other researchers can replicate the setup and to show the degree of validity of the simulation.