Automated System-Level Software Testing of Industrial Networked Embedded Systems
It addresses software reliability issues in industrial and transport embedded systems, where failures can cause production loss or safety risks, but the approach appears incremental.
This thesis tackles five key challenges in automating software testing for industrial networked embedded systems, including test environment management, test case selection, hardware configuration, analysis of random failures, and actionable test results visualization, to improve reliability in critical domains.
Embedded systems are ubiquitous and play critical roles in management systems for industry and transport. Software failures in these domains may lead to loss of production or even loss of life, so the software in these systems needs to be reliable. Software testing is a standard approach for quality assurance of embedded software, and many software development processes strive for test automation. Out of the many challenges for successful software test automation, this thesis addresses five: (i) understanding how updated software reaches a test environment, how testing is conducted in the test environment, and how test results reach the developers that updated the software in the first place; (ii) selecting which test cases to execute in a test suite given constraints on available time and test systems; (iii) given that the test cases are run on different configurations of connected devices, selecting which hardware to use for each test case to be executed; (iv) analyzing test cases that, when executed over time on evolving software, testware or hardware revisions, appear to randomly fail; and (v) making test results information actionable with test results exploration and visualization. The challenges are tackled in several ways. [Abstract truncated.]