BOSMI: A framework for non-intrusive monitoring and testing of embedded multithreaded software on the logical level

Traditional software testing methods are inefficient in cases where data inputs alone do not determine the outcome of a program's execution. In order to verify such software, testing is often complemented by analysis of the execution trace. For monitoring the execution trace, most approaches today insert additional instructions at the binary level, making the monitoring intrusive. Binary instrumentation operate on a low level, making it difficult to properly modify a program's states and to quantify its code coverage. In this paper, we present a framework for testing complex embedded multithreaded software on the logical level. Testing software on this level avoids dependency on concrete compilers and relates the execution to the source code, thus enabling coverage. Our non-intrusive execution monitoring and control is implemented using the LLVM interpreter compiler infrastructure. Instead of forcing thread interleaving, we suggest simulating interleaving effects through non-intrusive changes of shared variables. This makes it possible to test a single thread without executing the full software stack, which is especially useful in situations where the full software stack is not available (e.g., pre-integration testing). We complement existing approaches with new features such as dynamic configuration of monitoring and execution rollback to the checkpoints. Our approach introduces acceptable overhead without any complex setup