An Testing Environment to Accumulate Test Cases for Execution Order Variation

Abstract

Many kind of Internet of Things (IoT) devices are recently developed. The IoT devices connect to an open platform and communicate each other with message exchange. Since the order of the message exchange is not deterministic, developers tests IoT devices in exhaustive order. The tests are usually conducted manually, which brings two issues: p1) testers possibly miss some of the order; p2) testers possibly overlook some evaluation items. As a solution, we developed a testing environment, which has a model checker that communicates with software on actual devices or an emulator. The testing environment solves p1) by automatic evaluation. The testing environment solves also p2) by automatic and simultaneous evaluation of evaluation items expressed as a Linear Temporal Logic (LTL) expression by integrating multiple LTL expressions with and-operator. We applied the test execution environment to three market defects and confirmed that the test execution environment tested the defects correctly.