SE

main page

getting started

languages

dex

editors

publications

faq


SE

SE-Lab

RWTH Aachen

Fachgruppe Informatik

How-To: Testing MontiArc Components


back to MontiArc overview

MontiArc contains a language for input-output based black-box testing of MontiArc components. A generator takes I/O test models and generates JUnit tests that validate the modeled expected behavior. The following topics are part of this How-To:

Configure Component Tests

The I/O test generator tool is (yet) not integrated into the montiarc-maven-plugin. Hence, the common dsltool-maven-plugin has to be used. If your MontiArc project has been created by the New-Project-Wizard, this plugin is already configured. In the following, we expect that your project has been created by this wizard.
  • Test models are expected in directory src/test/models
  • JUnit tests are generated into directory target/generated-test-sources/montiarc
The following listing depicts how to integrate the test generator into a MontiArc maven build cycle. Simply add the pre-configured dsltool-maven-plugin into the build section of the project's pom.xml.
<build>
  <!-- ... -->
  <plugins>
    <!-- ... -->
    <!-- TEST CODE GENERATION -->
    <plugin>
      <groupId>de.monticore.maven</groupId>
      <artifactId>dsltool-maven-plugin</artifactId>
    </plugin>
  </plugins>
</build>

Basic I/O Testing

I/O test suites contain tests for a certain component (testee), are managed in packages, and have a name. As well as e.g. in Java the package has to match the folder structure and the name has to match the file name. The package path starts in the folder src/test/models. Hence, the following test suite has to be located in folder src/test/models/ma/ilc (l. 1) in the file InLightCtrlTest.mt (l. 4). The contained tests define input and expected output for component InLighCtrl (l. 4).
package ma.ilc;
import ma.ilc.InLightCtrl;

testsuite InLightCtrlTest for InLightCtrl { 
  // ...
}
A concrete test defines component input and expected output for the corresponding input. Input as well as output have to fit the testee's interface defined by its ports. The interface of component InLightCtrl is given in the following figure. It has three incoming ports named switchStatus, doorStatus, and alarmStatus and one outgoing port named lightCmd.

InLightCtrl port interface.


A test always has a name and contains an input block that assigns streams to incoming ports of the testee and an expect block that defines expected streams produced on the outgoing port of the testee. The following test noInputNoOutput (l. 2) defines empty streams (< Tk >) for each incoming port of testee InLightCtrl (ll. 4-6) and expects an empty stream as the resulting output of InLightCtrl's port lightCmd (l.9). A special value symbol is Tk that marks the end of a time unit. It may be used in all streams no matter what type the corresponding port has.
testsuite InLightCtrlTest for InLightCtrl { 
  test noInputNoOutput {
    input {
      switchStatus: < Tk >;
      doorStatus:   < Tk >;
      alarmStatus:  < Tk >;
    }
    expect {
      lightCmd:     < Tk >;
    }  
  }
}
Streams are usually not empty and contain values that are either fed into the testee or are expected to be emitted. Values are comma-separated and have to fit the type of the corresponding port. The following test delayedLightOff checks if the testee switches lightCmd from "true" to "false" two time units after the door has closed.
testsuite InLightCtrlTest for InLightCtrl { 
  test noInputNoOutput {
    // ...
  }
  test delayedLightOff {
    // time unit:     |    1     |     2    |     3    |    4     |
    input {
      switchStatus: < Tk, false, Tk, false, Tk, false, Tk, false, Tk >;
      doorStatus:   < Tk, true,  Tk, false, Tk, false, Tk, false, Tk >;
      alarmStatus:  < Tk, false, Tk, false, Tk, false, Tk, false, Tk >;
    }
    expect {
      lightCmd:     < Tk, true,  Tk, true,  Tk, true,  Tk, false, Tk >;
    }  
  }
}

Advanced I/O Testing

The previous section explained the basics of I/O testing. As it may be very time-consuming to define every single input and expected output, the I/O test language supports some extended concepts to ease the test case definition.

Special Values and Operators

Sometimes it might be important to assure that some messages do no occur or that the concrete value of a message is irrelevant but its arrival is important. Thus, following values and operators may be used in any stream contained in an expect block:
  • _: represents any value but a tick.
  • msg?: message msg is optional.
  • !msg: message msg must not occur.
testsuite InLightCtrlTest for InLightCtrl { 
  // ...
  test delayedLightOff2 {
    input {
      // ...
    }
    expect {
      lightCmd: < Tk, _, Tk, _, Tk, true?, Tk, !true, Tk >;
    }  
  }
}
Test delayedLightOff2 demonstrates this by means of a simple example. In time unit one and two, any message is accepted, in time unit three a "true" or no message is accepted while in time unit four anything but a "true" is accepted.

Variables

In order to avoid polluting test cases with complex value creation and ease stream reuse, fields may be defined within a test suite. The language offers variable fields, stream variables and stream matchers. The first are comparable to fields in Java and may be used within streams, stream variables and stream matchers. Stream variables assign a stream to a field that may be reused in various input/output definitions. They have to be precisely defined, e.g. must not contain ranges or optional values. Stream matcher are streams assignes to fields that may be exclusively used in the expected part of a test. In contrast to stream variables they may contain imprecise values like optional values or ranges of messages. The following listing explains this by adjusting the delayedLightOff test.
testsuite InLightCtrlTest for InLightCtrl { 
  Calendar c = new GregorianCalendar(1983, 3, 20);
  boolean on = true;
  boolean off = false;
  Stream<boolean>        allOff = < Tk, off, Tk, off, Tk, off, Tk, off, Tk >;
  StreamMatcher<boolean> result = < Tk, _,   Tk, _,   Tk, on,  Tk, off, Tk >;
  
  test delayedLightOff {
    input {
      switchStatus: allOff;
      doorStatus:   < Tk, on,  Tk, off, Tk, off, Tk, off, Tk >;
      alarmStatus:  allOff;
    }
    expect {
      lightCmd:     result;
    }  
  }
}
The variable c (l. 2) demonstrates how more complex objects may be created using a constructor call. The fields on and off (ll. 3f) are used to give the test a more intuitive semantics. An example of a stream variable definition named allOff is given in l. 5. This stream is used as input for the ports switchStatus (l. 9) and alarmStatus (l. 11).

Setup and Tear Down

It is not always possible to create complex objects with a single constructor call. Similar to JUnit, the test language offers setup and tear down mechanisms that are executed before or after the tests according to its configuration. Four different options are available for such setup blocks: @BeforeSuite, @AfterSuite, @Before, and @After. The first two are executed once before respectively after all tests, the last two are executed before respectively after each test. Optionally, local test setup and tear down blocks may be attached to a single test case. Pure Java is expected as content of these blocks.
testsuite InLightCtrlTest for InLightCtrl {
  var String[] s; 
  var Person p;
  
  @Before {
    s = new String[] {"some", "more", "complex", "stuff"};
    p = new Person();
    p.setName("Jeff");
    Dog d = new Dog("Lassie");
    p.setDog(d);
  }
  
  test someTest {
    @Before {
      p.setName("someTest");
    }
    //...
  }
  
  test someOtherTest {
    //...
  }
}
This example demonstrates how to set up variable s and p in a @Before block that is executed before each test of the test suite (ll. 5 - 11). Test someTest additionally has a local setup block (ll. 14-16) with further manipulations of field p. This way p has the name someTest in this test while being called Jeff in all other tests.

Repetitions

The test language also offers mechanisms to repeat messages and the execution of test cases. This might be useful if e.g. the testee is not deterministic regarding the timed behavior but if the expected results should occur in a certain time interval. The following repetitions are available:
  • x * msg: Repeat msg x times; x ∈ ℕ.
  • [lower, upper] * msg: Expect msg at lease lower and at max upper times; lower, upper ∈ ℕ. Only allowed in expect blocks.
  • test tc repeat x times {...}: Repeate test case tc x times; x ∈ ℕ.
This is demonstrated in the following example. Test delayedLightOff3 is repeated ten times (l. 2). Port switchStatus is fed with four "false" messages (l. 4) and we expect one to three times "true" on port lightCmd followed by a "false".
testsuite InLightCtrlTest for InLightCtrl { 
  test delayedLightOff3 repeat 10 times {
    input {
      switchStatus: < 4 * false >;
      doorStatus:   < true, 3 * false >;
      alarmStatus:  < 4 * false >;
    }
    expect {
      lightCmd:     < [1, 3] * true, false >;
    }  
  }
}

Value Groups

It is not possible to easily define repetitive complex message patterns using the presented language constructs. Comparing test case delayedLightOff3 with delayedLightOff, one can see that the message order is identical but the timed semantics may not be expressed with repetitions of single values. Thus, the I/O test language offers a mechanism to nest (repeated) values into value groups that may be also repeated themselves. As such a group is handled as a value itself, nesting of groups is also possible. This way, more complex input patterns as well as expected behavior patterns may be defined and tested. To create a group, a sequence of messages has to be surrounded by braces. The following listing depicts the refactored test delayedLightOff that uses value groups and repetitions instead of single messages.
testsuite InLightCtrlTest for InLightCtrl { 
  test delayedLightOff {
    input {
      switchStatus: < Tk, 4 * (false, Tk) >;
      doorStatus:   < Tk, true,  Tk, 3 * (false, Tk) >;
      switchStatus: < Tk, 4 * (false, Tk) >;
    }
    expect {
      lightCmd:     < Tk, 3 * (true, Tk), false, Tk >;
    }  
  }
}

Back to top.