Aus ExpeccoWiki

Test Editor

Notice: starting with release 1.8, this editor has been split into separate editor and execution tabs. The following document describes the previous combined editor, which has the same functionality, but slightly different layout and menu organization.

The test editor (also called "Block-Test Editor" or "Block Functionality Test Editor") is used to define and execute a test or demo for the edited action. As such, it conbines features of the network editor and an execution trace/log viewer. This should not be used for "official" test-cases; instead, use it for the "private" execution of a single block (i.e. unit tests for the test-suite itself).

To the right, you see a test editor after the successful execution of a test-block.

Buttons

"Run" - Start the test execution. All autostarted steps will be launched.
"Debug Run" - Start the test execution in debug mode causing the debugger to open on exceptions.
"Step" - Start the test execution in single step mode or continue single stepping (when at a breakpoint).
"Pause" - Pause the current test run. To proceed or single step, click on the according button.
"Stop" - Stop the execution of the current test run. Exception/cleanup handling will be done.

"Pause on Error" - This toggle controls how the executor behaves when an error is encountered. The defaul is to stop the test execution and to mark the testcase as "FAILED" or "ERROR". However, especially during the test development process itself, it is often useful to be able to proceed after an error. For example, when the test consists of a web-browser's input field values being checked and validated, you may want to continue after a validation error. With this toggle set, the executor goes into the pause-state whenever an error is encountered, and execution can be continued by pressing the "Run" or "Single Step" button again (i.e. it behaves as if "Pause" was pressed).
"Hard Stop" - Hard-Terminate the execution of the current testrun, without doing any exception/cleanup handling. This is useful e.g. for recursion hang-ups, or if a block's cleanup action leads to another blocking situation.

Open the logging dropdown menu
Generate a report from the log data

Follow execution: autoselect the currently active block in the log (while running).
In the log, select the next activity which has finished with an error.
In the log, select the next activity which has finished with an inconclusive state.
In the log, select the next activity which is currently executing.

In the log, select the previously visited activity (i.e. back in visited-history).

The remaining edit-related buttons are described in the network editor.

Logging Menu

Logging

Save Log...
This option allows saving the current logging information into an expecco log file or a XML formatted file. Selecting it brings up a file chooser to determine the target path of the log file.

Load Log ...
This option allows loading previously saved logging information from an expecco log file or a XML formatted file. Selecting it brings up a file chooser to determine the source path of the log file.

Remove all Results in System
This option is used to clear all generated log data in the system. No reports can be generated from any more.

Remove Result
This option is used to clear the last generated log data. No reports can be generated from any more.

Views

Activity Tree

The activity tree displays a hierarchical view of the activities that are invoked during the test run, each with a single entry in the tree. If an activity spawns sub activities, these sub activities are enlisted in the branch of the spawning activity, and you can expand or collapse the branch with the node browsing item. Only the first level is expanded automatically when the test run is started. Activities that perform elementary actions never have sub activities, while those that perform compound actions always have at least one sub activity (else they fail due to lack of executable steps inside). Activities on the same level are sorted chronologically by invocation time.

The symbol in front of the activtiy indicates the state of the activity. This can be either in progress or one of the testresults "PASSED", "FAIL", "INCONCLUSIVE" or "ERROR".

Context Menu

A context menu is available in the activity tree view, relating to the Activity that is currently selected:

Find Next Error
This option switches the selection in the tree from the currently selected activity to the next, failed activity in order of the tree hierarchy.

Generate Report from here...
This option generates a report of the selected activity and all nested activities.

Open page on selected Item
This opens a new page on the block corresponding to the selected activity.

Code View

This view is shown for elementary activities . It display the block's code and highlights the executed code-line in case of an error. Environmental parameters (path and arguments) are shown in the pin view.

Network View

This view is shown for compound activities. It displays the activity diagram and highlights steps as they are executed. A step's color reflects its execution state:

blue
About to be executed (i.e. ready, but not yet started)

light green
being executed

green
finished successful

red
finished with error

grey
finished inconclusive (aborted or skipped)

Note that in this view, editing is not possible. You can, however, press enter after selecting a step, causing a new browser page to open up, showing the selected step's diagram.

Log View

This view shows the log created by the activity. The above buttons can be used to show or hide different message types. The checkbox "Sublogs" can be used to show or hide the logs of sub activities.

Pin Data View

This view is available for all types of activities. It shows two lists, the input pin value list to the left, and the output pin value list to the right. In each of the lists, three columns arrange the information. The first field of each row displays the pin name, the second one displays the value, and the third one displays the sender, or the receiver, respectively.

Note that input information is not changing after an activity was spawned, while output values can still be produced or changed during the execution of the activity. In each list, only one row can be selected at a time. According to the selected row, the above icon buttons can be used:

Jump to the sender of the value. Only available for input pins when a sender object is available. This causes the selection in the activity tree to change to the activity that sent the selected value to this activity.
Jump to the receiver of the value. Only available for output pins when a receiver object is available. This causes the selection in the activity tree to change to the activity that received the selected value from this activity.
Inspect the value object.

Tasks

Running a Trial Test

You invoke the test wrapper by clicking the start button in the editor's toolbar. Each step which is marked to auto start in the network will then spawn an activity. If logging is enabled, the lower monitoring panel will show the execution status. If logging is not enabled, the monitoring panel will be cleared and remain empty. Logging is initially enabled.

The monitoring panel is split vertically, showing the activity tree on the left, and the observation panel to the right, which displays the current state of the selected activity. Only one activity can be selected in the activity tree at any time. The depth of the branches is delimited by the log depth limit. See settings.

Single Stepping

Steps in an Activity Diagram

Either place a breakpoint on a step and proceed by pressing the "Single Step" button, or start the execution right from the start via the "Single Step" button. Finally, you can also place a "breakpoint"-action into the network (this allows for conditional breakpoints to be added, by conditionally enabling these steps). Once a breakpoint has been reached, you can continue single stepping (by pressing the "Step" button), cancel execution ("Stop" or "Hard Stop" buttons) or continue with normal execution ("Run" button).

Notice that there are now two different single step buttons; one is showing steps as executed in the currently visible network, the other ("Step In") also halts before any step in any subnetwork is about to be executed.

Code Lines in an Elementary Action

Place a line such as:

this.halt();

or

halt();

(for Smalltalk code, write <"self halt") somewhere in your elementary block's code. When executed, a debugger will be opened, highlighting the line containing the code-breakpoint. In the debugger, press "Next Line" or "Continue".

The debugger is described in detail in http://live.exept.de/doc/online/english/tools/debugger/TOP.html.

Premature Stop of a Test

Ocasionally, it might be required to stop a test even though it has not completed. For example, if you detect an error or malfunction which is not detected by the test, or the test is cought in an endless loop due to an unexpected situation. In this case, press the "Stop" button. This button's function is to ask the test-executor for a controlled termination of the run. Controlled means, that any cleanup actions (post-execution actions) are to be executed. This is the preferred way to stop test execution, because those cleanup actions are meant to free any aquired resources, remove temporary files or to turn off hardware equipment which was aquired and configured by the test. I.e. to bring the state of your equipment back to a well known (initial) state. However, sometimes, the cleanup actions themself might encounter trouble. For example, if your test requires a login procedure to a remote machine, and the cleanup tries to perform a clean logout, this may block if for example the connection is lost during the test. In this case, the cleanup action will not be able to perform its logout and may even block forever.

To handle this situation gracefully, expecco will wait for some preconfigured time for all the cleanup actions to complete (this is a configurable settings value found in the "extra"-"execution settings"-dialog). After that, a dialog window will appear, asking if the cleanup should be aported or if you want to continue waiting for the cleanup.

If you are certain that no cleanup action is required, or if you know in advance, that it will fail or block, use the "Hard Stop" button, which terminates execution without even attempting to perform any cleanup actions. Use this with care, as it may leave allocated resources and devices in a non-determined state.

For other editors see: Editors

The full online documentation can be found under: Online Documentation