Starting expecco via Command Line
expecco can be executed automatically via the crontab (Unix) or scheduled tasks (Windows) mechanism of the operating system. Thus, you can have tests to execute unattended and without GUI "overnight".
Notice, that this is more of a "poor man's test-automation" - not the kind of automation and control that an expeccoNET automation environment would give you.
The simplest command line to execute a test suite is:
expecco --execute <testsuite filename> ...
- <testsuite filename> is the name as saved from within the expecco GUI application. This should be a saved ".ets" testSuite file. If more than one test suite filename is given, each is processed in sequence. All top-level test plans as found in the test suite(s) are executed (see below for more options).
- Only the overall execution state is returned by the executions return value, which is
- 0 - all PASSED
- non-zero - any ERROR, FAILURES or INCONCLUSIVE items.
The return value is useful if expecco is started via a makefile, for example.
Recommended Command Line Options
A typical command line is:
expecco --noBanner --noTray --verdicts --execute <testsuite filename>
or, if you want to use a special parameter-file:
expecco --noBanner --noTray --verdicts --parameter <param-file> --execute <testsuite filename>
All options are described below.
Prints an up-to-date list of possible options.
Additional startup, execution and result info is sent in human readable format to the stderr output. That may be too much of printout; try "--verdicts" instead.
Suppress the splash startup banner.
Open a debug console window for Stdout and Stderr (1.5.3).
- --testSuiteName <namePattern>
Only execute testSuites with a matching name (1.5.3).
- --testSuiteTag <tagPattern>
Only execute testSuites with a matching tag (1.5.3).
- --testSuiteID <uuid>
Only execute that particular testSuite (repeat to execute multiple) (1.5.3).
Send execution and result info in human readable format to the stderr output. (see example below.)
- --logFile <fileName>
Used to change the name of the generated log file, where stderr is written to.
This is only useful if running in server mode.
- --exitCodeOnError <n>
Changes the exit code in case any test ends with a test-error status. The default is <1>. If you run expecco out of make, jenkins or other tools which check for the exit status, you may want to change this to <0> (zero), in order for the calling program to not interpret this status as if expecco itself failed to execute.
- --exitCodeOnFail <n>
Changes the exit code in case any test ends with a test-failure status. The default is <1>. If you run expecco out of make, jenkins or other tools which check for the exit status, you may want to change this to <0> (zero), in order for the calling program to not interpret this status as if expecco itself failed to execute.
- --csvReportFile <csvFilename>
Writes a summary report in CSV format. Each row consists of:
- TESTSUITE - the filename
- TESTPLAN - the test plan
- TESTCASE - the test case
- VERDICT - the test result; one of SUCCESS, FAILED, ERROR or INCONCLUSIVE
- INFO - additional info in case of failure
- Additional pseudo verdicts such as STARTTIME, ENDTIME etc. are written. A typical CVS output file can be found in the examples.
- --xmlReportFile <xmlFilename>
Writes a summary report in XML format. A typical XML output file can be found in the examples.
- --textReportFile <textFilename>
Writes a summary report in a human readable textual format. A typical text output file can be found in the examples.
- --pdfReportFile <pdfFilename>
Writes a summary report in pdf format.
- --elfReportFile <elfFilename>
Writes a summary report in expecco logfile format
- --htmlReportFile <htmlFilename>
Writes a summary report in html format.
- --junitReportFile <htmlFilename>
Writes a summary report in a junit compatible XML format. Useful, when expecco is to be started by jenkins or similar driver programs.
Starts expecco in server mode. In this mode, HTTP and SOAP requests can be used to remote-control expecco's execution; especially, to execute tests, and upload results. The most obvious use for this is for expecco to act as an execution host for an expeccoNET server.
However, as standard protocols are used, this may also be used to interface the execution engine to any other QM management or automation system (HP Quality center).
- --port <portNr>
Used to change the HTTP and SOAP port numbers, when running in server mode.
The default is 9090 (server mode only).
Provide an additional info service which allows for monitoring and control of an expecco server via a web browser (server mode only).
Disable the tray control icon (server and execute modes only).
- --scripting <portNr>
Enable scripting (remote control) on portNr (see more below).
- --allowHost <hostName>
Allow hostName to connect to the scripting port. Repeat for multiple hosts (see more below) (1.5.3).
- --parameter <fileName>
Load parameter set from filename. This is in XML format and can be generated with the "save-parameter-set" menu function.
(1.5.3) these options are only supported by rel 1.5.3 and higher (see Release Notes expecco )
The following output files were generated by executing the test suite from the file:
which contains a single test plan named:
- "Example Testplan"
which contains three test cases named:
- "Test3-Should fail"
of which the first two are supposed to PASS and the last one generates an ERROR.
All timestamp information is written in ISO8601 format.
CSV File Example
By default, entries are separated by ";"-characters. Additional double quotes (") are used to delimit strings with embedded ";" or spaces. Any double quote within such a string is doubled.
TESTSUITE;TESTPLAN;TESTCASE;VERDICT;INFO ;;;StartTime;2008-10-31T08:57:13.689 "C:\Programme\exept\expecco\testsuites\projects\Example.ets";"Example Testplan";Test1;PASSED; "C:\Programme\exept\expecco\testsuites\projects\Example.ets";"Example Testplan";Test2;PASSED; "C:\Programme\exept\expecco\testsuites\projects\Example.ets";"Example Testplan";"Test3 - Should fail";ERROR; ;;;EndTime;2008-10-31T08:57:18.746
XML File Example
<SummaryReport> <StartTime>2008-10-31T08:56:49.774</StartTime> <TestSuite> <File>C:\Programme\exept\expecco\testsuites\projects\Example.ets</File> <TestPlan> <Name>Example Testplan</Name> <TestCase> <Name>Test 1</Name> <Verdict>PASSED</Verdict> </TestCase> <TestCase> <Name>Test 2</Name> <Verdict>PASSED</Verdict> </TestCase> <TestCase> <Name>Test 3 - Should fail</Name> <Verdict>ERROR</Verdict> </TestCase> </TestPlan> </TestSuite> <EndTime>2008-10-31T08:56:55.142</EndTime> </SummaryReport>
Textual File Example
Test Execution Summary Start: 2008-10-31 08:56:12.751 TestSuite: "C:\Programme\exept\expecco\testsuites\projects\Example.ets TestPlan: Example Testplan Test 1 ....................... PASSED Test 2 ....................... PASSED Test 3 - Should fail ......... ERROR Finish: 2008-10-31 08:56:18.009
the "--verdicts" option generates a simple trace on the standard error (to be redirected into a file):
C:\selftest\commandLineArgs> expecco --noBanner --noTray --verdicts --parameter paramesForTest.xpar --execute parameterTest.ets
Expecco [info]: start testSuite "parameterTest.ets"... Expecco [info]: using parameters "paramesForTest.xpar"... Expecco [info]: start testPlan "Test Parameters From CommandLine" Expecco [info]: start testCase "Validate Parameters" Expecco [info]: done testCase "Validate Parameters"; STATUS=PASSED Expecco [info]: done testPlan "Test Parameters From CommandLine"; STATUS=PASSED Expecco [info]: done testSuite "parameterTest.ets"; STATUS=PASSED
Expecco can be scripted (remote controlled) via a telnet like command-line interface. This can be used for self-testing expecco, using another expecco to control its GUI or to look into its internal data structures. It can also be used to execute and automate more complex scenarios without using expeccoNET.
similar to the language used for elementary block definition. Additional connection- and execution
related commands are introduced by lines starting with the "~" (tilde) escape character (similar to telnet).
For example, "~." (tilde-period) is used to close the connection.
Press "~?" (tilde-question) for help.
The scripting service is enabled if a "--scripting" command line argument is given. By default, the service only accepts connections from the local host. Using the "--allowHost" option, more hosts can be gained access to the scripting feature (but be aware, that this opens a potential security hole).
The following is a trace of a typical scripting session:
c:\programs\expecco> expecco --scripting 8008 c:\programs\expecco> ... expecco now running with GUI ...
c:\programs\expecco> telnet localhost 8008
Welcome to ST/X (Type ~? for help) > print(expecco); an Expecco::Browser > expecco.menuNewProject(); ...expecco performs the new-function from its menu... > expecco.loadProjectFromFile("C:\Users\cg\testsuites\webedition\projects\BearerTest.ets"); ...expecco loads the testSuite... > expecco.close();
Lost Connection. c:\programs\expecco>
Scripting API Overview
A few objects are predefined by name; among them are "expecco", which refers to the browser itself (i.e. the UI) and "environment", which refers to a collection of variable bindings.
New variables can defined by entering a "var <name>;" statement. For example:
var ws; ws = new ExpeccoWorkspaceApplication; ws.open(); ws.close();
The variable named "expecco" refers to the opened GUI-object, which represents the expecco application itself.
These execute the underlying menu functions (as if clicked by the user).
creates a new testSuite
opens a dialog requesting a testSuite
closes the expecco application
load a testSuite
save the testSuite
select a testplan item by name
select a testplan item by uuid
retrieves the testSuite object
get the author name string
set the author name string
get the version string
set the version string
the directory, where attachments etc. are found
an instantiated environment (concrete values)
[ ... to be continued ... ]
Back to Online Documentation