Starting expecco via Command Line
expecco can be executed automatically via the crontab (Unix) or scheduled tasks (Windows) mechanism of the operating system. Thus, you can have tests to execute unattended and without GUI "overnight".
Notice, that this is more of a "poor man's test-automation" - not the kind of automation and control that an expeccoNET automation environment would give you.
The simplest command line to execute a test suite is:
expecco --execute <testsuite filename> ...
- <testsuite filename> is the name as saved from within the expecco GUI application. This should be a saved ".ets" testSuite file. If more than one test suite filename is given, each is processed in sequence. All top-level test plans as found in the test suite(s) are executed (see below for more options).
- By default, the overall execution outcome is returned by the command's return value (exit code), which is
- 0 - all PASSED
- non-zero - any ERROR, FAILURES or INCONCLUSIVE items.
The return value is useful if expecco is started via a makefile, or from jenkins for example.
Command Line Options
Command line options are handled at two places: the runtime system (Smalltalk/X VM) and the expecco application itself. When expecco is started, the VM is first to look for command line arguments and processes any of the VM options, until a non-VM option is encountered. It passes the remaining options to expecco's main entry. Thus, any option which is given after the first non-VM option will not be handled by the VM, but by expecco. Therefore, no VM options should be placed after the first non-VM option. VM options are marked as "VM option" in the list below.
Also, some options are specific to the operating system, on which expecco is executed. These are marked as "Windows", "Linux" etc.
Recommended Command Line Options
A typical command line is:
expecco --noBanner --noTray --verdicts --execute <testsuite filename>
or, if you want to use a special parameter-file:
expecco --noBanner --noTray --verdicts --parameter <param-file> --execute <testsuite filename>
All options are described below.
Prints an up-to-date list of possible options.
Prints the expecco release identification and exits.
- --verbose (VM option)
Additional startup, execution and result info is sent in human readable format to the stderr output. That may be too much of printout; try "--verdicts" instead.
- --noBanner (VM option)
Suppress the splash startup banner.
- --console (VM option, Windows only)
Open a debug console window for Stdout and Stderr (1.5.3).
Turns on startup debugging. By default, if an error occurs during startup, expecco terminates itself. With this option enabled, a command line debugger is opened instead. This is probably only useful for plugin developers.
Test Selection / Test Execution
- --testSuiteName <namePattern>
Only execute testSuites with a matching name (1.5.3).
- --testSuiteTag <tagPattern>
Only execute testSuites with a matching tag (1.5.3).
- --testSuiteID <uuid>
Only execute that particular testSuite (repeat to execute multiple) (1.5.3).
- --parameter <fileName>
Load parameter set from filename. A parameter set contains values for a test suite's top environment. This is in XML format and can be generated with the "save-parameter-set" menu function, or provided by a QM system (such as expeccoNET, HP Quality Center or similar), or be generated by another controlling program.
- --scriptFile <fileName>
read and execute a script (see scripting below) from a file.
Send execution and result info in human readable format to the stderr output. (see example below.)
- --logFile <fileName>
Used to change the name of the generated log file, where stderr is written to.
This is only useful if running in server mode.
- --exitCodeOnError <n>
Changes the exit code in case any test ends with a test-error status. The default is <1>. If you run expecco out of make, jenkins or other tools which check for the exit status, you may want to change this to <0> (zero), in order for the calling program to not interpret this status as if expecco itself failed to execute.
- --exitCodeOnFail <n>
Changes the exit code in case any test ends with a test-failure status. The default is <1>. If you run expecco out of make, jenkins or other tools which check for the exit status, you may want to change this to <0> (zero), in order for the calling program to not interpret this status as if expecco itself failed to execute.
- --csvReportFile <csvFilename>
Writes a summary report in CSV format. Each row consists of:
- TESTSUITE - the filename
- TESTPLAN - the test plan
- TESTCASE - the test case
- VERDICT - the test result; one of SUCCESS, FAILED, ERROR or INCONCLUSIVE
- INFO - additional info in case of failure
- Additional pseudo verdicts such as STARTTIME, ENDTIME etc. are written. A typical CVS output file can be found in the examples.
- --xmlReportFile <xmlFilename>
Writes a summary report in XML format. A typical XML output file can be found in the examples.
- --textReportFile <textFilename>
Writes a summary report in a human readable textual format. A typical text output file can be found in the examples.
- --pdfReportFile <pdfFilename>
Writes a summary report in pdf format.
- --elfReportFile <elfFilename>
Writes a summary report in expecco logfile format
- --htmlReportFile <htmlFilename>
Writes a summary report in html format.
- --junitReportFile <htmlFilename>
Writes a summary report in a junit compatible XML format. Useful, when expecco is to be started by jenkins or similar driver programs.
Disable the tray control icon (server and execute modes only).
Do not open the main window (useful when scripting).
Disable plugin loading. This makes startup slightly faster. You can still force individual plugins to be loaded using the "--loadPlugin" option.
- --loadPlugin <pluginName>
Force loading a specific plugin. This is only useful if a "noPlugins" option was given before. pluginName is the name of the folder in the "plugin" folder (under the expecco installation folder).
If configured by the user's settings, expecco automatically checks for updates and patches when started (by checking the exept web-server for the presence of new patch files). This option disables that check, even if configured in the settings. Notice, that this is usually controlled by the user settings; however, in situations where settings are shared among multiple machines (network drive) and a test host has no internet access, it may be useful to disable this from the command line.
Starts expecco in server mode. In this mode, HTTP and SOAP requests can be used to remote-control expecco's execution; especially, to execute tests, and upload results. The most obvious use for this is for expecco to act as an execution host for an expeccoNET server.
However, as standard protocols are used, this may also be used to interface the execution engine to any other QM management or automation system (HP Quality center).
- --port <portNr>
Used to change the HTTP and SOAP port numbers, when running in server mode.
The default is 9090 (server mode only).
Provide an additional info service which allows for monitoring and control of an expecco server via a web browser (server mode only).
- --scripting <portNr>
Enable scripting (remote control) on portNr (see more below).
- --allowHost <hostName>
Allow hostName to connect to the scripting port. Repeat for multiple hosts (see more below) (1.5.3).
(1.5.3) these options are only supported by rel 1.5.3 and higher (see Release Notes expecco )
The following output files were generated by executing the test suite from the file:
which contains a single test plan named:
- "Example Testplan"
which contains three test cases named:
- "Test3-Should fail"
of which the first two are supposed to PASS and the last one generates an ERROR.
All timestamp information is written in ISO8601 format.
CSV File Example
By default, entries are separated by ";"-characters. Additional double quotes (") are used to delimit strings with embedded ";" or spaces. Any double quote within such a string is doubled.
TESTSUITE;TESTPLAN;TESTCASE;VERDICT;INFO ;;;StartTime;2008-10-31T08:57:13.689 "C:\Programme\exept\expecco\testsuites\projects\Example.ets";"Example Testplan";Test1;PASSED; "C:\Programme\exept\expecco\testsuites\projects\Example.ets";"Example Testplan";Test2;PASSED; "C:\Programme\exept\expecco\testsuites\projects\Example.ets";"Example Testplan";"Test3 - Should fail";ERROR; ;;;EndTime;2008-10-31T08:57:18.746
XML File Example
<SummaryReport> <StartTime>2008-10-31T08:56:49.774</StartTime> <TestSuite> <File>C:\Programme\exept\expecco\testsuites\projects\Example.ets</File> <TestPlan> <Name>Example Testplan</Name> <TestCase> <Name>Test 1</Name> <Verdict>PASSED</Verdict> </TestCase> <TestCase> <Name>Test 2</Name> <Verdict>PASSED</Verdict> </TestCase> <TestCase> <Name>Test 3 - Should fail</Name> <Verdict>ERROR</Verdict> </TestCase> </TestPlan> </TestSuite> <EndTime>2008-10-31T08:56:55.142</EndTime> </SummaryReport>
Textual File Example
Test Execution Summary Start: 2008-10-31 08:56:12.751 TestSuite: "C:\Programme\exept\expecco\testsuites\projects\Example.ets TestPlan: Example Testplan Test 1 ....................... PASSED Test 2 ....................... PASSED Test 3 - Should fail ......... ERROR Finish: 2008-10-31 08:56:18.009
the "--verdicts" option generates a simple trace on the standard error (to be redirected into a file):
C:\selftest\commandLineArgs> expecco --noBanner --noTray --verdicts --parameter paramesForTest.xpar --execute parameterTest.ets
Expecco [info]: start testSuite "parameterTest.ets"... Expecco [info]: using parameters "paramesForTest.xpar"... Expecco [info]: start testPlan "Test Parameters From CommandLine" Expecco [info]: start testCase "Validate Parameters" Expecco [info]: done testCase "Validate Parameters"; STATUS=PASSED Expecco [info]: done testPlan "Test Parameters From CommandLine"; STATUS=PASSED Expecco [info]: done testSuite "parameterTest.ets"; STATUS=PASSED
Expecco can be scripted either remote controlled via a telnet like command-line interface, or by reading a script from a file. This is useful for self-testing expecco, to use another expecco to control its GUI or to look into its internal data structures, or to execute complex test scenarios and meta tasks. It can also be used to execute and automate more complex scenarios without using expeccoNET.
When used with a remote script conenction, additional connection- and execution related commands are introduced by lines starting with the "~" (tilde) escape character (similar to telnet). For example, "~." (tilde-period) is used to close the connection. Press "~?" (tilde-question) for help.
The remote scripting service is enabled when a "--scripting" command line argument is given. By default, the service only accepts connections from the local host. Using the "--allowHost" option, more hosts can be gained access to the scripting feature - but be aware, that this opens a potential security hole, as all of the internal functions can be accessed via the scripting system.
The following is a trace of a typical scripting session:
c:\programs\expecco> expecco --scripting 8008 c:\programs\expecco> ... expecco now running with GUI ...
c:\programs\expecco> telnet localhost 8008
Welcome to ST/X (Type ~? for help) > print(expecco); an Expecco::Browser > expecco.menuNewProject(); ...expecco performs the new-function from its menu... > expecco.loadProjectFromFile("C:\Users\cg\testsuites\webedition\projects\BearerTest.ets"); ...expecco loads the testSuite... > expecco.close();
Lost Connection. c:\programs\expecco>
Scripting API Overview
A few objects are predefined by name; among them are "expecco", which refers to the browser itself (i.e. the UI) and "environment", which refers to a collection of variable bindings.
New variables can defined by entering a "var <name>;" statement. For example:
var ws; ws = new ExpeccoWorkspaceApplication; ws.open(); ws.close();
The variable named "expecco" refers to the opened GUI-object, which represents the expecco application itself.
These execute the underlying menu functions (as if clicked by the user).
creates a new testSuite
opens a dialog requesting a testSuite
closes the expecco application
load a testSuite
save the testSuite
select a testplan item by name
select a testplan item by uuid
retrieves the testSuite object
get the author name string
set the author name string
get the version string
set the version string
the directory, where attachments etc. are found
an instantiated environment (concrete values)
[ ... to be continued ... ]
Back to Online Documentation