Aus ExpeccoWiki

expecco can be executed automatically via the crontab (Unix) or scheduled tasks (Windows) mechanism of the operating system. Thus, you can have tests to execute unattended and without GUI "overnight".

Notice, that this is more of a "poor man's test-automation" - not the kind of automation and control that an expeccoNET automation environment would give you.

Command Line

The simplest command line to execute a test suite is:

expecco --execute <testsuite filename> ...

<testsuite filename> is the name as saved from within the expecco GUI application. This should be a saved ".ets" testSuite file. If more than one test suite filename is given, each is processed in sequence. All top-level test plans as found in the test suite(s) are executed (see below for more options).

Only the overall execution state is returned by the executions return value, which is

0 - all PASSED

non-zero - any ERROR, FAILURES or INCONCLUSIVE items.

The return value is useful if expecco is started via a makefile, for example.

Recommended Command Line Options

A typical command line is:

expecco --noBanner --noTray --verdicts --execute <testsuite filename>

or, if you want to use a special parameter-file:

expecco --noBanner --noTray --verdicts --parameter <param-file> --execute <testsuite filename>

All options are described below.

Additional Options

Information/Debugging/Logging

• --help
  Prints an up-to-date list of possible options.

• --verbose
  Additional startup, execution and result info is sent in human readable format to the stderr output. That may be too much of printout; try "--verdicts" instead.

• --noBanner
  Suppress the splash startup banner.

• --console
  Open a debug console window for Stdout and Stderr (1.5.3).

Test Selection

• --testSuiteName <namePattern>
  Only execute testSuites with a matching name (1.5.3).

• --testSuiteTag <tagPattern>
  Only execute testSuites with a matching tag (1.5.3).

• --testSuiteID <uuid>
  Only execute that particular testSuite (repeat to execute multiple) (1.5.3).

Reporting

• --verdicts
  Send execution and result info in human readable format to the stderr output. (see example below.)

• --logFile <fileName>
  Used to change the name of the generated log file, where stderr is written to.
  This is only useful if running in server mode.

• --exitCodeOnError <n>
  Changes the exit code in case any test ends with a test-error status. The default is <1>. If you run expecco out of make, jenkins or other tools which check for the exit status, you may want to change this to <0> (zero), in order for the calling program to not interpret this status as if expecco itself failed to execute.

• --exitCodeOnFail <n>
  Changes the exit code in case any test ends with a test-failure status. The default is <1>. If you run expecco out of make, jenkins or other tools which check for the exit status, you may want to change this to <0> (zero), in order for the calling program to not interpret this status as if expecco itself failed to execute.

• --csvReportFile <csvFilename>
  Writes a summary report in CSV format. Each row consists of:
  
  • TESTSUITE - the filename
  • TESTPLAN - the test plan
  • TESTCASE - the test case
  • VERDICT - the test result; one of SUCCESS, FAILED, ERROR or INCONCLUSIVE
  • INFO - additional info in case of failure

Additional pseudo verdicts such as STARTTIME, ENDTIME etc. are written. A typical CVS output file can be found in the examples.

• --xmlReportFile <xmlFilename>
  Writes a summary report in XML format. A typical XML output file can be found in the examples.

• --textReportFile <textFilename>
  Writes a summary report in a human readable textual format. A typical text output file can be found in the examples.

• --pdfReportFile <pdfFilename>
  Writes a summary report in pdf format.

• --elfReportFile <elfFilename>
  Writes a summary report in expecco logfile format

• --htmlReportFile <htmlFilename>
  Writes a summary report in html format.

• --junitReportFile <htmlFilename>
  Writes a summary report in a junit compatible XML format. Useful, when expecco is to be started by jenkins or similar driver programs.

Operation

• --server
  Starts expecco in server mode. In this mode, HTTP and SOAP requests can be used to remote-control expecco's execution; especially, to execute tests, and upload results. The most obvious use for this is for expecco to act as an execution host for an expeccoNET server.
  However, as standard protocols are used, this may also be used to interface the execution engine to any other QM management or automation system (HP Quality center).

• --port <portNr>
  Used to change the HTTP and SOAP port numbers, when running in server mode.
  The default is 9090 (server mode only).

• --info
  Provide an additional info service which allows for monitoring and control of an expecco server via a web browser (server mode only).

• --noTray
  Disable the tray control icon (server and execute modes only).

• --scripting <portNr>
  Enable scripting (remote control) on portNr (see more below).

• --allowHost <hostName>
  Allow hostName to connect to the scripting port. Repeat for multiple hosts (see more below) (1.5.3).

• --parameter <fileName>
  Load parameter set from filename. This is in XML format and can be generated with the "save-parameter-set" menu function.

(1.5.3) these options are only supported by rel 1.5.3 and higher (see Release Notes expecco )

Output Examples

The following output files were generated by executing the test suite from the file:

"C:\Programme\exept\expecco\testsuites\projects\Example.ets"

which contains a single test plan named:

"Example Testplan"

which contains three test cases named:

"Test1"

"Test2"

"Test3-Should fail"

of which the first two are supposed to PASS and the last one generates an ERROR.

All timestamp information is written in ISO8601 format.

CSV File Example

By default, entries are separated by ";"-characters. Additional double quotes (") are used to delimit strings with embedded ";" or spaces. Any double quote within such a string is doubled.

TESTSUITE;TESTPLAN;TESTCASE;VERDICT;INFO
;;;StartTime;2008-10-31T08:57:13.689
"C:\Programme\exept\expecco\testsuites\projects\Example.ets";"Example Testplan";Test1;PASSED;  
"C:\Programme\exept\expecco\testsuites\projects\Example.ets";"Example Testplan";Test2;PASSED;  
"C:\Programme\exept\expecco\testsuites\projects\Example.ets";"Example Testplan";"Test3 - Should fail";ERROR;   
;;;EndTime;2008-10-31T08:57:18.746

XML File Example

<SummaryReport>
  <StartTime>2008-10-31T08:56:49.774</StartTime> 
  <TestSuite>
    <File>C:\Programme\exept\expecco\testsuites\projects\Example.ets</File> 
    <TestPlan>
      <Name>Example Testplan</Name> 
      <TestCase>
        <Name>Test 1</Name> 
        <Verdict>PASSED</Verdict> 
      </TestCase>
      <TestCase>
        <Name>Test 2</Name> 
        <Verdict>PASSED</Verdict> 
      </TestCase>
      <TestCase>
        <Name>Test 3 - Should fail</Name> 
        <Verdict>ERROR</Verdict> 
      </TestCase>
    </TestPlan>
  </TestSuite>
  <EndTime>2008-10-31T08:56:55.142</EndTime> 
</SummaryReport>

Textual File Example

Test Execution Summary
  Start: 2008-10-31 08:56:12.751
  TestSuite: "C:\Programme\exept\expecco\testsuites\projects\Example.ets
    TestPlan: Example Testplan
      Test 1 ....................... PASSED
      Test 2 ....................... PASSED
      Test 3 - Should fail ......... ERROR
  Finish: 2008-10-31 08:56:18.009

Verdicts Example

the "--verdicts" option generates a simple trace on the standard error (to be redirected into a file):

C:\selftest\commandLineArgs> expecco --noBanner --noTray --verdicts 
--parameter paramesForTest.xpar --execute parameterTest.ets

Expecco [info]: start testSuite "parameterTest.ets"...
Expecco [info]: using parameters "paramesForTest.xpar"...
Expecco [info]: start testPlan "Test Parameters From CommandLine"
Expecco [info]: start testCase "Validate Parameters"
Expecco [info]: done testCase "Validate Parameters"; STATUS=PASSED
Expecco [info]: done testPlan "Test Parameters From CommandLine"; STATUS=PASSED
Expecco [info]: done testSuite "parameterTest.ets"; STATUS=PASSED

Scripting Example

Expecco can be scripted (remote controlled) via a telnet like command-line interface. This can be used for self-testing expecco, using another expecco to control its GUI or to look into its internal data structures. It can also be used to execute and automate more complex scenarios without using expeccoNET.

The scripting interface expects expressions in JavaScript syntax (but with a Smalltalk object model), similar to the language used for elementary block definition. Additional connection- and execution related commands are introduced by lines starting with the "~" (tilde) escape character (similar to telnet).
For example, "~." (tilde-period) is used to close the connection.
Press "~?" (tilde-question) for help.

The scripting service is enabled if a "--scripting" command line argument is given. By default, the service only accepts connections from the local host. Using the "--allowHost" option, more hosts can be gained access to the scripting feature (but be aware, that this opens a potential security hole).

The following is a trace of a typical scripting session:

c:\programs\expecco> expecco --scripting 8008
c:\programs\expecco>     ... expecco now running with GUI ...


c:\programs\expecco> telnet localhost 8008


Welcome to ST/X (Type ~? for help)
> print(expecco);
an Expecco::Browser
> expecco.menuNewProject();
...expecco performs the new-function from its menu...
> expecco.loadProjectFromFile("C:\Users\cg\testsuites\webedition\projects\BearerTest.ets");
...expecco loads the testSuite...
> expecco.close();


Lost Connection.
c:\programs\expecco>

Scripting API Overview

A few objects are predefined by name; among them are "expecco", which refers to the browser itself (i.e. the UI) and "environment", which refers to a collection of variable bindings.
New variables can defined by entering a "var <name>;" statement. For example:

    var ws;

    ws = new ExpeccoWorkspaceApplication;
    ws.open();
    ws.close();

expecco Object

The variable named "expecco" refers to the opened GUI-object, which represents the expecco application itself.

Menu Functions

These execute the underlying menu functions (as if clicked by the user).

• menuNewProject()
  creates a new testSuite
• menuOpen()
  opens a dialog requesting a testSuite
• menuExit()
  closes the expecco application

• loadProjectFromFile(fileNameString)
  load a testSuite
• saveProjectToFile(fileNameString)
  save the testSuite
• selectTestPlanWithName(testPlaneNameString)
  select a testplan item by name
• selectTestPlanWithFunctionalId(uuid)
  select a testplan item by uuid

• testSuite()
  retrieves the testSuite object

testSuite Object

• authorName()
  get the author name string
• authorName(aString)
  set the author name string
• versionString()
  get the version string
• versionString(aString)
  set the version string

• projectsWorkingDirectory()
  the directory, where attachments etc. are found

• ensureEnvironment()
  an instantiated environment (concrete values)

environment Object

[ ... to be continued ... ]

Back to Online Documentation