4.3 Using listener interface

Robot Framework has a listener interface that can be used to receive notifications about test execution. Listeners are classes or modules with certain special methods, and they can be implemented both with Python and Java. Example uses of the listener interface include external test monitors, sending a mail message when a test fails, and communicating with other systems.

• 4.3.1 Taking listeners into use
• 4.3.2 Available listener interface methods
  • Listener interface versions
  • Listener interface method signatures
• 4.3.3 Listeners logging
• 4.3.4 Listener examples
• 4.3.5 Test libraries as listeners
  • Registering listener
  • Called listener methods

4.3.1 Taking listeners into use

Listeners are taken into use from the command line with the --listener option, so that the name of the listener is given to it as an argument. The listener name is got from the name of the class or module implementing the listener interface, similarly as test library names are got from classes implementing them. The specified listeners must be in the same module search path where test libraries are searched from when they are imported. Other option is to give an absolute or a relative path to the listener file similarly as with test libraries. It is possible to take multiple listeners into use by using this option several times:

pybot --listener MyListener tests.robot

jybot --listener com.company.package.Listener tests.robot

pybot --listener path/to/MyListener.py tests.robot

pybot --listener module.Listener --listener AnotherListener tests.robot

It is also possible to give arguments to listener classes from the command line. Arguments are specified after the listener name (or path) using a colon (:) as a separator. If a listener is given as an absolute Windows path, the colon after the drive letter is not considered a separator. Starting from Robot Framework 2.8.7, it is possible to use a semicolon (;) as an alternative argument separator. This is useful if listener arguments themselves contain colons, but requires surrounding the whole value with quotes on UNIX-like operating systems:

pybot --listener listener.py:arg1:arg2 tests.robot

pybot --listener "listener.py;arg:with:colons" tests.robot

pybot --listener C:\Path\Listener.py;D:\data;E:\extra tests.robot

4.3.2 Available listener interface methods

Robot Framework creates an instance of the listener class with given arguments when test execution starts. During the test execution, Robot Framework calls listeners' methods when test suites, test cases and keywords start and end. It also calls the appropriate methods when output files are ready, and finally at the end it calls the close method. A listener is not required to implement any official interface, and it only needs to have the methods it actually needs.

Listener interface versions

The signatures of methods related to test execution progress were changed in Robot Framework 2.1. This change was made so that new information can be added to listener interface without breaking existing listeners. The old signatures will continue to work, but they will be deprecated in some future version, so all new listeners should be implemented with signatures described in the table below. The most recent detailed description of the old listener interface can be found in User Guide of Robot Framework 2.0.4.

Note

A listener must have attribute ROBOT_LISTENER_API_VERSION defined in order to be recognized as a new style listener. Value of the ROBOT_LISTENER_API_VERSION attribute must be 2, either as a string or as an integer. The examples below are implemented as new style listeners.

Listener interface method signatures

All listener methods related to test execution progress have the same signature method(name, attributes), where attributes is a dictionary containing details of the event. The following table lists all the available methods in the listener interface and the contents of the attributes dictionary, where applicable. Keys of the dictionary are strings. All of these methods have also camelCase aliases. Thus, for example, startSuite is a synonym to start_suite.

The available methods and their arguments are also shown in a formal Java interface specification below. Contents of the java.util.Map attributes are as in the table above. It should be remembered that a listener does not need to implement any explicit interface or have all these methods.

public interface RobotListenerInterface {

public static final int ROBOT_LISTENER_API_VERSION = 2;

void startSuite(String name, java.util.Map attributes);

void endSuite(String name, java.util.Map attributes);

void startTest(String name, java.util.Map attributes);

void endTest(String name, java.util.Map attributes);

void startKeyword(String name, java.util.Map attributes);

void endKeyword(String name, java.util.Map attributes);

void logMessage(java.util.Map message);

void message(java.util.Map message);

void outputFile(String path);

void logFile(String path);

void reportFile(String path);

void debugFile(String path);

void close();

}

4.3.3 Listeners logging

Robot Framework 2.6 introduced new programmatic logging APIs that also listeners can utilize. There are some limitations, however, and how different listener methods can log messages is explained in the table below.

Note

To avoid recursion, messages logged by listeners are not sent to listener methods log_message and message.

Warning

There were severe problems with listeners logging prior to Robot Framework 2.6.2. Using this functionality with earlier versions is thus not recommended.

4.3.4 Listener examples

The first simple example is implemented in a Python module. It mainly illustrates that using the listener interface is not very complicated.

ROBOT_LISTENER_API_VERSION = 2

def start_test(name, attrs):

print 'Executing test %s' % name

def start_keyword(name, attrs):

print 'Executing keyword %s with arguments %s' % (name, attrs['args'])

def log_file(path):

print 'Test log available at %s' % path

def close():

print 'All tests executed'

The second example, which still uses Python, is slightly more complicated. It writes all the information it gets into a text file in a temporary directory without much formatting. The filename may be given from the command line, but also has a default value. Note that in real usage, the debug filefunctionality available through the command line option --debugfile is probably more useful than this example.

import os.path

import tempfile

class PythonListener:

ROBOT_LISTENER_API_VERSION = 2

def init(self, filename='listen.txt'):

outpath = os.path.join(tempfile.gettempdir(), filename)

self.outfile = open(outpath, 'w')

def start_suite(self, name, attrs):

self.outfile.write("%s '%s'\n" % (name, attrs['doc']))

def start_test(self, name, attrs):

tags = ' '.join(attrs['tags'])

self.outfile.write("- %s '%s' [ %s ] :: " % (name, attrs['doc'], tags))

def end_test(self, name, attrs):

if attrs['status'] == 'PASS':

self.outfile.write('PASS\n')

else:

self.outfile.write('FAIL: %s\n' % attrs['message'])

def end_suite(self, name, attrs):

self.outfile.write('%s\n%s\n' % (attrs['status'], attrs['message']))

def close(self):

self.outfile.close()

The third example implements the same functionality as the previous one, but uses Java instead of Python.

import java.io.*;

import java.util.Map;

import java.util.List;

public class JavaListener {

public static final int ROBOT_LISTENER_API_VERSION = 2;

public static final String DEFAULT_FILENAME = "listen_java.txt";

private BufferedWriter outfile = null;

public JavaListener() throws IOException {

this(DEFAULT_FILENAME);

}

public JavaListener(String filename) throws IOException {

String tmpdir = System.getProperty("java.io.tmpdir");

String sep = System.getProperty("file.separator");

String outpath = tmpdir + sep + filename;

outfile = new BufferedWriter(new FileWriter(outpath));

}

public void startSuite(String name, Map attrs) throws IOException {

outfile.write(name + " '" + attrs.get("doc") + "'\n");

}

public void startTest(String name, Map attrs) throws IOException {

outfile.write("- " + name + " '" + attrs.get("doc") + "' [ ");

List tags = (List)attrs.get("tags");

for (int i=0; i < tags.size(); i++) {

outfile.write(tags.get(i) + " ");

}

outfile.write(" ] :: ");

}

public void endTest(String name, Map attrs) throws IOException {

String status = attrs.get("status").toString();

if (status.equals("PASS")) {

outfile.write("PASS\n");

}

else {

outfile.write("FAIL: " + attrs.get("message") + "\n");

}

}

public void endSuite(String name, Map attrs) throws IOException {

outfile.write(attrs.get("status") + "\n" + attrs.get("message") + "\n");

}

public void close() throws IOException {

outfile.close();

}

}

4.3.5 Test libraries as listeners

Sometimes it is useful also for test libraries to get notifications about test execution. This allows them, for example, to perform certain clean-up activities automatically when a test suite or the whole test execution ends.

Note

This functionality is new in Robot Framework 2.8.5.

Registering listener

A test library can register a listener by using ROBOT_LIBRARY_LISTENER attribute. The value of this attribute should be an instance of the listener to use. It may be a totally independent listener or the library itself can act as a listener. To avoid listener methods to be exposed as keywords in the latter case, it is possible to prefix them with an underscore. For example, instead of using end_suite or endSuite, it is possible to use _end_suite or _endSuite.

Following examples illustrates using an external listener as well as library acting as a listener itself:

import my.project.Listener;

public class JavaLibraryWithExternalListener {

public static final Listener ROBOT_LIBRARY_LISTENER = new Listener();

public static final String ROBOT_LIBRARY_SCOPE = "GLOBAL";

// actual library code here ...

}

class PythonLibraryAsListenerItself(object):

ROBOT_LIBRARY_SCOPE = 'TEST SUITE'

ROBOT_LISTENER_API_VERSION = 2

def init(self):

self.ROBOT_LIBRARY_LISTENER = self

def _end_suite(self, name, attrs):

print 'Suite %s (%s) ending.' % (name, attrs['id'])

# actual library code here ...

As the seconds example above already demonstrated, library listeners can specify listener interface versions using ROBOT_LISTENER_API_VERSION attribute exactly like any other listener.

Called listener methods

Library's listener will get notifications about all events in suites where the library is imported. In practice this means that start_suite, end_suite, start_test, end_test, start_keyword, end_keyword, log_message, and message methods are called inside those suites.

If the library creates a new listener instance every time when the library itself is instantiated, the actual listener instance to use will change according to the test library scope. In addition to the previously listed listener methods, close method is called when the library goes out of the scope.

See Listener interface method signatures section above for more information about all these methods.