User actions during the load test

• Additional user actions during performance test execution
• Listener example
• Listeners
• Services
• Service usage example

Additional user actions during performance test execution

We consider that at this point, you have already learned how to write load test and know the structure of the load scenario
Execution of the load scenario can be described with following simplified sequence:

• Start the load scenario
  • Start the first parallel test group inside scenario
    • Start all load tests inside the test group
    • Provide SUT invocations (send requests to the SUT, work with the responses)
    • Stop all the tests
  • Stop the first test group
  • Start the second tests group
    • ...
  • Stop the second test group
• Stop the scenario

Jagger allows to execute custom code on the every step from the list above. This code can prepare test data, reconfigure the SUT, store additional custom metrics, provide additional logging, etc
The feature is available via listeners classes. User can implement his own listener (load scenario listener, test group listener, etc) and attach it to the particular element of the load scenario.

Listener example

Create custom test group listener

In this simple example test group listener will just provide additional logging during test group start and stop

/* begin: following section is used for docu generation - example of the test group listener */

public class ExampleTestGroupListener extends ServicesAware implements Provider<TestGroupListener> {
    
    private final static Logger log = LoggerFactory.getLogger(ExampleTestGroupListener.class);
    
    @Override
    protected void init() {
        super.init();
    }
    
    @Override
    public TestGroupListener provide() {
        return new TestGroupListener() {
            @Override
            public void onStart(TestGroupInfo infoStart) {
                log.info("Started {} test group execution", infoStart.getTestGroup().getTaskName());
            }
            
            @Override
            public void onStop(TestGroupInfo infoStop) {
                log.info("{} test group execution took {}ms", infoStop.getTestGroup().getTaskName(), infoStop.getDuration());
            }
        };
    }
}

/* end: following section is used for docu generation - example of the test group listener */

Attach test group listener to the parallel test group in your load scenario

Another example. Invocation listeners

You can attach as many listeners as necessary to particular step in the test flow. They will be executed in the same order like they are added
Below is an example of the test definition with two invocation listeners

Listeners

You can implement following listeners:

LoadScenarioListener - Load scenario listener. Is triggered on start and stop of the load scenario
TestGroupListener - Parallel test group listener. Is triggered on start and stop of the test group
TestListener - Load test listener. Is triggered on start, stop and during load test execution
InvocationListener - Invocation listener. Is triggered on invocation start, success, error and failure

Framework is providing additional information in the listeners:

LoadScenarioInfo - Load scenario listener. Is triggered on start and stop of the load scenario
TestGroupInfo - Parallel test group listener. Is triggered on start and stop of the test group
TestInfo - Load test listener. Is triggered on start, stop and during load test execution
InvocationInfo - Invocation listener. Is triggered on invocation start, success, error and failure

Java doc for listeners and examples

Listener implementations and examples

Services

In all listeners framework is providing access to the Jagger internals via services. Main goal of the services - simplify development of the custom code. With the help of services you can store custom metrics, access test results, store additional information about your test in the Jagger test results database

You can use following services:

DefaultSessionInfoService - Allows to add comments to the test results, mark results with tags
DefaultMetricService - Allows to crete and save custom metrics
DefaultDataService - Provides access to the test results, stored in the results DB

Java doc for services

Service implementations and examples

Services availability

All custom listeners have access to Jagger services. List of available services can be different for different types of listeners.
Please see table below to find what services can be used in different listeners.

        /* begin: following section is used for docu generation - listeners to services relation */

        /* Services available for test listener */
        if (environment.equals(JaggerPlace.TEST_LISTENER)){
            metricService       = new DefaultMetricService(sessionId, taskId, context);       /* Available */
            sessionInfoService  = new DefaultSessionInfoService(context);                     /* Available */
            dataService         = new DefaultDataService(context);                            /* Available */
        }
        /* Services available for test group listener */
        if (environment.equals(JaggerPlace.TEST_GROUP_LISTENER)){
            metricService       = new DefaultMetricService(sessionId, taskId, context);       /* Available */
            sessionInfoService  = new DefaultSessionInfoService(context);                     /* Available */
            dataService         = new DefaultDataService(context);                            /* Available */
        }
        /* Services available for test suite listener */
        if (environment.equals(JaggerPlace.LOAD_SCENARIO_LISTENER)){
            metricService       = new EmptyMetricService(JaggerPlace.LOAD_SCENARIO_LISTENER); /* NOT AVAILABLE */
            sessionInfoService  = new DefaultSessionInfoService(context);                     /* Available */
            dataService         = new DefaultDataService(context);                            /* Available */
        }
        /* Services available for decision maker listener */
        if (environment.equals(JaggerPlace.TEST_GROUP_DECISION_MAKER_LISTENER)){
            metricService       = new EmptyMetricService(JaggerPlace.TEST_GROUP_DECISION_MAKER_LISTENER); /* NOT AVAILABLE */
            sessionInfoService  = new DefaultSessionInfoService(context);                                 /* Available */
            dataService         = new DefaultDataService(context);                                        /* Available */
        }

        /* Services available for invocation listener */
        if (environment.equals(JaggerPlace.INVOCATION_LISTENER)){
            metricService       = new DefaultMetricService(sessionId, taskId, context);         /* Available */
            sessionInfoService  = new EmptySessionInfoService(JaggerPlace.INVOCATION_LISTENER); /* NOT AVAILABLE */
            dataService         = new EmptyDataService(JaggerPlace.INVOCATION_LISTENER);        /* NOT AVAILABLE */
        }

        /* end: following section is used for docu generation - listeners to services relation */

If you will try to use unavailable service in some listener - nothing will happen and warning will be logged

Service usage example

Example of metric service usage in the invocation listener

Example code creates custom metric and stores test results to the Jagger database after every successful request to SUT

/* begin: following section is used for docu generation - example of the invocation listener with metric service */
public class NotNullInvocationListener extends ServicesAware implements Provider<InvocationListener> {

    private final String metricName = "not-null-responses";

    @Override
    protected void init() {
        getMetricService().createMetric(new MetricDescription(metricName).
                                            displayName("Number of not null responses").
                                            showSummary(true).
                                            plotData(false).
                                            addAggregator(new SumMetricAggregatorProvider()));
    }

    @Override
    public InvocationListener provide() {
        return new InvocationListener() {
            @Override
            public void onStart(InvocationInfo invocationInfo) {
            }

            @Override
            public void onSuccess(InvocationInfo invocationInfo) {
                if (invocationInfo.getResult() != null){
                    getMetricService().saveValue(metricName, 1);
                }
            }

            @Override
            public void onFail(InvocationInfo invocationInfo, InvocationException e) {
            }

            @Override
            public void onError(InvocationInfo invocationInfo, Throwable error) {
            }
        };
    }
}
/* end: following section is used for docu generation - example of the invocation listener with metric service */