We consider that at this point, you have already learned how to create simple load tests

Overview

Terms

Http user scenario - sequence of the related http requests provided by virtual user
Scenario step - single http requests in the scenario

Main features

Ability to provide Http user scenario tests with different load profiles
Global context per scenario execution to set endpoint, headers, execute all steps within single authentication session
Ability to change global context on the run
Ability to change next step in the scenario based on the results of the previous step
Step execution validation. Stop scenario execution on fail
Configurable metrics collection per step / scenario: response time, success rate, number of iterations
Support of the Basic authentication
Ability to execute every scenario exclusively. Only one virtual user is running scenario at a time
Ability to execute every scenario only single time

Http user scenario components

Http user scenario configuration components

Http user scenario load configuration

Execute http user load scenario: //begin: following section is used for docu generation - User scenario execution

@Configuration

public class UserScenarioJLoadScenarioProvider {

@Bean

public JLoadScenario exampleSimpleJaggerLoadScenarioUS() {

//begin: following section is used for docu generation - Load balancer setup

JTestDefinition jTestDefinition =

JTestDefinition.builder(Id.of("td_user_scenario_example"), new UserScenarioEndpointsProvider())

.withInvoker(new JHttpUserScenarioInvokerProvider())

// Exclusive access - single scenario is executed only by one virtual user at a time. No parallel execution

// Random seed - different virtual users execute scenarios in different order

.withLoadBalancer(JLoadBalancer.builder(ROUND_ROBIN)

.withExclusiveAccess()

.withRandomSeed(1234)

.build())

.addListener(JHttpUserScenarioInvocationListener.builder()

.withLatencyAvgStddevAggregators()

.withLatencyMinMaxAggregators()

.withLatencyPercentileAggregators(50D, 95D, 99D)

.build())

.build();

//end: following section is used for docu generation - Load balancer setup

JLoadProfile jLoadProfileInvocations =

JLoadProfileInvocation.builder(InvocationCount.of(100), ThreadCount.of(2))

.build();

JTerminationCriteria jTerminationCriteria =

JTerminationCriteriaIterations.of(IterationsNumber.of(500), MaxDurationInSeconds.of(50));

// We are setting acceptance criteria for particular metric of the selected step in the scenario

JLimit avgLatencyLimit =

JLimitVsRefValue.builder(SCENARIO_ID, STEP_1_ID, StandardMetricsNamesUtil.LATENCY_AVG_AGG_ID, RefValue.of(1.2))

.withOnlyErrors(LowErrThresh.of(0.25), UpErrThresh.of(2.0))

.build();

JLimit stdDevLatencyLimit =

JLimitVsRefValue.builder(SCENARIO_ID, STEP_1_ID, StandardMetricsNamesUtil.LATENCY_STD_DEV_AGG_ID, RefValue.of(0.5))

.withOnlyErrors(LowErrThresh.of(0.5), UpErrThresh.of(1.5))

.build();

JLimit maxLatencyLimit =

JLimitVsRefValue.builder(SCENARIO_ID, STEP_2_ID, StandardMetricsNamesUtil.LATENCY_MAX_AGG_ID, RefValue.of(2.0))

.withOnlyErrors(LowErrThresh.of(0.5), UpErrThresh.of(1.5))

.build();

JLimit minDevLatencyLimit =

JLimitVsRefValue.builder(SCENARIO_ID, STEP_2_ID, StandardMetricsNamesUtil.LATENCY_MIN_AGG_ID, RefValue.of(0.2))

.withOnlyErrors(LowErrThresh.of(0.5), UpErrThresh.of(1.5))

.build();

JLimit percentile99LatencyLimit =

JLimitVsRefValue.builder(SCENARIO_ID, STEP_2_ID, JMetricName.PERF_LATENCY_PERCENTILE(99D), RefValue.of(2.0))

.withOnlyErrors(LowErrThresh.of(0.5), UpErrThresh.of(1.5))

.build();

JLimit successRateLimit =

JLimitVsRefValue.builder(SCENARIO_ID, STEP_1_ID, JMetricName.PERF_SUCCESS_RATE_OK, RefValue.of(1.0))

.withOnlyErrors(LowErrThresh.of(0.99), UpErrThresh.of(1.01))

.build();

JLimit errorsLimit =

JLimitVsRefValue.builder(SCENARIO_ID, STEP_2_ID, JMetricName.PERF_SUCCESS_RATE_FAILS, RefValue.of(0.0))

.withOnlyErrors(LowErrThresh.of(0.99), UpErrThresh.of(1.01))

.build();

JLoadTest jLoadTest =

JLoadTest.builder(Id.of("lt_user_scenario_example"), jTestDefinition, jLoadProfileInvocations, jTerminationCriteria)

.withLimits(avgLatencyLimit, stdDevLatencyLimit, minDevLatencyLimit, maxLatencyLimit, percentile99LatencyLimit, successRateLimit, errorsLimit)

.build();

JParallelTestsGroup jParallelTestsGroup =

JParallelTestsGroup.builder(Id.of("ptg_user_scenario_example"), jLoadTest)

.build();

// To launch your load scenario, set 'jagger.load.scenario.id.to.execute' property's value equal to the load scenario id

// You can do it via system properties or in the 'environment.properties' file

return JLoadScenario.builder(Id.of("ls_user_scenario_example"), jParallelTestsGroup)

.build();

}

}

//end: following section is used for docu generation - User scenario execution

Provider example code is available in the archetype

User scenario load has the same configuration like simple examples from the previous chapter: Load scenario configuration
Important points in the example above:

Provider of the endpoints for the test definition returns list of User scenarios. User scenarios contains full description of the test: endpoints, queries, headers, etc. In such case we don't need query provider
We are using dedicated invoker to run user scenario test
We are using load balancer with exclusive setup. That means at any time, every scenario will be executed only by single virtual user. Other virtual users will execute different scenarios.
We are setting additional configurable user scenario invocation listener to collect metrics per step and scenario. Without this listener metrics will be not collected
There is an ability to set acceptance criteria per different metric in the steps of the scenario

Creating scenario

Example of the user load scenario creation: // begin: following section is used for docu generation - User scenario provider

public class UserScenarioEndpointsProvider implements Iterable {

public static final String SCENARIO_ID = "my-user-scenario";

public static final String STEP_1_ID = "step321";

public static final String STEP_2_ID = "step2";

public static final String STEP_3_ID = "step3";

private static Logger log = LoggerFactory.getLogger(UserScenarioEndpointsProvider.class);

private List<JHttpUserScenario> userScenarios = new ArrayList<>();

public UserScenarioEndpointsProvider() {

// First scenario example

JHttpUserScenario userScenario = new JHttpUserScenario(SCENARIO_ID, "My User Scenario");

userScenario

.withScenarioGlobalContext(new JHttpScenarioGlobalContext()

.withGlobalEndpoint(new JHttpEndpoint("https://httpbin.org/")))

.addStep(JHttpUserScenarioStep.builder(STEP_1_ID)

.withDisplayName("Step #321")

.withWaitAfterExecutionInSeconds(3)

.build())

.addStep(JHttpUserScenarioStep.builder(STEP_2_ID)

.withDisplayName("Step #2")

.withWaitAfterExecutionInSeconds(3)

.withQuery(new JHttpQuery().get().path("/get"))

// global context can be changed before or after step execution.

// E.g. we can add headers to all following requests in this scenario

.withPreProcessGlobalContextFunction((prevStep, context) -> {

context.withGlobalHeaders(prevStep.getResponse().getHeaders());

})

// you can decide if step was successful or not

.withPostProcessFunction(response -> {

if (response.getStatus().is2xxSuccessful()) {

log.info("Step 2 is successful!");

return true;

}

return false;

})

.build())

.addStep(JHttpUserScenarioStep.builder(STEP_3_ID)

.withDisplayName("Step #3")

.withWaitAfterExecutionInSeconds(3)

.withQuery(new JHttpQuery().get().path("/response-headers?key=val"))

// You can modify this step, based of the results of the previous one

// Here we are setting endpoint and query values

.withPreProcessFunction((prevStep, currentStep) -> {

currentStep.setEndpoint(new JHttpEndpoint("http://www.scala-lang.org"));

currentStep.setQuery(new JHttpQuery().get().path("/"));

})

.build());

// Scenario example with basic authentication

JHttpUserScenario userScenarioBasicAuthAuto = new JHttpUserScenario("my-user-scenario-basic-auth",

"Basic Auth User Scenario");

userScenarioBasicAuthAuto

// All requests in this scenario will use following basic authentication

.withScenarioGlobalContext(new JHttpScenarioGlobalContext()

.withGlobalEndpoint(new JHttpEndpoint("https://httpbin.org/"))

.withBasicAuth("userName", "userPassword")

)

.addStep(JHttpUserScenarioStep.builder("basic_auto_1")

.withQuery(new JHttpQuery().get().path("/basic-auth/userName/userPassword"))

.withDisplayName("Auth pass")

.withWaitAfterExecutionInSeconds(3)

.build())

.addStep(JHttpUserScenarioStep.builder("basic_auto_2")

.withQuery(new JHttpQuery().get().path("/basic-auth/userName/userPassword"))

.withDisplayName("Auth pass (validation)")

.withWaitAfterExecutionInSeconds(3)

.withPostProcessFunction(response -> {

if (response.getStatus().value() != 200) {

log.error("Unexpected status returned. Expected: 200. Returned: {}", response.getStatus().value());

return false;

}

return true;

})

.build())

.addStep(JHttpUserScenarioStep.builder("basic_auto_3")

.withQuery(new JHttpQuery().get().path("/basic-auth/userName/userPassword"))

.withDisplayName("Auth fail (validation)")

.withWaitAfterExecutionInSeconds(3)

.withPreProcessFunction((prevStep, currentStep) -> {

// Reset all headers => auth will fail

currentStep.getQuery().headers(null);

})

.withPostProcessFunction(response -> {

if (response.getStatus().value() != 401) {

log.error("Unexpected status returned. Expected: 401. Returned: {}", response.getStatus().value());

return false;

}

return true;

})

.build());

userScenarios.add(userScenario);

userScenarios.add(userScenarioBasicAuthAuto);

}

@Override

public Iterator<JHttpUserScenario> iterator() {

return userScenarios.iterator();

}

}

// end: following section is used for docu generation - User scenario provider

Provider example code is available in the archetype

Important points in the example above:

There is an option to setup global context per scenario. This context will be available for every step in the scenario, can be modified on the run. Context will be reset to default state at the beginning of the scenario execution
It is recommended to set delays after every step in the scenario. That will simulate real user behaviour
There is an option to validate response of the step and stop scenario execution if validation fails
There is an option to change next step in the scenario based on the results of the previous one
You can see an example of the Basic Authentication setup. User name and password from the global context will be used to generate basic authentication header for every step in the scenario