Test execution control modes

Section describes different approaches to launch test execution

• Running in the Right away mode
• Running in the Stand by mode

Running in the Right away mode

We consider you are already familiar with different load generation configurations. In all configurations we are launching ./start.sh script to run test framework with one or another role. In all examples framework is running in the Right away mode. Start framework, execute tests, save results, terminate. That means during single framework launch you are able to execute single load scenario.

Such setup can be useful during debugging or local running of your performance tests. Besides Right away mode, Jagger framework is able to run in the Stand by mode

Running in the Stand by mode

What is the idea of the Stand by mode. In this mode load generating components will initialise on some environment and wait for the test project to execute. You can submit any new test project to the environment via REST API. Tests will be executed, results saved and load generating components will return to the Stand by mode, waiting for the next test project.

Such setup can be useful in the CI environments. It allows to run large number of tests without restarting framework and even without access to the environment where framework is running.

We are recommending to use Jenkins JaaS plugin to launch tests on the environments, running in the Stand by mode

Let's review Stand by mode working sequence in details with default load generating configuration

• Results storage and representation components are always running in the docker containers
• Start load generating components in single node or distributed mode on some environment. To run in Stand by mode you will need to set additional properties:

  ./start.sh profiles/basic/environment.properties
              -Drealtime.enable.standby.mode=true
              -Drealtime.jaas.endpoint=http://{jaas host}:8088/jaas
              -Drealtime.environment.id=my_test_environment

  Information about mentioned properties you can find in the Jagger configuration chapter
  
• After load generating components initialisation Master/Coordinator will start sending environment information to JaaS via Environment REST API . JaaS will return commands in the responses.
• Via Execution REST API you can submit new test project to selected environment. You will need to send URL to the .jar file with you test project, environment Id where it should be executed, id of the load scenario to execute, time to start the test
• JaaS will save your execution request to the execution queue. Execution will stay in the queue not more than time to start parameter. If during this time, there will be request from Master/Coordinator on the dedicated environment, JaaS will send a command to the environment to execute your test project. If there is no request from dedicated environment during time to start interval, execution will be marked as timeout and will be not submitted.
• Test project execution will start, results will be collected in the test results DB. Later you can access results via web interface or REST API.
• After test execution is over, environment will return to Stand By mode and will wait for the next command

Default configuration of the test framework