Custom collectors

Collectors main page

There are three ways to collect some information from responses. One can create custom metric calculator,
validator or collector. Metric calculator and validator are simplified versions of collectors.
See comparison below. Order of execution is under link on the top of the page

  Param                                           Metric calculator               Validator                  Collector


- What information is coming from invoker         Response from SUT               Endpoint                   Endpoint
                                                                                  Query                      Query
                                                                                  Response from SUT          Response from SUT
                                                                                                             Invoke duration
                                                                                                             Was invoke fail or pass


- When is executed                                Invoke was successful           Invoke was successful      Always
                                                  All validators - successful     Previous validators pass


- What custom code is required                    Metric class                    Validator class            Collector class
                                                  Aggregator class [optional]                                Collector provider class
                                                                                                             Aggregator class [optional]


- Can save custom metric to DB                    Yes                             No                         Yes
- Can set invoke result to fails                  No                              Yes                        No

Custom validator Custom metric calculator Custom collector

1. Create class which implements ResponseValidator<Q,E,R>
   Will validate responce from SUT after every successful invocation

   /* begin: following section is used for docu generation - validator-custom source */
   /* Will compare result of invokation with expected result, read from file
    * @author Grid Dynamics
    *
    * @param <Q> - Query type
    * @param <E> - Endpoint type
    * @param <R> - Result type
    */
   public class ResponseFromFileValidator<Q, E, R> extends ResponseValidator<Q, E, R> {
   
       private String filePath= "suite/validator/resources/response.txt";
       private String expectedResponse;
   
       public ResponseFromFileValidator(String taskId, String sessionId, NodeContext kernelContext) {
           super(taskId, sessionId, kernelContext);
       }
   
       @Override
       public String getName() {
           return "ResponseFromFileValidator";
       }
   
       private void initiate(){
           StringBuilder sb = new StringBuilder();
           String fs = System.getProperty("line.separator");
           Scanner scanner=null;
           try {
               scanner= new Scanner(new FileInputStream(filePath));
               while (scanner.hasNextLine()){
                   sb.append(scanner.nextLine() + fs);
               }
           } catch (FileNotFoundException e) {
               throw new TechnicalException("Error during read file ", e);
           } finally{
               if(scanner!=null){
                   scanner.close();
               }
           }
           expectedResponse=sb.toString();
       }
   
       /* Following method will be called after every successful invoke to validate result
        * @author Grid Dynamics
        *
        * @param query    - Query that was sent to endpoint
        * @param endpoint - Endpoint - service under test
        * @param result   - Result returned from endpoint
        * @param duration - Duration of invoke
        * */
       @Override
       public boolean validate(Q query, E endpoint, R result, long duration) {
           if(expectedResponse==null){
               synchronized (filePath){
                   if(expectedResponse==null){
                       initiate();
                   }
               }
           }
           return expectedResponse.equals(result);
       }
   }
   /* end: following section is used for docu generation - validator-custom source */

   
   

2. If your validator doesn't have any properties, create validator-custom collector in info-collectors block in test-description.
   Set the name of validator class to attribute validator.

           <!-- begin: following section is used for docu generation - validator-custom -->
           <info-collectors parent="defaultCollectors">
               <validator xsi:type="validator-custom" validator="${package}.validator.ResponseFromFileValidator"/>
           </info-collectors>
           <!-- end: following section is used for docu generation - validator-custom -->

1. Create class which implements MetricCalculator<R>
   Will calculate some parameters according to SUT response

   /* begin: following section is used for docu generation - metric calculator source */
   /* Will calculate some parameter from endpoint response
    * @author Grid Dynamics */
   public class ResponseSize implements MetricCalculator<String> {
   
       /* Following method will be called after every successful invoke to evaluate result
        * In this simplified example we will just calculate length of result
        * @author Grid Dynamics
        *
        * @param response - Result returned from endpoint
        * */
       @Override
       public Integer calculate(String response) {
           return response.length();
       }
   }
   /* end: following section is used for docu generation - metric calculator source */

   
   

2. Create bean of this class in some configuration file. Put some id for it.

       <!-- begin: following section is used for docu generation - metric calculator -->
       <bean id="responseSize" class="${package}.calculator.ResponseSize"/>
       <!-- end: following section is used for docu generation - metric calculator -->

   
   

3. Add metric-custom collector to info-collectors block.
   Set id of custom metric class bean to calculator attribute
   Attribute id will be used as metric name in reports
   Set attribute plotData to true if you want to plot 'metric vs time' plot in report

           <!-- begin: following section is used for docu generation - metric calculator usage-->
           <info-collectors id="defaultCollectors">
               <validator xsi:type="validator-not-null-response"/>
               <validator xsi:type="validator-consistency"/>
               <metric id="responseSize"  xsi:type="metric-custom" calculator="responseSize" plotData="true"/>
               <metric xsi:type="metric-ref" ref="successRateCollector" plotData="true"/>
           </info-collectors>
           <!-- end: following section is used for docu generation - metric calculator usage-->

   
   

4. Optional. Add custom aggregator to metric.
   How to create aggregator provider - see 'Custom collector' example

   <beans:bean id="minAggregator" class="test.MinAggregatorProvider"/>
           ...
   <metric id="customMetric1" xsi:type="metric-custom" calculator="customMetric" plotData="true">
   <metric-aggregator xsi:type="metric-aggregator-ref" ref="minAggregator"/>
   </metric>

1. Create collector class which implements MetricCollector<Q,R,E>
   Will proceed data after every invoke and save to Kernel storage

   public class ExampleSuccessRateCollector<Q, R, E> extends MetricCollector<Q, R, E> {

   
   

2. Create collector provider class which implements MetricCollectorProvider<Q,R,E>
   Will provide an instance of custom collector to Jagger

   public class ProviderOfExampleSuccessRateCollector<Q, R, E> extends MetricCollectorProvider<Q, R, E> {

   
   

3. Create aggregator class which implements MetricAggregatorProvider
   Will proceed data after all tests are over and prepare data for DB
   Collector provider class is associating aggregator with collector.

   public class ProviderOfExampleSuccessRateAggregator implements MetricAggregatorProvider {

   
   

4. Create bean of provider class in some configuration file. Put some id for it.

       <!-- begin: following section is used for docu generation - custom collector -->
       <bean id="successRateCollector" class="${package}.provider.ProviderOfExampleSuccessRateCollector"/>
       <!-- end: following section is used for docu generation - custom collector -->

   
   

5. Add metric-ref collector to info-collectors block.
   Set id of bean to ref attribute.

           <!-- begin: following section is used for docu generation - metric calculator usage-->
           <info-collectors id="defaultCollectors">
               <validator xsi:type="validator-not-null-response"/>
               <validator xsi:type="validator-consistency"/>
               <metric id="responseSize"  xsi:type="metric-custom" calculator="responseSize" plotData="true"/>
               <metric xsi:type="metric-ref" ref="successRateCollector" plotData="true"/>
           </info-collectors>
           <!-- end: following section is used for docu generation - metric calculator usage-->

Note:

• full examples of the code are available in maven archetype-examples
• instead of ${package} write the name of your package