Generating Virtuoso test execution report using APIs

Article Overview:

This article will cover how you can achieve the following:

1. How to use Virtuoso’s API to generate execution reports as per your requirements.

2. How to generate a basic report with step details and screenshots using a JavaScript code.

Problem Statement:

Generating execution reports in Virtuoso can be a complex task, especially when trying to customize the report output according to specific project requirements. Users often face challenges in extracting detailed execution data, such as step-level outcomes, screenshots, and overall execution statistics, from the Virtuoso platform. Additionally, the need to generate user-friendly, easily shareable formats such as PDFs, which contain comprehensive details of the execution steps, further complicates the reporting process.

This article addresses these challenges by providing a solution that leverages Virtuoso’s API to generate customized execution reports using JavaScript code.

Solution:

The following API can be used to fetch the execution details of an execution job.

API details:

• Endpoint: https://api.virtuoso.qa/api/testsuites/execution
• Parameters: goalId, projectId, journeyId, snapshotId, jobId (same as executionId)

More information about this API endpoint can be found here.

Note: You can pass any of the above parameters when making the API call to generate the execution report. However, to get the report for a particular execution, you only need to pass the “jobId”.

eg: https://api.virtuoso.qa/api/testsuites/execution?jobId={{YOUR-JOB-ID}}

Here is a sample curl request for reference:

curl --location 'https://api.virtuoso.qa/api/testsuites/execution?jobId=12345' \

--header 'Authorization: Bearer <YOUR-API-TOKEN>'

API Response:

The API response JSON has multiple nesting levels. Below is a rough structure to understand the nesting before you can parse it:

• API response
  • item
    • journeys
      • journeyId (encoded uuid)
        • journey details
        • last change details
          • changelog, user who made the change, time of change, etc
        • last execution details
          • job
            • basic job details like environment, start time, user who launched the execution, etc
          • execution
            • execution data like start time, end time, total duration, etc
          • statistics
            • execution statistics like outcome and duration 
          • report
            • step-by-step details of execution, screenshots, side effects, network calls, etc
    • journey statistics
      • high-level statistics like outcome (pass/fail), execution duration, etc

Note: The “report” tag in the API response JSON holds the maximum information, as it contains step-level data like individual screenshots, step pass/fail data, step duration, side effects, and much more.

Using the API:

In order to extract the data from the report tag, make sure that you parse the response JSON correctly.

Attached below is a sample response JSON for your reference. <attach file sampleResponse.json>

Generate execution report pdf using the API:

You can use this API to create custom reports for your executions. Based on the most frequent customer queries, we have built a script that lets you generate an execution report using simple JavaScript. <attach file screenshotReport.js>

This script does the following:

1. Generates an execution report (PDF file) for an individual execution (provide jobId as input).
2. The output PDF file contains one page for each step in the journey.
3. Each page contains the following details for a single step:
   1. JourneyId
   2. Journey title
   3. Goal name
   4. StepId
   5. Step’s execution outcome (pass/fail)
   6. Checkpoint title
   7. Step’s execution duration
   8. Screenshot of the browser when that particular step is executed
   9. Error message (if the step failed)

How to use the script:

1. Install NodeJS on your local system.
2. Please make sure you have created a token from your profile in Virtuoso (keep the token handy it will be used in step 4)
3. Download the screenshotReport.js file (from the attachment section of this article) and store it in a local drive (E.g. C:/Temp/screenshotReport.js)
4. Open the screenshotReport.js file and update the token in line no 282 and update the jobId in line no 283
5. Optional step - if you are using an instance other than app.virtuoso.qa, please replace all the API endpoints in the screenshotReport.js file accordingly (e.g. api-us for app-us.virtuoso.qa instance and api-uk for app-uk.virtuoso.qa instance)
6. Open the cmd prompt in your machine
7. Navigate to the location C:/Temp on cmd
8. Install “axios” using the command: npm install axios
9. Install “pdf-lib” using the command: npm install pdf-lib
10. Install “date-fns” using the command: npm install date-fns
11. Execute the command - node screenshotReport.js to generate the pdf report
12. The output pdf report will be generated in C:\Temp