Integration of Virtuoso with Azure DevOps Test Management using Robot Framework

Article Overview:

This article will cover how you can achieve the following:

How to guide users through the steps required to integrate Virtuoso with Azure DevOps Test Management, utilizing the Robot Framework.

Problem Statement:

To extract the transaction number from a message containing the phrase "transaction number" followed by the actual number.

Solution:

Step 1: Import the GitHub Repository

The first step involves importing the required repository into your Azure DevOps project. You can find the repository here: Robot Framework Virtuoso CI/CD Azure Example.

Instructions:

1. Create a new Git repository in your Azure DevOps project.
   

2. Clone the contents from the root folder of the repository to your newly created git repository.
   
3. Copy the azure-pipelines.yml file from the repository and place it in the root of the folder in Azure DevOps.
   
   
   
   
   

Step 2: Set Up Environment Variables

You need to configure specific environment variables to enable the correct execution of your pipeline. There are two categories of environment variables: those from Azure DevOps and those from Virtuoso.

Azure DevOps Environment Variables:

• ORGANIZATION_URL: The URL for your Azure DevOps organization (e.g., https://dev.azure.com/virtuoso).
• PROJECT_NAME: The name of the project in Azure DevOps where your test plans are located.
• PERSONAL_ACCESS_TOKEN (PAT): The token used to authenticate and access Azure DevOps. This PAT should have read and write permissions to the Test Management feature.
• PLAN_ID: The ID of the test plan. You can find this ID by navigating to the plan in Azure DevOps and looking at the URL (e.g., planId=15).
• SUITE_ID: The ID of the test suite. Like the PLAN_ID, you can retrieve this by checking the URL (e.g., suiteId=16).
  
  

Virtuoso Environment Variables:

• VIRTUOSO_TOKEN: This token is required to access the Virtuoso API. Ensure the user generating the token has read and write permissions for the necessary projects. For instructions on generating this token, refer to the Virtuoso documentation.
  
  
  

Both the Azure Devops Environment Variables and Virtuoso Environment Variables are basically defined in the azure-pipelines.yml.

Note: These environment variables are crucial for ensuring proper test synchronzation between Virtuoso and Azure DevOps. By updating them, you enable correct execution of jobs in Virtuoso and ensure that results are accurately reflected in the Azure DevOps Test Plan used for synchronization.

Set Up Variables in Azure DevOps CI/CD Pipeline

To successfully execute the pipeline, set the required environment variables within your Azure CI/CD pipeline. You can also add secret variables for security-sensitive information like tokens and passwords.

For detailed steps on setting and securing variables, consult Microsoft’s Azure CI/CD documentation.


Step 3: Adjust the virtuoso.robot File

The next step involves making changes to the virtuoso.robot file located in the Test folder. The purpose of these adjustments is to ensure that the file works with the updated job execution logic.

Key Changes:

• Make sure all Project, Goals, Execution planner IDs are correctly referenced.
• Make sure all the conditions are correctly referenced.

Step 4: Creation of Azure DevOps Pipeline

After setting up the variables, the next step is to create the pipeline using the azure-pipelines.yml file. This file contains the pipeline structure and is crucial for linking Virtuoso, Robot Framework, and Azure DevOps.

Instructions:

1. Navigate to your Azure DevOps project.
2. Open the Pipelines section and create a new pipeline.

3. Choose the Git repository where you uploaded the files.
   
4. Set the pipeline configuration using the azure-pipelines.yml file.
   

5. Save and run the pipeline.

Upon triggering the pipeline, the execution process is initiated in Virtuoso. Once all Virtuoso jobs have successfully completed, the test results are synchronized and can be accessed under Test Plan → Runs in Azure DevOps, providing detailed execution reports and logs for analysis

Please refer to below screenshots-

1. Pipeline triggered , Stages - Run Virtuoso Test and Publish Test Results

2. Executed Plan with Condition as Pass at Run Virtuoso Stage

3. Virtuoso Project Activity showing the job initiated in Virtuoso Platform.

4. Published Test Result-

5. Test Result - Test Plan -> Runs

Note:

Additional Information: If the "Install Requirements" stage fails during pipeline execution, remove the line specifying install==1.3.5 from the requirements.txt file and rerun the pipeline. This may resolve dependency conflicts that could be causing the failure.