Category: Continuous Integration

This integration enables ALM Octane to run tests in your testing framework via your CI server.

This is useful in a number of scenarios. For example, suppose your CI server ran automated nightly tests and the ALM Octane pipeline shows that a specific test failed. After pushing a fix you can run the specific test directly from ALM Octane, rather than running the full pipeline which can be time-consuming.

In this article, the testrunner is configured for  MAVEN & JENKINS.

Overview

• Pre-requisites
• Configure a Jenkins job
• Create TestRunner in ALM Octane
• Run Tests from ALM Octane
• Conclusion

Pre-requisites

In order to configure the test runners with ALM Octane, make sure you have a testing framework in place with Maven, Gradle or Protractor and the automated test cases are being executed properly within the framework. 

1. You must have a CI server connected to ALM Octane. For more details, see Set up CI servers.
2. Within ALM Octane, create a pipeline to inject tests from your testing framework via your CI server. For more details, see Create and configure pipelines. 
3. Out-of-the-box, the integration supports the following frameworks via Jenkins, Bamboo, or Team City:
   1. JUnit and TestNG tests over Maven
   2. Protractor
   3. Gradle
4. Using the Jenkins plugin version 5.9 or later, you can trigger automated runs in other frameworks using the Custom option, as described below.

You must have a CI server connected to ALM Octane.

This example show a Jenkins pipeline with 80 automated tests. Lets assume, from 80 tests in a future job run 5 tests fails, and you want to run only the failed tests instead of the whole pipeline, then you would need test runners. 

Step 1 –  Create a job on your CI server to run tests in your testing framework

On your CI server, create a job that runs specific tests on your testing framework.

In addition to the command to run tests on your framework, this job must include the testsToRun parameter. The testsToRun parameter receives a lists of tests from ALM Octane that the job will then run. The parameter uses the following syntax: v1:package name|class name|test name;package name|class name|test name.

You will then need to convert the contents of this parameter to a syntax that can be used by your testing framework.

In addition to the testsToRun parameter, add the following to your job in Jenkins:

• Micro Focus ALM Octane testing framework converter build step/task.
• Publish JUnit test result report post-build action.

Create – On your CI server, create a job that runs specific tests on your testing framework.

Create a job of type “Freestyle Project” in Jenkins.

Parametrize – On the job add the testsToRun parameter

Add a build tasks in Jenkins

In addition to the testsToRun parameter, add the following to your job in Jenkins.

Micro Focus ALM Octane testing framework converter – this should be before running your Maven goal. It makes sense to previously check, if Maven is running the tests using mvn from the commandline.

Add an additional build task after the framework converter:

Invoke top-level Maven targets directly after the tasks Micro Focus ALM Octane testing framework converter. 

Add Post Build Task – Publish JUnit test result report post-build action.

Step 2: Create a test runner in ALM Octane

After the job has been configured on your CI Server, goto the workspace configuration under the DevOps section to add a new test runner.

• In ALM Octane, click Settings  > Spaces and select a workspace.
• Select the DevOps > Test Runners tab.
• Click + Test Runner.
• Define a name for the test runner, for example JUnit Runner.
• Select your testing framework, for example JUnit.
• Select your CI server.
• Select the job that you created in the CI server to run the tests in your testing framework. The list displays the jobs that have the testsToRun parameter.
• Click Save. ALM Octane creates a test runner to trigger the job that runs your tests.

If you build is parametrized, ALM Octane will show the default dataset with all the parameters and their default values. You can define additional parameter sets by adding (“+”) a new set. For example, this makes sense when you want to run same test on different environments with different users. 

Step 3: Run tests from ALM Octane

• Assign test runners to tests in the Quality module. You can then add the tests to test suites and run them from ALM Octane.
• ALM Octane triggers test runs via the job in the CI server that is specified in the test runner. After adding tests to a suite you can select a different test runner for a test in the suite. In addition, after planning a suite you can select a different test runner for a test run.
• A suite can include automated tests using multiple test runners. When a suite includes both manual tests and automated tests, the automated tests run in the background while you perform the manual tests.
• When you run a test, its status in ALM Octane is Planned until the test runner starts to run the tests in the CI server. The test’s status changes to In Progress, until the test results are returned to ALM Octane.

Identify failed tests from pipeline run

Once there are test failures during a pipeline run, perform your failure analysis and fix the problem and run only the failed tests using the defined test runners.

Create a new Test Suite in ALM Octane

You want to run after the fix only the 5 tests and not the comeplete pipeline. To do this, create a new test suite in ALM Octane for the failed tests. These tests can be executed by different testing tools and frameworks. The mapping between tests and testrunners happens in the test suite. 

Select the Test Runner for the Tests

For instance, if your pipeline had test execution from UFT One, UFT Developer, Selenium, SoapUI and JUnit Tests, then you could define test runners per testing tool or testing framework. You can decide which test would be part of the next run.

Run the Testuite

• Enter the name of this Run.
• Press „Lets run“ to run the suite.
• ALM Octane will start the test execution and the selected tests will be executed.

The test suite will run directly from ALM Octane.

You can track the execution state in the suite run under the RUNS tab. 

Track the Results ALM Octane

Test Runner will report the execution result to ALM Octane.

In case your tests are parametrized, you need to select the correct dataset when planning the suite run. 

Conclusion

Test runners are a great approach to execute test cases directly from ALM Octane. It allows users to drilldown on tests that matter to validate a fix instead to run the full pipeline including all automated tests. It also allows a flexible way to integrate test automation tools of your choice and implement the transition from manual testing to automated testing in an integrated way by linking manual tests to its corresponding automated tests directly in ALM Octane. 

This allows you to capture the transition from manual to automated testing on a timeline to better understand the automation coverage towards continuous testing.