Skip to main content
Version: 3.1

Test Run

Definition​

The Test Run contains the results of an executed Test Case, Test Suite or Test Configuration.

The results are stored in subsequent Test Suite runs, Test Case runs and Teststep runs.

Regardless of the Level of a Test Run, Test Suite runs, Test Case runs and Teststep runs are always created.

Properties​

Key​

The identifying number assigned by MTA that is used in the MTA Public API endpoints.

Application Instances​

The Application Instances that were used to execute the Test Run on.

Application Revisions​

The Application Revisions that were used to execute the Test Run on.

Start date​

The moment when the Test Run was started.

Full output​

Determines if the test was executed with "Full output" enabled. Recognizable by the icon.

Pinned​

If a Test Run is pinned by a user it will not be deleted overnight. Recognizable by the icon.

Result​

The test result ( 'Passed' or 'Failed').

Execution Scope​

SettingResult
AllThe Test Run was executed normally.
Failed onlyOnly Test Cases that had not Passed in the previous Test Run were executed.
Changed onlyOnly Test Cases (or Teststeps inside) that were changed since the previous Test Run were executed.
Changed & failedCombination of both options.

Note that Test Cases with Teststeps that provide data are always executed. See also Teststep dependencies.

When choosing Changed only:

  • only changes that were made in MTA are considered, not revision changes (in the Mendix model in Studio pro).
  • only changes that may affect the result of the Test Run are considered.
  • if a change is made in a Teststep or Test Case without Datavariation, the corresponding Test Case will be included.
  • if a change is made in a Teststep or Test Case with Datavariation that affects all variations, they will all be included.
  • if a change is made in a Teststep or Test Case with Datavariation that affects only a specific variation, only that specific variation will be included.

Level​

The level from which the test was executed, if not Test Configuration ('Test Suite' or 'Test Case')

Status​

This indicates if the test is running or not; can be 'Running' or 'Finished'.

Retention period​

Only applicable for Test Runs triggered through the Public API.
Sets the amount of days, after which the Test Run is permanently deleted.

Trace files​

A collection of Playwright Traces in the Test Suite Run.

Business rules​

Actions on Test Run​

View Test Runs​

  • Navigate to the Test configurations page.
  • Select the Test Configuration you want to view Test Runs for.
  • Navigate to Test runs.

View Test Suite Run​

A Test Suite run is an executed Test Suite in a Test Run.

  • View Test Runs.
  • Select the Test Run you want to view the results for.
  • Click Results on the right side of the page, for the executed Test Suite.

View Test Case Run​

A Test Suite run is an executed Test Case in a Test Suite Run.

  • View Test Suite Run.
  • Select the Test Case run on the left side of the page, to view the results for.

View Teststep Run​

A Teststep run is an executed Teststep in a Test Case Run.

  • View Test Case Run.
  • Select the Teststep run in the middle of the page, to view the results for.
note

For each Teststep run that has output, you can click on that output, to view in which teststep(s) it was used.
For each Teststep run that has input, you can click on the input, to view which teststep provided the input.

Create a new Test Run​

Delete a Test Run​

  • Navigate to the Test configurations page.
  • Select the Test Configuration you want to view Test Runs for.
  • Navigate to Test runs.
  • Click for the Test run you want to delete.

Cleanup of manually-executed Test Runs​

In a Test Configuration, only the last two executed Test Runs are persisted overnight, plus a maximum of two pinned Test Runs. The rest is cleaned up. Read the Pin a Test Run section in this page for more information.

Cleanup of API-triggered Test Runs​

For Test Runs that are executed through the Public API, instead of this, the Retention period applies. The date on which the Test Run will be deleted, is visible in the Test runs page.

Persisted dashboard data​

If a Test Run is cleaned up, the following information about the Test Run is stored for the statistics on the dashboard page:

  • start date
  • result (Pas, Fail, Error)
  • number of testcaseruns OK
  • number of testcaseruns NOK
  • number of testcaseruns not executed
  • per Test Case run: Test Case name, application, environment, revision, Test Run result

Pin a Test Run​

Pinning a Test Run will prevent it from being cleaned up overnight.

  • Navigate to the Test Run overview in MTA under Test Run.
  • Click on next to the Test Run you want to keep.

Feedback?​

Missing anything? Let us know!

Last updated 26 November 2025