Testspace is used to measure and track metrics that help teams manage the quality of their software. Our goal is to provide actionable insights from test output produced during software development — Quality Analytics for the test automation workflow.

Filtering and Tracking Test Failures

We’ve discovered through our product development and from our customers that even with today’s automated toolchains, some aspects of managing software quality can still be very manual and time-consuming.

Today, we’re excited to announce new features that significantly reduce the time and effort when triaging and managing test failures in large result sets.

Why would I use them?

As software projects grow in size and complexity, the same occurs for test output. Development teams, including ours, manage the growing content using test suites and folders and by adding levels of hierarchy as needed.

When failures occur they can be widely dispersed and deeply nested.

Until now, triaging failures required manual click-navigation, with large result sets often time-consuming and even frustrating to manage. Failing cases in test suites spanning multiple screen views were difficult to locate when scrolling in a browser.

Once found, a failed binary result status was limited to the state of the current build — without historical context.

Although critically important, the binary status failed to answer other important questions.

Understanding and managing failures came with a cost that increased over time.

How do they work?

When viewing results, a single click on the new Failures button reduces a complex hierarchy of folders and suites into a single flat view showing only the suites that contain test failures.

When opening a test suite with failures, only the failing test cases are shown by default. A simple click removes the filter and renders all cases viewable. Another option in the failure view is to further filter based on new and/or frequency of failures. High frequency failures are cases that have failed at least three times in the last five results.

Managing Test Regressions

Test cases that fail are now tracked and monitored over subsequent result sets and are reported in one of three states

The counts of test cases in each of the above three states are rolled up and listed under a new Failures column at every level in the results hierarchy. Test cases in a Tracked state display a failure frequency counter, the rate of failure over the last five results. Each test case in a Tracked state supports a new historical list view of up to the five most recent results of passed, failed, N/A, or missing.

Read more

Get setup in minutes!

Try Testspace risk-free. No credit card is required.

Have questions? Contact us.