Historically, the focus of Test Management tools has been on Manual Testing and features to facilitate the managing of activities for a team of testers. With the popularity of Continuous Integration, Continuous Testing, and DevOps much of the quality assessment for a software project is now driven by automation.

Automated testing can leverage the GitHub platform and Continuous Integration tools to execute tests daily or even with every software change committed, resulting in a continuous stream of test results.

In contrast, manual testing is typically managed by a separate team using its own processes and toolsets. Specific builds are targeted for testing, producing a lower volume of test result content due to the constraints of time and test resources. Failures discovered require a much slower, more involved triage process often requiring defects to be opened. Even with these challenges, manual testing is still critical and a very important process for evaluating quality.

To optimize this process, we are enabling QA teams to leverage the same GitHub feature set as development teams.

Connecting Manual and Automation

Testspace has been designed specifically to connect the workflows of automated and manual testing as well as gather and process test results from both. This is realized by providing a test platform built for Manual testing and CI reporting:

  • Manual tests managed as code in the Repo
  • Auto-generated GitHub Project Boards representing Test Plans
  • Seamless aggregation of both manual & automated testing results

Testspace’s built-in Manual Runner automatically includes results. However, to get a holistic view of the health of a project automated results are also required. Although most Test Management applications provide a means (i.e. API) to include automated tests results, they are not well suited for handling the large number of results that are typically generated by automated CI systems.

Testspace has been specifically designed for high volume automation. It aggregates data from all your build and test systems and performs analysis on the content. See our article on connected Repos for CI testing.

To upload content to the Testspace server only a single command is required:

testspace result*.xml coverage.xml log.txt .. 

Now the Status of the software, regardless of the type of testing, is seamlessly aggregated. All of the metrics: test results, code coverage, defects, requirements, etc., are collected and used together.

Aggregated reporting

See our article on Test Analytics and how it is applied to the data collected. And for more advanced metric collection see our article on creating custom graphs.

Authoring Manual Tests

Traditional Test Management applications almost universally use a proprietary User Interface to capture the test specifications/instructions. We’ve taken what we believe is a better approach by using natural language (plain text) for a common understanding that can be automated while maintaining the instructions in a Repository (i.e. use a Pull Request when adding new test cases).

Using natural language for common understanding that can be easily automated!

As Testspace has built-in support for continuous integration reporting, we felt it important to facilitate automating manual tests – when the timing makes sense. We decided to standardize the test specification format on ThoughtWorks open source automation test framework called Gauge. Gauge is similar to the Cucumber’s automation framework, which uses the Gherkin language. But we wanted a less rigid specification syntax, so we went with the Gauge format. Refer to the ThoughtWorks article on why gauge was built. Note that automating a test requires no changes to the instructions (specification).

The tests are maintained in the Code section of the Repo using folders and markdown files. The following example captures a test that includes two specific test cases.

Test Specifications in Repo

For details on the test format refer here.

Test Plans

GitHub has implemented project management functionality, which has significant adoption amongst the development community. Rather than re-invent our own proprietary management features (i.e. separated workflow), we’ve integrated with GitHub.

To create a Test Plan a GitHub kanban board is used. Based on a simple configuration, when a new board is created, tests contained in the Repo are automatically discovered, with corresponding Cards (Notes) automatically added to the board for status tracking.

Repo Projects for managing

Testers can be automatically assigned to execute tests based on the Project Board. The Notes corresponding to the tests are automatically updated with status.

Testers can execute independent of GitHub, project stakeholders can manage and track independent of Testspace

Running Manual Tests

Tests are checked into a Repo and managed the same as development code. Testspace is used to execute tests with its built-in Runner.

Test runs

Get setup in minutes!

Try Testspace risk free. No credit card required.

Have questions? Contact us.

Testspace works seamlessly with the vast majoritiy of CI systems and online services, including

Travis CI Circle CI Jenkins Appveyor Shippable GitLab CI Bitbucket Pipelines