OpenTest Reporting Library

Question

I am currently seeking information around the available reporting capabilities for OpenTest. I would need information regarding the following:

Portability of reports/logging - can these results be published in various formats

Granularity of reports/logging - is there a way of getting very detailed in what is reported on and/or strategies to ensure that enough information is logged to allow for debugging of automated tests and System Under Test(SUT)

Screenshots - is there current functionality to allow for screenshots to be taken and published/posted to external systems?

Answer 1

Portability of reports/logging

You can obtain the test session results by using the API, either in JSON format (which contains a lot of detail) or the JUnit XML format:

http://localhost:3000/api/session/<SESSION_ID>?format=json
http://localhost:3000/api/session/<SESSION_ID>?format=junit

The detailed log of the test session can be retrieved in: JSON or human-readable format:

http://localhost:3000/api/session/<SESSION_ID>/log?format=json
http://localhost:3000/api/session/<SESSION_ID>/log?format=pretty

Granularity of reports/logging

The test results in JSON format will tell you everything you need to know about the pass/fail status for each test and each individual test action within a test, arguments that were used for test actions, the name of the screenshot captured for each test action, execution times and many other useful information.

When you want to troubleshoot a failed test, most of the time you'll need the detailed log information, which can be retrieved using the APIs I mentioned above. Aside from the log information generated by OpenTest itself, you can always log additional information that is specific to your application or test scenario using the $log JavaScript API.

Screenshots

Screenshots are automatically captured for Web and UI tests, whenever a test action fails. If you need to capture additional screenshots during your test, you can do so using the TakeScreenshot keyword for either Web testing or mobile testing. You can also capture a screenshot after any test action by using the $screenshot global test action argument:

- description: Click product 1 and capture a screenshot
  action: org.getopentest.selenium.Click
  args:
    locator: { id: product1 }
    $screenshot: true

You can download screenshots using this API:

https://localhost:3000/api/screenshot/SID1554380072_WEB_T05_SG01_ST01_after_03.png

SID1554380072_WEB_T05_SG01_ST01_after_03.png is the name of the screenshot file, which you can find in the test execution results in JSON format.

Integrating with custom reporting solutions

At some point you will need to integrate with a dedicated reporting product that can give you all the nice features that OpenTest is not be able to provide out-of-the-box. This is possible to do using the APIs I described. In order to notify interested parties of the current status of test sessions, OpenTest also provides a WebSocket API. You can use that to be notified when a test session has finished, then you can extract all the information you need via APIs. You can find a Java project that does all that here: https://github.com/adrianth/opentest-monitor. This project is intended as a starting point for your own custom integration.