Reports
The Test ProjectSDK tightly integrates with the TestProject application. Test that are run with the SDK, can send data to TestProject application so that you can use the powerful reporting functionaliy in there to view and share information about test runs.
When running tests with the SDK, TestProject will automaticaly create a project for you so that it can use if for reporting. This project needs to have a name. You can explicitly set that name wiht the constructor if you want.
Project and Job names can be specified explicitly using this constructor:
public ChromeDriver(final ChromeOptions options, final String projectName, final String jobName)
For example the following code would create a project called My First Project and the job that ran tests using this would be called My First Job :
ChromeDriver driver = new ChromeDriver(new ChromeOptions(), "My First Project", "My First Job");
Alternatively, you could do this with the
DriverBuilderusing code like this:ChromeDriver driver = new DriverBuilder<ChromeDriver>(new ChromeOptions())
.withProjectName("My First Project")
.withJobName("My First Job")
.build(ChromeDriver.class);
You can see some example of examples of tests using explicit Project & Job names configuration in the following files:
If you do not explicitly specify project and job names, the SDK will attempt to infer them from JUnit / TestNG annotations.
It will use the following logic when naming projects and jobs in the TestProject App:
- Package name of the class containing the method is used for Project name.
- JUnit 4 / 5
- Class name or the @DisplayName annotation (JUnit 5 only) on the class is used for the Job name
- Method name or the @DisplayName annotation (JUnit 5 only) on the method is used for the Test name(s)
- TestNG
- Class name or description from @BeforeSuite / @BeforeClass annotations if found, are used for the Job name
- Method name or the @Test annotation (and it's testName / description fields) on the method is used for the Test name(s)
You can see some example of examples of tests using implicit Project & Job names inferred from annotations at these links:
Tests are reported automatically when a test ends or when driver quits.
This behavior can be overridden or disabled.
In order to determine that a test ends, the call stack is traversed searching for an annotated methods. When an annotated method execution starts and a previously detected one ends, it is assumed that a test has completed.
Any unit testing framework annotations is considered, creating a separate test in the report for every annotated method. For example, the following JUnit based code, will generate the following six tests in the report:
@BeforeEach
void beforeTestExample(TestInfo testInfo) {
driver.report().step("Preparing Test: " + testInfo.getDisplayName());
}
@Test
@DisplayName(value = "Google")
void testGoogle() {
driver.navigate().to("https://www.google.com/");
}
@Test
@DisplayName(value = "Yahoo!")
void testYahoo() {
driver.navigate().to("https://yahoo.com/");
}
@AfterEach
void afterTestExample(TestInfo testInfo) {
driver.report().step("Finishing Test: " + testInfo.getDisplayName());
}
Report:
Report
├── beforeTestExample
│ ├── Preparing Test: Yahoo!
├── Yahoo! Test
│ ├── Navigate To https://yahoo.com/
├── afterTestExample
│ ├── Finishing Test: Yahoo!
├── beforeTestExample
│ ├── Preparing Test: Google
├── Google Test
│ ├── Navigate To https://google.com/
└── afterTestExample
└── Finishing Test: Google
JUnit5 dynamic test names cannot be inferred, and should be reported manually. These will be reported as Dynamic Test when reported automatically.
To report tests manually, use
driver.report().tests() method and it's overloads. For example:ChromeDriver driver = new ChromeDriver(new ChromeOptions());
driver.report().disableTestAutoReports(true);
driver.report().test("My First Test").submit();
Note that
driver.report().test() returns a ClosableTestReport object. An explicit call to submit() or closing the object is required for the report to be sent.Using this closable object can be beneficial in the following case:
ChromeDriver driver = new ChromeDriver(new ChromeOptions());
try (ClosableTestReport report = driver.report().test("Example Test with Exception")) {
driver.findElement(By.id("NO_SUCH_ELEMENT")).click();
}
Assuming there is no element on the DOM with such an ID:
NO_SUCH_ELEMENT, an exception will be thrown and test will fail, but before that, the closable object will get closed and so the test will still be reported.Test steps are reported automatically when driver commands are executed. If this feature is disabled, or in addition, manual reports can be performed, for example:
ChromeDriver driver = new ChromeDriver(new ChromeOptions());
driver.report().step("User logged in successfully");
If you don't want to see reports for your runs, you can disable them by setting the
disableReports option in the constructor.public ChromeDriver(final ChromeOptions options, final boolean disableReport
If reports were not disabled when the driver was created, they can be disabled or enabled later. However, if reporting was explicitly disabled when the driver was created, it can not be enabled later in the file.
The following code will disable all types of reports:
ChromeDriver driver = new ChromeDriver(new ChromeOptions());
driver.report().disableReports(true);
If you only want to disable automatic reports, but still allow for manul reporting, you can use the the following code:
ChromeDriver driver = new ChromeDriver(new ChromeOptions());
driver.report().disableTestAutoReports(true);
When you do this, all test steps will reside in a single test report, unless the tests are reported manually using
driver.report().tests():The following code will disable the reporting of driver commands.
ChromeDriver driver = new ChromeDriver(new ChromeOptions());
driver.report().disableCommandReports(true);
In the case, the report will have no steps, unless they are reported manually using
driver.report().step():When reporting driver commands, the SDK performs a redaction of sensitive data (values) sent to secured elements. If the element is one of the following:
- Any element with
typeattribute set topassword - With XCUITest, on iOS an element type of
XCUIElementTypeSecureTextField
Values sent to these elements will be converted to three asterisks -
***. This is here for security purposes, but if you want to you can disabled it with the following code:ChromeDriver driver = new ChromeDriver(new ChromeOptions());
driver.report().disableRedaction(true);
By default, the execution report is uploaded to the cloud, and a local report is created, as an HTML file in a temporary folder.
At the end of execution, the report is uploaded to the cloud and SDK outputs to the console/terminal the path for a local report file:
Execution Report: {temporary_folder}/report.htmlThis behavior can be controlled, by requesting only a
LOCAL or only a CLOUD report.When the Agent is offline, and only a cloud report is requested, execution will fail with appropriate message.
Via a driver constructor:
ChromeDriver driver = new ChromeDriver(new ChromeOptions(), ReportType.LOCAL);
Or via the builder:
ChromeDriver driver = new DriverBuilder<ChromeDriver>(new ChromeOptions())
.withReportType(ReportType.LOCAL)
.build(ChromeDriver.class);
Last modified 1yr ago