http://blog.xebia.com/conditionally-running-tests-in-testng/
https://stackoverflow.com/questions/33476985/how-to-skip-or-ignore-execution-of-tests-in-testng
public class SuiteListener implements ISuiteListener {
private static boolean hasFailedTest;
@Override
public void onStart(final ISuite suite) {
if (hasFailedTest) {
throw new SkipException("Skip suite: " + suite.getName());
}
}
@Override
public void onFinish(final ISuite suite) {
final Map<String, ISuiteResult> testResults = suite.getResults();
if (testResults != null) {
for (final ISuiteResult suitResult : testResults.values()) {
if (suitResult.getTestContext().getFailedTests().size() != 0) {
hasFailedTest = true;
break;
}
}
}
}
}
Fail instead of Skip a Test when TestNG’s DataProvider throws an Exception
http://rolf-engelhard.de/2011/10/fail-instead-of-skip-a-test-when-testngs-dataprovider-throws-an-exception/
http://stackoverflow.com/questions/23872603/testng-skips-test-after-raising-exception-in-dataprovider-method
https://groups.google.com/forum/#!topic/testng-users/BkIQeI0l0lc
https://examples.javacodegeeks.com/enterprise-java/testng/testng-listeners-example/
http://stackoverflow.com/questions/30682107/how-to-get-the-passed-failed-and-skipped-method-count-from-the-testng-suite-lev
Use the alwaysRun=true attribute on the @AfterX.
https://niharikawhatsnext.wordpress.com/2014/10/14/testng-how-to-stop-further-execution-of-a-test-on-x-times-failure-for-invocationcount/
https://groups.google.com/forum/#!topic/testng-users/3RmAe_UICGw
Yes, and @After methods will run in the reverse order.
This is unspecified, so you should not rely on any order.
http://stackoverflow.com/questions/30587454/difference-between-beforeclass-and-beforetest-in-testng
https://examples.javacodegeeks.com/enterprise-java/testng/testng-parameters-annotation-example/
Pass environment data
http://stackoverflow.com/questions/13245411/how-to-pass-java-code-a-parameter-from-maven-for-testing
https://groups.google.com/forum/embed/#!topic/testng-users/afs6YmrAsB0
Share data
https://rationaleemotions.wordpress.com/2014/02/06/sharing-parameters-among-different-testng-suites/
lets create a Suite of Suites as below : [ We will call this suite file as “testng-mastersuite.xml”]
Approach (2)
Now lets create a basic TestNG xml suite file and then refer to the above mentioned parameters.xml in it.
When a method is annotated with @DataProvider, it becomes a data feeder method by passing the test data to the test case.
@DataProvider
public Object[][] testData() {
return new Object[][] {
new Object[] { "160", "45", "17.6", "Underweight" },
new Object[] { "168", "70", "24.8", "Normal" },
new Object[] { "181", "89", "27.2", "Overweight" },
new Object[] { "178", "100", "31.6", "Obesity" }, };
}
The test case method is linked to the data feeder method by passing the name of the dataProvider method to the @Test annotation:
@Test(dataProvider = "testData")
public void testBMICalculator(String height, String weight, String bmi, String category)
Integrate with Spring
https://examples.javacodegeeks.com/enterprise-java/testng/testng-spring-integration-example/
https://stackoverflow.com/questions/33476985/how-to-skip-or-ignore-execution-of-tests-in-testng
You can throw
SkipException
's in your tests if assumptions does not hold, and that mechanism is flexible enough to even mark the test as skipped or failed. This article shows how to integrate this approach in a declarative manner in your test suite. Basically you annotate methods with @Assumes
and have a custom IInvokedMethodListener
.public class SuiteListener implements ISuiteListener {
private static boolean hasFailedTest;
@Override
public void onStart(final ISuite suite) {
if (hasFailedTest) {
throw new SkipException("Skip suite: " + suite.getName());
}
}
@Override
public void onFinish(final ISuite suite) {
final Map<String, ISuiteResult> testResults = suite.getResults();
if (testResults != null) {
for (final ISuiteResult suitResult : testResults.values()) {
if (suitResult.getTestContext().getFailedTests().size() != 0) {
hasFailedTest = true;
break;
}
}
}
}
}
Fail instead of Skip a Test when TestNG’s DataProvider throws an Exception
http://rolf-engelhard.de/2011/10/fail-instead-of-skip-a-test-when-testngs-dataprovider-throws-an-exception/
http://stackoverflow.com/questions/23872603/testng-skips-test-after-raising-exception-in-dataprovider-method
Solution 4: FailListener
TestNG gives the ability to register listeners to your test execution [2]. So you can code a “FailListener” which switches every skipped test to a failed one:
public class FailListener extends TestListenerAdapter { @Override public void onTestSkipped(ITestResult tr) { tr.setStatus(ITestResult.FAILURE); } }
For me best way happened to add test for data providers and here is brief illustration of idea:
public class MyClass {
@DataProvider(name = "my-data-provider")
private Object [][] myProvider() {
// ...
}
@Test
public void testDataProviders() {
Assert.assertTrue(myProvider().length > 0);
}
@Test
// ... Real tests.
}
how to stop a test suite if one method failshttps://groups.google.com/forum/#!topic/testng-users/BkIQeI0l0lc
https://examples.javacodegeeks.com/enterprise-java/testng/testng-listeners-example/
http://stackoverflow.com/questions/30682107/how-to-get-the-passed-failed-and-skipped-method-count-from-the-testng-suite-lev
You can get all information through following code :
//Iterating over each suite included in the test
for (ISuite suite : suites) {
//Following code gets the suite name
String suiteName = suite.getName();
//Getting the results for the said suite
Map<String,ISuiteResult> suiteResults = suite.getResults();
for (ISuiteResult sr : suiteResults.values()) {
ITestContext tc = sr.getTestContext();
System.out.println("Passed tests for suite '" + suiteName +
"' is:" + tc.getPassedTests().getAllResults().size());
System.out.println("Failed tests for suite '" + suiteName +
"' is:" +
tc.getFailedTests().getAllResults().size());
System.out.println("Skipped tests for suite '" + suiteName +
"' is:" +
tc.getSkippedTests().getAllResults().size());
System.out.println("Total excution time for test '" + tc.getName() +
"' is:" + (tc.getEndDate().getTime()- tc.getStartDate().getTime()));
I didn't find any method at Suite level to achieve this
Use the alwaysRun=true attribute on the @AfterX.
https://niharikawhatsnext.wordpress.com/2014/10/14/testng-how-to-stop-further-execution-of-a-test-on-x-times-failure-for-invocationcount/
Problem Statement : A test which has been set with invocationCount=10 (say), failed 4 times. In such a situation, it makes sense to stop execution for this particular testcase and save further execution resources.
Solution : IInvokedMethodListener
The listener exposes two methods :
1.
beforeInvocation(IInvokedMethod method, ITestResult testResult)
2.
afterInvocation(IInvokedMethod method, ITestResult testResult)
The methods of this interface are called everytime before and after an @Test method runs.
//Map to maintain method name and its failure count
static Map<String, Integer> methodFailCount = Collections.synchronizedMap(new HashMap<String, Integer>());
public void beforeInvocation(IInvokedMethod method, ITestResult testResult) {
//If the failure count for this method has surpassed the set limit,
//skip the next execution
String methodName = method.getTestMethod().getMethodName();
if(methodFailCount.get(methodName)!= null && methodFailCount.get(methodName) > 4)
throw new SkipException("Skipped due to failure count > 4");
}
public void afterInvocation(IInvokedMethod method, ITestResult testResult) {
String methodName = method.getTestMethod().getMethodName();
//If test has failed, add to fail count
if(testResult.getStatus() == TestResult.FAILURE){
if(methodFailCount.get(methodName ) == null)
methodFailCount.put(methodName,1);
else{
methodFailCount.put(methodName, methodFailCount.get(methodName )+1);
}
}
}
Especially if there is a class hierarchy, will the Annotated methods
of superclasses execute first ?
for instance:
suppose that SuperClass and ChildClass contain both a method annotated
wirh @BeforeMethod.
Will the @BeforeMethod of the superclass run first and the
@BeforeMethod of the child second ?
Yes, and @After methods will run in the reverse order.
another case:
If a class contains multiple @BeforeMethods, will the methods be
executed by method-name-order ?
This is unspecified, so you should not rely on any order.
http://stackoverflow.com/questions/30587454/difference-between-beforeclass-and-beforetest-in-testng
https://examples.javacodegeeks.com/enterprise-java/testng/testng-parameters-annotation-example/
http://stackoverflow.com/questions/13245411/how-to-pass-java-code-a-parameter-from-maven-for-testing
mvn clean test -Denv.USER=UAT -Dgroups=Sniff
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.12.4</version>
<configuration>
<systemPropertyVariables>
<environment>${env.USER}</environment>
</systemPropertyVariables>
</configuration>
</plugin>
</plugins>
</build>
@Test (groups = { "Regression" },parameters = {"environment"})
public void failedAuthenticationTest(String environment){
System.out.println("Regression-"+environment);
}
@Parameters("environment")
@Test (groups = { "Sniff"})
public void newUserAuthenticationTest(String environment){
System.out.println("Sniff-"+environment);
}
The above works well. Additionally, if you need to use
testng.xml
, you can specify the suiteXmlFile
like ... <plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.12.4</version>
<configuration>
<systemPropertyVariables>
<environment>${env.USER}</environment>
</systemPropertyVariables>
<suiteXmlFiles>
<suiteXmlFile>testng.xml</suiteXmlFile>
</suiteXmlFiles>
</configuration>
</plugin>
Also, I prefer using
@Parameters
instead of parameters
in @Test()
as the later is deprecated.Share data
https://rationaleemotions.wordpress.com/2014/02/06/sharing-parameters-among-different-testng-suites/
Leverage what TestNG directly has to offer, which is “create a suite of suites”. Incase you are wondering how to get this done, here’s how you do it.
@Test
@Parameters
({
"username"
})
public
void
f(String userName) {
Assert.assertTrue(userName.equals(
"testng"
));
}
<
suite
name
=
"MasterSuite"
parallel
=
"false"
>
<
parameter
name
=
"username"
value
=
"testng"
></
parameter
>
<
suite-files
>
<
suite-file
path
=
"src/test/resources/testng-childsuite.xml"
></
suite-file
>
</
suite-files
>
</
suite
>
<!-- MasterSuite -->
Now lets create a simple xml file and add up all the parameters that we feel need to be shared across multiple suites.
I will call it my “parameters.xml” and it is available under
I will call it my “parameters.xml” and it is available under
src/test/resources
folder in my project.<
parameter
name
=
"username"
value
=
"testng"
></
parameter
>
<!
DOCTYPE
suite SYSTEM "http://testng.org/testng-1.0.dtd" [<!ENTITY parent SYSTEM "src/test/resources/parameters.xml">]>
<
suite
name
=
"MyTestSuite"
parallel
=
"false"
verbose
=
"2"
>
<
parameters
>&parent;</
parameters
>
<
test
name
=
"MyTest"
>
<
classes
>
<
class
name
=
"rationale.emotions.ReadParameters"
/>
</
classes
>
</
test
>
</
suite
>
[<!ENTITY parent SYSTEM "src/test/resources/parameters.xml">]>
is basically our way of saying that this xml file has an entity (which we will call as “parent”) and it maps to the resource parameters.xml that resides under src/test/resources
folder.
&parent; is basically our way of saying that within the tag we are going to be replacing the contents of parameters.xml, which we are denoting via the name “parent” (notice that we gave our entity its name as “parent”). The entity name is supposed to be prefixed by an ampersand and it is supposed to end with a semi-colon.
Approach 1: Via the TestClass’s instance.
public
class
Generatorv2 {
public
static
final
String MY_ATTRIBUTE =
"sumOfValues"
;
private
List<Integer> sumValue;
@AfterClass
(alwaysRun =
true
)
public
void
insertValueIntoAttribute(ITestContext ctx) {
ctx.setAttribute(MY_ATTRIBUTE, sumValue);
}
public
Generatorv2() {
sumValue =
new
ArrayList<Integer>();
}
@Test
(groups =
"parent"
, dataProvider =
"generateNumbers"
)
public
void
testNumbers(Integer a, Integer b) {
assertTrue(a !=
0
);
assertTrue(b !=
0
);
sumValue.add(a + b);
}
@DataProvider
public
Object[][] generateNumbers() {
return
new
Object[][] { {
1
,
2
}, {
3
,
4
} };
}
}
public
class
Consumerv2 {
private
List<Integer> allTheValues =
null
;
@BeforeClass
(alwaysRun=
true
)
@SuppressWarnings
(
"unchecked"
)
public
void
fetchData(ITestContext ctx) {
allTheValues = (List<Integer>) ctx.getAttribute(Generatorv2.MY_ATTRIBUTE);
}
@Test
(dependsOnGroups =
"parent"
, groups =
"child"
)
public
void
myCrazyTest() {
assertNotNull(allTheValues);
assertFalse(allTheValues.isEmpty());
assertTrue(allTheValues.contains(
new
Integer(
3
)));
assertTrue(allTheValues.contains(
new
Integer(
7
)));
}
}
<
suite
name
=
"Suite"
parallel
=
"false"
verbose
=
"2"
>
<
test
name
=
"Test"
>
<
groups
>
<
run
>
<
include
name
=
"parent"
></
include
>
<
include
name
=
"child"
></
include
>
</
run
>
</
groups
>
<
classes
>
<
class
name
=
"organized.chaos.testng.Generatorv2"
/>
<
class
name
=
"organized.chaos.testng.Consumerv2"
/>
</
classes
>
</
test
>
</
suite
>
https://community.perfectomobile.com/posts/988612-manage-testng-execution-and-data
http://stackoverflow.com/questions/25943952/testng-pass-test-parameter-between-different-test-class
DataProvider
http://www.mkyong.com/unittest/testng-tutorial-6-parameterized-test/
http://stackoverflow.com/questions/25943952/testng-pass-test-parameter-between-different-test-class
DataProvider
http://www.mkyong.com/unittest/testng-tutorial-6-parameterized-test/
@Test(dataProvider = "dataProvider", groups = {"groupA"})
public void test1(int number) {
Assert.assertEquals(number, 1);
}
@Test(dataProvider = "dataProvider", groups = "groupB")
public void test2(int number) {
Assert.assertEquals(number, 2);
}
@DataProvider(name = "dataProvider")
public Object[][] provideData(ITestContext context) {
Object[][] result = null;
//get test name
//System.out.println(context.getName());
for (String group : context.getIncludedGroups()) {
System.out.println("group : " + group);
if ("groupA".equals(group)) {
result = new Object[][] { { 1 } };
break;
}
}
if (result == null) {
result = new Object[][] { { 2 } };
}
return result;
}
In TestNG 5.10 setAttribute() and getAttribute() methods are
implemented at suite level. Yo can use those methods to share data
within test suite.
implemented at suite level. Yo can use those methods to share data
within test suite.
When a method is annotated with @DataProvider, it becomes a data feeder method by passing the test data to the test case.
@DataProvider
public Object[][] testData() {
return new Object[][] {
new Object[] { "160", "45", "17.6", "Underweight" },
new Object[] { "168", "70", "24.8", "Normal" },
new Object[] { "181", "89", "27.2", "Overweight" },
new Object[] { "178", "100", "31.6", "Obesity" }, };
}
The test case method is linked to the data feeder method by passing the name of the dataProvider method to the @Test annotation:
@Test(dataProvider = "testData")
public void testBMICalculator(String height, String weight, String bmi, String category)
Integrate with Spring
- – we will be loading
- – to access spring’s testing framework.
The Spring test framework is able to understand extension over configuration. It means that you only need to extend
MySpecialIntegrationTest
from MyIntegrationTest
:@ContextConfiguration(classes = MySpecialIntegrationTestConfig.class, loader = SpringApplicationContextLoader.class)
public class MySpecialIntegrationTest extends MyIntegrationTest {
@Configuration
public static class MySpecialIntegrationTestConfig {
@Bean
public MyBean theBean() {}
}
}
and create the necessary Java Config class and provide it to
@ContextConfiguration
. Spring will load the base one and extend it with the one that you specialize in your extended test case.
The
org.springframework.mock.env
package contains mock implementations of the Environment
and PropertySource
abstractions (see Section 6.13.1, “Bean definition profiles” and Section 6.13.3, “PropertySource abstraction”). MockEnvironment
and MockPropertySource
are useful for developing out-of-container tests for code that depends on environment-specific properties.@RunWith(SpringJUnit4ClassRunner.class) @ContextConfiguration({"/app-config.xml", "/test-data-access-config.xml"}) @ActiveProfiles("dev") @Transactional public class UserRepositoryTests { }
We can reduce the above duplication by introducing a custom composed annotation that centralizes the common test configuration like this:
@Target(ElementType.TYPE) @Retention(RetentionPolicy.RUNTIME) @ContextConfiguration({"/app-config.xml", "/test-data-access-config.xml"}) @ActiveProfiles("dev") @Transactional public @interface TransactionalDevTest { }
Then we can use our custom
@TransactionalDevTest
annotation to simplify the configuration of individual test classes as follows:@RunWith(SpringJUnit4ClassRunner.class) @TransactionalDevTest public class OrderRepositoryTests { }
@RunWith(SpringJUnit4ClassRunner.class)
// ApplicationContext will be loaded from the static inner ContextConfiguration class
@ContextConfiguration(loader=AnnotationConfigContextLoader.class)
public class OrderServiceTest {
@Configuration
static class ContextConfiguration {
// this bean will be injected into the OrderServiceTest class
@Bean
public OrderService orderService() {
OrderService orderService = new OrderServiceImpl();
// set properties, etc.
return orderService;
}
}
@Autowired
private OrderService orderService;
@Test
public void testOrderService() {
// test the orderService
}
}