March 6, 2021

How to Combine, Order and Run test methods programmatically as ordered tests using visual studio ( Coded UI + .Net )

  March 6, 2021

 Since the ordered tests in visual studio are depreciated, we now need to find another way to combine different test methods and run them in a specific order. 

For this, we need to programmatically invoke the test methods by calling all the test methods we need inside another test method.

A sample is given below.

//Test cases
[TestMethod]
public void TestMethod1()
{
    reporthandle.LogReport("Passed", "Test", "Test Description");
}

//Test cases
[TestMethod]
public void TestMethod2()
{
    reporthandle.LogReport("Failed", "Test", "Test Description");
}


//Ordered test creation using .Net Code
[TestMethod] 
public void OrderedStepsCustomRun()
{
	OrderedTestSetup.Run(TestContext, new List<OrderedTestSetup>
	{
		//Write test Method names in the order you need
		new OrderedTestSetup ( TestMethod1, true ), // continue on failure
		new OrderedTestSetup ( TestMethod2, false ), // not continue on failure
		// ...
	});
}

Now we need the backend code to handle our ordered tests created using .Net code.

For this, we have created a whole class which deals with needed methods and it invokes each test method using testcontext as the common variable.
It also provides an option to continue on failure or not, if any of the test cases fail in the ordered test that we created.

using Microsoft.VisualStudio.TestTools.UnitTesting;
using System;
using System.Collections.Generic;
using System.Diagnostics;
using System.Linq;

namespace MyExperiments.CommonLibrary
{
    public class OrderedTestSetup
    {
        /// <summary>Test Method to run</summary>
        public Action TestMethod { get; private set; }

        /// <summary>Flag indicating whether testing should continue with the next test if the current one fails</summary>
        public bool ContinueOnFailure { get; private set; }

        /// <summary>Any Exception thrown by the test</summary>
        public Exception ExceptionResult;

        /// <summary>
        /// Constructor
        /// </summary>
        /// <param name="testMethod"></param>
        /// <param name="continueOnFailure">True to continue with the next test if this test fails</param>
        public OrderedTestSetup(Action testMethod, bool continueOnFailure = false)
        {
            TestMethod = testMethod;
            ContinueOnFailure = continueOnFailure;
        }

        /// <summary>
        /// Run the test saving any exception within ExceptionResult
        /// Throw to the caller only if ContinueOnFailure == false
        /// </summary>
        /// <param name="testContextOpt"></param>
        public void Run()
        {
            try
            {
                TestMethod();
            }
            catch (Exception ex)
            {
                ExceptionResult = ex;
                throw;
            }
        }

        /// <summary>
        /// Run a list of OrderedTest's
        /// </summary>
        static public void Run(TestContext testContext, List<OrderedTestSetup> tests)
        {
            Stopwatch overallStopWatch = new Stopwatch();
            overallStopWatch.Start();

            List<Exception> exceptions = new List<Exception>();

            int testsAttempted = 0;
            for (int i = 0; i < tests.Count; i++)
            {
                OrderedTestSetup test = tests[i];

                Stopwatch stopWatch = new Stopwatch();
                stopWatch.Start();

                testContext.WriteLine("Starting ordered test step ({0} of {1}) '{2}' at {3}...\n",
                    i + 1,
                    tests.Count,
                    test.TestMethod.Method,
                    DateTime.Now.ToString("G"));

                try
                {
                    testsAttempted++;
                    test.Run();
                }
                catch
                {
                    if (!test.ContinueOnFailure)
                        break;
                }
                finally
                {
                    Exception testEx = test.ExceptionResult;

                    if (testEx != null)  // capture any "continue on fail" exception
                        exceptions.Add(testEx);

                    testContext.WriteLine("\n{0} ordered test step {1} of {2} '{3}' in {4} at {5}{6}\n",
                        testEx != null ? "Error:  Failed" : "Successfully completed",
                        i + 1,
                        tests.Count,
                        test.TestMethod.Method,
                        stopWatch.ElapsedMilliseconds > 1000
                            ? (stopWatch.ElapsedMilliseconds * .001) + "s"
                            : stopWatch.ElapsedMilliseconds + "ms",
                        DateTime.Now.ToString("G"),
                        testEx != null
                            ? "\nException:  " + testEx.Message +
                                "\nStackTrace:  " + testEx.StackTrace +
                                "\nContinueOnFailure:  " + test.ContinueOnFailure
                            : "");
                }
            }

            testContext.WriteLine("Completed running {0} of {1} ordered tests with a total of {2} error(s) at {3} in {4}",
                testsAttempted,
                tests.Count,
                exceptions.Count,
                DateTime.Now.ToString("G"),
                overallStopWatch.ElapsedMilliseconds > 1000
                    ? (overallStopWatch.ElapsedMilliseconds * .001) + "s"
                    : overallStopWatch.ElapsedMilliseconds + "ms");

            if (exceptions.Any())
            {
                // Test Explorer prints better msgs with this hierarchy rather than using 1 AggregateException().
                throw new Exception(String.Join("; ", exceptions.Select(e => e.Message), new AggregateException(exceptions)));
            }
        }
    }
}

As given in the first part of this article, We can create the object of this class and execute our test method using OrderedStepsCustomRun() in our OrderedTestSetup class.

From the above example, we can call OrderedStepsCustomRun() as a normal test method from test explorer and it invokes all the test methods inside it in the order we provided.
logoblog

Thanks for reading How to Combine, Order and Run test methods programmatically as ordered tests using visual studio ( Coded UI + .Net )

Previous
« Prev Post

No comments:

Post a Comment

Bookmark this website for more workarounds and issue fixes.

Verify Zip file contents without extracting using C# + Selenium

While doing automation testing, we may get a scenario where we need to download a zip file and to validate the contents inside it. One way t...