Views: 8588
Number of votes: 9
Average rating:

Automated Web Testing Gadget

I usually find myself trying to automate stuff that I find to be the "less funny part", such as testing :), so when I heard of the Gadget Contest, it felt like a good time to start the automatization process :). This is my contribution:

This gadget is a tool to support automated testing of EPiServer-based web applications. The gadget is targeted to both developers but mostly to the end-users of the EPiServer web application, like webmasters, editors and so on.

Why automate?

Testing is often conducted manually and this is usually effective enough. Much critical functionality in a web application could however be monitored by programmed automated testcases which can be executed without any human interaction. Automated testcases can be repeatable and executed at times when you're too tired to monitor your application :-). Automated tests can therefore be a great way to secure the critical functions on a website.

Testcases could for example be used to improve the overall quality of the delivered solution where both developers and the customers can verify that the functionality is working correctly and if not, get a notification about this. Failing functionality will hopefully be detected by the automated tests before your visitors suffers. Testcases could also be used to get information about uptime/availability for certain isolated functionality (and not just the IIS), like the payment system in the webshop.

What does the gadget do?

Tests are either executed manually from the control panel, or scheduled for automated and repeatable execution with the help of EPiServers Scheduling Service.

image

It's up to you to design and program your own testcases (see the included example testcases for EPiServer World). The gadget supports existing web application frameworks like Selenium or Watin. I’d recommend you to use the Selenium IDE add-on for Firefox to record your tests and export it as C# code. Insert the generated code into a class library project, compile, and you’re almost good to go.

The gadget uses reflection to "parse" the dll files in you /bin directory. All classes marked with an [WebPageTypeTest] will be shown in the gadget as a TestSuite. All methods withing the class marked with [WebUnitTest] attribute will be TestCases beloning to the TestSuite which will be shown in the gadget. So basicly you have to follow 3 rules to make your testcases show up in the UI.

  1. Create one class per testsuite and mark your class with [WebPageTypeTest]
  2. Create one method per testcase and mark it with [WebUnitTest]
  3. Always make your testcase method to return a TestResult object.

See the following example:

 image

The located testsuites/testcases gets listed in the gadget “Live monitor” view. Once you hit the start button in the control panel, the “Live Monitor” will be opened and you will be able to follow the progress of the test procedure.

image

The Alert checkboxes can be checked if you would like to be notified by mail when a specific test fails. The alert functionality is personalized so each user can have their own personalized alerts. The notification mail is sent to EPiServerProfile.EmailWithMembershipFallback. When the test is finished and the progress bar have reached 100% you’re able to view a more detailed report in the “Report View”.

The report can be generated in predefined timespans like,

  1. The last day
  2. The last week
  3. The last month
  4. The last year

image

The report can be further customized by using column sorting or free text filtering, in the pictures below I used the textsearch to filter testcases executed the 16th November. If you choose to generate a report which contains more than one date of test results, a graph will be generated showing failed tests in percent per date..

image 

image

Testresults are stored with the help of the new DynamicDataStore. Thank you for this excellent feature Mr EPi, love the linq support in combination with MVC Json object!

Feature idea’s

  • Make it possible to attach your own "notify on fail" plugins so developers can program their own, some example alert-plugins could be sending text messages (sms), writing to the eventlog etc.

Source

Download the gadget source here

Get started

  1. Download AutomatedTestGadget.epimodule
  2. Install the gadget by using “Install compressed module” selection in EPiServer Deployment Center.
  3. Add the following to web.config as described here:
    1. <add assembly="AutomatedWebTestGadget" />
  4. Develop your testcases, deploy them to your /bin directory or try out the included demo testcases.
  5. Check out a sample project if you need help to get started with your testcases.

The sample project includes a couple of “fake” testcases and one testcase written for selenium. The testcase will be executed in both Firefox and IE. If you don’t like the example testcases, just delete SeleniumTests.dll from your bin directory.

Happy testing!

Nov 19, 2009

Guest
(By Guest, 9/21/2010 12:32:54 PM)

Selenium is a fantastic tool, but I've always struggled to find a natural place for it to sit and verify Ui functionality once a site has gone live. I've generally always used it for acceptance testing. Integrating this into the Gadget framework and providing reporting to all stakeholders is a really really great idea.
/ Mark Everard

Sofia Max
(By Sofia Max, 9/21/2010 12:32:55 PM)

Great work Jonas. I am very proud of you!

Guest
(By Guest, 9/21/2010 12:32:55 PM)

I LOVE IT. Great idea, great implementation and very very useful. You won my vote (if I had one, that is).
/ Allan Thraen

Guest
(By Guest, 9/21/2010 12:32:55 PM)

Very good work (as usual with you :) Thank you for a very good gadget and the inspiration for the rest of us!
/ Jörgen Niemi

jonas.persson
(By jonas.persson, 9/21/2010 12:32:55 PM)

Thanks for your feedback! I'm glad to hear that you find it usefull.

I've added some new features and corrected some minor issues. The link in the post still applies. The following have been changed:

* The old "Print table" function has been removed (since the print didnt look very good) and replaced by 3 new functions. You're now able to export the report as csv, xls or just print it (new print function, thanks to jQuery TableTools).

* The gadget now reports the correct exception message in the report (which is the Exception.InnerException) if something unexpected happens inside your testcase.

If you come across any other issues, please let me know!

Guest
(By Guest, 9/21/2010 12:32:55 PM)

Great work! I love it
/ flip

Guest
(By Guest, 9/21/2010 12:32:55 PM)

This is exactly what I need!

I'm looking forward to install it and check it out..

Great job Jonas!
/ Stefan

jonas.persson
(By jonas.persson, 9/21/2010 12:32:55 PM)

To increase compability with other testing frameworks I've added another feature for this. Testcases written for NUnit/VisualStudio should now be compatible to present and execute in the gadget.

To achieve this, I changed the testexecution so that a testmethod does no longer need to return a TestResult object, which means that you're able to use NUnit/Microsofts Assert.xxxx methods instead. (You can however still return such an object if you like this approach).
If this doesnt make sense to you, just take a look at the example testcase project above.

Happy testing!

Dmytro.Duk
(By Dmytro.Duk, 9/21/2010 12:32:55 PM)

I think this is the best in GC 2009.

Guest
(By Guest, 9/21/2010 12:32:55 PM)

One word: Fantastic!
/ Tom Stenius

Guest
(By Guest, 9/21/2010 12:32:55 PM)

If I had a vote it would be yours.
/ Joel Abrahamsson

Please login to comment.