Automating Selenium tests
This post explains how to setup automated Selenium tests on an EPiServer website.
Beyond the scope of this post:
- Installation and configuration of the build server itself. I’m using Team Foundation Server but I’m sure any build server will work.
- How Selenium works in any great detail. I will only provide a few examples how to login to EPiServer and verify content.
What is Selenium and why should you care?
Selenium is a set of tools for testing web applications by recording all clicks, keystrokes and what-not, and replaying them and verify that certain elements contain certain values.
Key components include:
- Selenium IDE
- A plugin for Firefox allowing you to record, edit and play back tests. It can export the test you have recorded to a C# class.
- You don’t have to use this, you can code your tests from scratch as well.
- Selenium Remote Control (RC)
- A java server which starts up browsers and run commands you pass along from your tests.
- Listens to a TCP port and receives commands from the Client Drivers.
- I’m running this on the build server.
- Selenium Client Drivers
- These are language specific drivers used for wrapping calls to Selenium RC.
This is awesome because it will allow you to remote control a browser from within a C# class, used in a Visual Studio Test project which your Build Server will run in your Continuous Integration environment. How cool is that? This will ensure that not only does your code compile, but you can automate the whole process of clicking around the site after the build to verify that everything still seem to work.
Requirements
- Java SE 6 JRE on the server running Selenium RC
- Firefox on the same server running RC
- Selenium RC (http://seleniumhq.org/download/)
- Selenium IDE (http://seleniumhq.org/download/)
- Some build server, obviously.
1. Install Selenium RC on the Build Server
Download and unzip selenium-remote-control-1.0.3.zip.
It contains the client drivers you need in your Visual Studio project, and the Remote Control server you will run on your build server.
Put selenium-server.jar somewhere. Fire up a cmd.exe and run “java –jar selenium-server.jar”. With default settings the server will listen to TCP 4444 on all networks. You probably want to wrap this into a Windows Service.
2. Install Firefox on the server
Go get Firefox (http://www.getfirefox.com) and make sure the path to Firefox.exe is included in the environment variable PATH, or the RC will not find it.
3. Install Selenium IDE
Install the plugin in Firefox on your workstation.
4. Create a new test case and start recording traffic
Fire up Selenium IDE and go to your favorite site. Hit the red record button in the top right corner to start recording. In the example screenshot I have recorded the steps involving login to EPiServer, a few clicks on the site, and I have added a few assertions about the pages.
The IDE will add a right-click menu to ease the adding of assertions. Just right-click some element and it will give you the option to assert that a certain text is displayed “anywhere” on the page (verifyTextPresent) or a text has to be in a certain xpath(assertText).
A very good idea is to add tests for the Edit and Admin modes to catch errors with custom properties modified, plugins crashing etc.
5. Export the test to a C# class
Export the test case to a C# class and you will have a TestFixture class ready to use in a Visual Studio Test project. You will need to add references to the NUnit and Selenium dll:s that came with the RC zip.
This is key:
This will tell Selenium to send your commands to the RC server running at 192.168.0.123 on port 4444.
Make sure you add a sufficient timeout using the setTimeout() method which is used for all Open calls. Default is 30 seconds which probably isn’t enough since we’re hitting a brand new compiled EPiServer site.
If you are running the TFS Build Server you probably want to replace the NUnit attributes with VS attributes, or make some magic to the Build Definition in order to make the build server run your tests.
6. Run the unit test
In Visual Studio, use the Run Unit Tests command to verify that the test passes. Watch the RC command window on the build server as it starts up Firefox and step-by-step runs your test.
7. Pretty much done!
If you have a build server automatically executing test dll:s, this should work without problems. Just make sure the RC server is running.
I had some issues with timeouts because the site was too slow to start. Another totally unrelated problem was that the TFS Build Server refused to build my EPiServer 6 .NET 4 project, claiming that it was missing a “resgen.exe” executable included in Windows SDK. Some googling hinted me to install Visual Studio on the build server. No way, Jose! Absolutely not, never. Turns out the Windows SDK 7.0A included in VS 2010 hasn’t been released for download separately, so I simply copied the whole SDK folder and registry keys to the build server. Worked like a charm!
I have some ideas for integrating screenshots of failing tests into the build reports, but that’s a subject for another blog post!
Nice! I've done something similar with CruiseControl.NET, and it is very powerful stuff. Have you looked into either starting Selenium manually from the build script in some way, or running it as a service? There are some blog posts about running Selenium as a service, but they all seem kind of shaky to me (or I've been reading the wrong ones.)
I've probably been reading the same blog posts as you have, because I didn't dare install any of those "service wrappers". I don't want to but if I can't find anything useful, I'll code a wrapper myself.
We're running multiple build agents that potentially could build and test projects at the same time, so starting Selenium from the build script would not work unless you configure each project to run Selenium on dedicated TCP ports.. But that seems like a huge hassle.
I know exactly what you mean. I've been thinking the same myself. I'm sure someone must have done such a wrapper already, something we perhaps could build upon? I'm also looking into making a CCNet task that does UI testing based on an input file (csv), using NUnit to run it (which helps in reporting).
An admin plugin to EPiServer is also in the works, to generate a list of urls to do simple smoke testing from, making sure we have at least one page for all page types. It could help catch UI side effects early.