Testing Web Applications using Selenium and Selenium IDE

Ted Ruegsegger, CSC
6 April 2006

Contents

1. About this document
2. What is Selenium?
3. What is Selenium IDE?
4. Test Architecture
5. Application Architecture
6. Building Tests
7. Running Tests
8. References
9. Contacting the Author

About this document

I have only just become aware of Selenium's existence and, together with my colleagues, I'm feeling my way as I go along. Since so many of us develop web applications or web sites and could use some automated testing support, I maintain this document to share what I learn as I learn it. If you know things that aren't in here, please pass them on to me and I'll update. Same goes for any errors or omissions or anything that isn't as clear or complete as it could be.

Disclaimer: I'm no professional tester and I don't know any more, perhaps less, than the average developer about software testing theory. If you are, you'll probably note some howlers—please let me know!

What is Selenium?

Selenium is one of several products of OpenQA, a community dedicated to developing quality Free ("open source") QA tools. The Selenium web page says "Selenium is a test tool for web applications. Selenium tests run directly in a browser, just as real users do."

There are several ways to run Selenium; this document addresses "TestRunner mode" where the Selenium code, the tests, and the application to be tested all reside under the web server's document root. The tester opens Selenium's URL in a browser with an argument pointing to the tests. Selenium reads the tests and sends Javascript to the browser to execute them. This works with any browser that can run Javascript.

What is Selenium IDE?

Selenium IDE is a FireFox extension that records mouse and keyboard actions as Selenium commands we can save as test scripts, saving labor and reducing mistakes.

As the name implies, it's an integrated development environment that can also play back the recorded actions (from within its own folder on the client). It doesn't need or use the Selenium code that resides on the server, but the test script formats are the same and the tests are playable on the server.

Why not use Selenium IDE alone? For one thing, it works only with FireFox, version 1.5 or later. If we run the tests on the server, we can test our application using any browser.

Test Architecture

In TestRunner mode, as documented in the TestRunner Reference, we execute a test suite that contains one or more test cases.

Individual test cases

Each test is an HTML page that contains a table with rows of three columns. Selenium ignores the first row; this is a good place to put titles and explanatory text for us humans. Each of the remaining rows contains a test command and arguments.

The Selenium Reference documents the commands and their arguments

Test suites

Each suite is an HTML page that contains a table with a single column. Again, Selenium ignores the first row, so use this for titles and explanatory text. Each of the remaining rows contains a link to an individual test.

Selenium executes the tests in order or, at the operator's option, individually as selected.

Application Architecture

The procedures below make certain assumptions about the layout and management of the web applications under test, so I'll summarize those here. Adjust to suit your own project's style.

Generally, our web applications are based on the PHP Web Kit. The primary programming language is PHP, interacting with the PostgreSQL database management system and the Apache webserver. The platform is GNU/Linux (Debian stable for production, Kubuntu for development).

We can run multiple environments at the same time, each in its own named subfolder of the service root (the service root is either the document root or a named subfolder therein), for example:

	/				Document root
		myapp/			Service root (possibly containing production environment)
			devl/		Developer's individual environment
			devn/		Developer's individual environment
			devt/		Developer's individual environment
			tst/		Test environment

Each environment has its own PostgreSQL database, independent of the others. Each database name consists of a base name for the application (e.g., myapp or simply my or ma) followed by an underscore and the environment identifier, for example: ma_devt. The production database uses just the base name.

We maintain all project work products under a version-control system, currently CVS, organized as folders. Each developer checks out an individual copy of the project, for example:

	projects
		myapp
			design
			doc
			reqts
			scripts			Setup scripts (typically sh and SQL) common to all apps
				local		Setup scripts peculiar to this app
			web			PHP code for the app
				tests		Selenium test suites and test cases

The first thing each developer does after checking out the project is move the contents of the web folder (PHP code and tests) to the developer's environment under the service root, and then replace the now-empty web folder with a link (still named web) pointing to the environment.

Building Tests

Every developer is responsible for adding tests as new features are added, and maintaining the appropriate tests to keep track of changes. Where practical, associate tests with explicit requirements, and vice-versa.

• As far as possible, record individual test cases with Selenium IDE and touch them up as needed, for example, to replace environment-specific references with general ones.

• Save the test cases in HTML format (not Ruby) under web/tests/ and be sure to add new files to the CVS tree.

• In special cases, a test case may be a PHP script that generates HTML; this is how I detect the environment and open the appropriate URL.

• Edit the test suites, adding a one-cell row containing a link to each desired test case.

• As with the test cases, keep the test suites under web/tests/ and manage them under CVS.

• Testing requires that the database be in a known state; ideally, we can set up a single data set containing raw material for all tests. The script to load this data into a specified database is scripts/local/resettestdb.sh.

  

  If you need to modify this data, that's the file to edit, but be sure to notify the rest of the team and be sure you don't break other tests.

Running Tests

Besides running the specific tests for the task at hand, every developer should run the "All Tests" test suite frequently as a regression test, to ensure the changes didn't break something that was working before.

• On the development server, set up a separate test environment. If the server is your own personal development box, call it tst; if several developers share the server, make your own personal test environment, for example, tstl, tstn, tstt, etc. Do this by creating a subfolder of the service root and then checking out only the web subtree from CVS, for example:

  cd /home/web/webroot-ssl/myapp
  cvs co myapp/web -d tst

When you have completed some work and are ready to test:

1. Commit the code to be tested (and, of course, the tests) into CVS.

2. Update the tst environment from CVS.

3. Install the test data in the test database, for example,

   scripts/local/resettestdb.sh ma_tst

4. In any browser, run the Selenium TestRunner, pointing at a test suite in the desired environment, for example:

   https://server/selenium/TestRunner.html?test=/myapps/tst/tests/AllTests.html

   where server is the network address of the server; if you're running the browser and the server on the same machine, use localhost.

5. You'll see a page split into frames displaying the test suite, the individual tests, the Selenium controls, and what the browser sees as it runs the tests.

6. Use the controls to select individual test cases or run all in sequence, either a step at a time ("Step"), continuously ("Walk"), or at maximum speed ("Run").

References

I'll add some notes about XPath, since that'll probably come in handy. So far I've found an XPath tutorial at http://www.w3schools.com/xpath/default.asp and the language spec at http://www.w3.org/TR/xpath, but I haven't yet looked either of them over; comments welcome.

Selenium home page at OpenQA.org

http://www.openqa.org/selenium/

Selenium IDE home page at OpenQA.org

http://www.openqa.org/selenium-ide/
The easiest way to install Selenium IDE is from the FireFox extensions repository (currently at https://addons.mozilla.org/firefox/2079/) but note that it requires FireFox version 1.5.

Contacting the Author

I'm grateful for comments and advice sent to

Ted Ruegsegger