Professional Documents
Culture Documents
Selenium Web Testing - Demonstration of A Custom Framework
Selenium Web Testing - Demonstration of A Custom Framework
Aaron Shaver
2012-05-17
High level view of the framework and test suite All test logic and assertions in the test_ directories Categorized, e.g. can run "nosetests tests_smoke"
Subfolders contain individual tests Test dirs are Python packages via the init files, can be imported nose (test runner) discovers tests based on "test" regular expression
setup.cfg: nose, WebDriver, and application-specific settings Change timeouts for WebDriverWait, application task waits Adjust server URLs, usernames, passwords
util.py -- utilities module Product-neutral code: generic functions and classes Could be reused to test other web apps
shift.py (our product is called Shift) Module for all application-specific code Pulls configuration settings from setup.cfg
data.py -- another application-specific module Keeps document count assertions outside of the _test.py files Easier to maintain: doc counts change sometimes as the app improves or degrades, e.g. file formats become supported that weren't earlier
log.txt -- optional test results file Version-controlled via TortoiseHg (Mercurial) + Bitbucket, so I can see past test runs Useful if results exceed cmd.exe buffer size nose needs special syntax to pipe output of console to the log
Example of test run outside of log.txt (outputs to cmd.exe window) Tests slow by usual unit testing standards? App performs many operations on uploads (recursively extracts, indexes metadata properties, deduplicates, generates reports, etc.)
Entire test suite takes ~6 hours, easily fitting into an overnight run, so I haven't bothered to setup Selenium Grid yet
90% of tests in suite use standard setup routine App under test needs to ingest data before doing anything useful All it needs: project name, data source, total docs assertion
Example of a simple test which does not use the "standard intake workflow" Note the product "action" functions: user_login, add_proj clean_up deletes the project and closes the browser, but only if the test was successful Failed tests leave the browser window open, in case it's a new bug
Built my framework before hearing about PageObjects Plan was to make "atomic" app actions ("save a search", etc.) Not as fine-grained as PageObjects Still a robust layer of abstraction for when UI changes Ended up clumping together app actions; even less fine-grained than my own plan
Developers were nice to me: many UI elements generic across app No AJAX yet (fingers crossed) -- timing issues from Webdriver itself
UI is very good / simple / elegant... but not always Example: reuse of list_item_detail class for both <a> and <div> Have to ensure what we're clicking is a link Could do a CSS selector, but would be more fragile
Potential difficulty: can't automate Java or Silverlight uploaders Thankfully, administrator backend allows data intake Uses standard HTML form