So ware Testing on the Web

Martin Schtte

So ware Quality . .

Documentation . . . . .

Unit Testing . . . . . . . . . . . . .

Web Drivers . . . . . . . . . .

Behaviour Testing . . . . . . . . . . . . . . . . .

Conclusion . .

Outline
So ware Quality Documentation Unit Testing PHPUnit Continuity Web Drivers Behaviour Testing Meta PHP Behat and Ruby Cucumber Codeception

Martin Schtte

So ware Testing on the Web

2012-10-13

2 / 51

So ware Quality . .

Documentation . . . . .

Unit Testing . . . . . . . . . . . . .

Web Drivers . . . . . . . . . .

Behaviour Testing . . . . . . . . . . . . . . . . .

Conclusion . .

Main Questions

How can I know (my) so ware is correct? How does my boss know so ware is correct? How can we discuss what correct is, anyway? We always need:
implicit assumptions, explicit specifications.

Martin Schtte

So ware Testing on the Web

2012-10-13

3 / 51

So ware Quality . .

Documentation . . . . .

Unit Testing . . . . . . . . . . . . .

Web Drivers . . . . . . . . . .

Behaviour Testing . . . . . . . . . . . . . . . . .

Conclusion . .

Walking on water and developing so ware from a specification are easy if both are frozen. Edward V. Berard

Lets just update PHP Lets just change the DB schema Lets just add feature X Is the so ware still correct?

Martin Schtte

So ware Testing on the Web

2012-10-13

4 / 51

So ware Quality . .

Documentation . . . . .

Unit Testing . . . . . . . . . . . . .

Web Drivers . . . . . . . . . .

Behaviour Testing . . . . . . . . . . . . . . . . .

Conclusion . .

Maintainability requires Documentation

Consistency of: 1. Code 2. Comments 3. Documentation

When code and comments disagree, both are probably wrong. Norm Schryer

Martin Schtte

So ware Testing on the Web

2012-10-13

5 / 51

So ware Quality . .

Documentation . . . . .

Unit Testing . . . . . . . . . . . . .

Web Drivers . . . . . . . . . .

Behaviour Testing . . . . . . . . . . . . . . . . .

Conclusion . .

Levels of Documentation
. abstract UI External API Subsystems

Libraries, Functions

concrete

Martin Schtte

So ware Testing on the Web

2012-10-13

6 / 51

So ware Quality . .

Documentation . . . . .

Unit Testing . . . . . . . . . . . . .

Web Drivers . . . . . . . . . .

Behaviour Testing . . . . . . . . . . . . . . . . .

Conclusion . .

PHPdoc/JavaDoc/Doxygen etc.

/** * Gets single server parameter for specified key. * * @param string $key A key of the parameter to get * @param string $default A default value when key is undefined * * @return string A value of the parameter */ public function getServerParameter($key, $default = ) { return (isset($this->server[$key])) ? $this->server[$key] : $default; }

Martin Schtte

So ware Testing on the Web

2012-10-13

7 / 51

So ware Quality . .

Documentation . . . . .

Unit Testing . . . . . . . . . . . . .

Web Drivers . . . . . . . . . .

Behaviour Testing . . . . . . . . . . . . . . . . .

Conclusion . .

Low Level Documentation

Martin Schtte

So ware Testing on the Web

2012-10-13

8 / 51

So ware Quality . .

Documentation . . . . .

Unit Testing . . . . . . . . . . . . .

Web Drivers . . . . . . . . . .

Behaviour Testing . . . . . . . . . . . . . . . . .

Conclusion . .

High Level Documentation

Martin Schtte

So ware Testing on the Web

2012-10-13

9 / 51

So ware Quality . .

Documentation . . . . .

Unit Testing . . . . . . . . . . . . .

Web Drivers . . . . . . . . . .

Behaviour Testing . . . . . . . . . . . . . . . . .

Conclusion . .

Levels of Testing
. abstract Acceptance Tests

System Tests

Unit Tests

concrete

Martin Schtte

So ware Testing on the Web

2012-10-13

10 / 51

So ware Quality . .

Documentation . . . . .

Unit Testing . . . . . . . . . . . . .

Web Drivers . . . . . . . . . .

Behaviour Testing . . . . . . . . . . . . . . . . .

Conclusion . .

Regression Testing
Bug Tracker = Documentation

Martin Schtte

So ware Testing on the Web

2012-10-13

11 / 51

So ware Quality . .

Documentation . . . . .

Unit Testing . . . . . . . . . . . . .

Web Drivers . . . . . . . . . .

Behaviour Testing . . . . . . . . . . . . . . . . .

Conclusion . .

PHPUnit
class DTABaseTest extends PHPUnit_Framework_TestCase { protected function setUp() { $this->fixture = new DTABase(); } public function testGetNumTooShort() { $input = 12345; $off = 3; $len = 4; $rc = $this->fixture->getNum($input, $off, $len); $this->assertEquals(45, $rc); } // ... }
Martin Schtte So ware Testing on the Web 2012-10-13 12 / 51

So ware Quality . .

Documentation . . . . .

Unit Testing . . . . . . . . . . . . .

Web Drivers . . . . . . . . . .

Behaviour Testing . . . . . . . . . . . . . . . . .

Conclusion . .

PHPUnit
~/Payment_DTA> phpunit tests/DTABaseTest.php PHPUnit 3.6.11 by Sebastian Bergmann. F.... Time: 0 seconds, Memory: 3.75Mb There was 1 failure: 1) DTABaseTest::testGetNumTooShort Failed asserting that 45.001 matches expected 45. /usr/home/mschuett/Payment_DTA/tests/DTABaseTest.php:50 /usr/local/bin/phpunit:46 FAILURES! Tests: 5, Assertions: 5, Failures: 1.
Martin Schtte So ware Testing on the Web 2012-10-13 13 / 51

So ware Quality . .

Documentation . . . . .

Unit Testing . . . . . . . . . . . . .

Web Drivers . . . . . . . . . .

Behaviour Testing . . . . . . . . . . . . . . . . .

Conclusion . .

Tests are Documentation

~/Payment_DTA> phpunit --testdox tests/DTABaseTest.php PHPUnit 3.6.11 by Sebastian Bergmann. DTABase [ ] Get [x] Get [x] Get [x] Get [x] Get

num num str str str

too short offset too big too short offset too big invalid

Martin Schtte

So ware Testing on the Web

2012-10-13

14 / 51

So ware Quality . .

Documentation . . . . .

Unit Testing . . . . . . . . . . . . .

Web Drivers . . . . . . . . . .

Behaviour Testing . . . . . . . . . . . . . . . . .

Conclusion . .

WordPress Unit Tests

.
http://unit-tests.trac.wordpress.org/
vagrant@wp:~/trunk$ phpunit Running as single site... To run multisite, use -c multisite.xml Not running ajax tests... To execute these, use --group ajax. PHPUnit 3.6.12 by Sebastian Bergmann. Configuration read from /home/vagrant/trunk/phpunit.xml ................F....................................SS.....S S...SSSSSS.............S..SSS..........SS................S... .......SS.SSSS...............S.................S............. ...............................S............S................ .S........................S.................S.......I........ .........................SSSS.S..SSS......................... ............................................................. ............................................................. ............................................................. ............................................................. ..................................................S.......... .......SSSSSSSSSSSSSS...........SS........................... .............SS...............SSSSSSSSSSSSSSSSSS............. ................SSS...S.SSSSSSS................SSSSSSS....... .............S..SS....FFF....SSSS.SS......................SSS ............................................................. ......S..................................................S..S ................SS.S.........................S............... ...............S..................S.......................... ............................................................. .......................F..............................S...... ............................................................. .............................................SS. Time: 08:56, Memory: 122.75Mb [...] FAILURES! Tests: 1390, Assertions: 5640, Failures: 5, Incomplete: 1, Skipped: 117. 61 122 183 244 305 366 427 488 549 610 671 732 793 854 915 976 1037 1098 1159 1220 1281 1342 / / / / / / / / / / / / / / / / / / / / / / 1437 1437 1437 1437 1437 1437 1437 1437 1437 1437 1437 1437 1437 1437 1437 1437 1437 1437 1437 1437 1437 1437 ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( 4%) 8%) 12%) 16%) 21%) 25%) 29%) 33%) 38%) 42%) 46%) 50%) 55%) 59%) 63%) 67%) 72%) 76%) 80%) 84%) 89%) 93%)

Martin Schtte

So ware Testing on the Web

2012-10-13

15 / 51

So ware Quality . .

Documentation . . . . .

Unit Testing . . . . . . . . . . . . .

Web Drivers . . . . . . . . . .

Behaviour Testing . . . . . . . . . . . . . . . . .

Conclusion . .

PHPUnit Code Coverage

Martin Schtte

So ware Testing on the Web

2012-10-13

16 / 51

So ware Quality . .

Documentation . . . . .

Unit Testing . . . . . . . . . . . . .

Web Drivers . . . . . . . . . .

Behaviour Testing . . . . . . . . . . . . . . . . .

Conclusion . .

Things to build upon Automated Testing

git bisect run phpunit:

find commit that introduces a bug Continuous Integration:

frequent check-ins to mainline branch verified by automated builds and tests

Continuous Deployment: automated deployment to production useful: automated rollback

Martin Schtte

So ware Testing on the Web

2012-10-13

17 / 51

So ware Quality . .

Documentation . . . . .

Unit Testing . . . . . . . . . . . . .

Web Drivers . . . . . . . . . .

Behaviour Testing . . . . . . . . . . . . . . . . .

Conclusion . .

Example: Jenkins CI

Martin Schtte

So ware Testing on the Web

2012-10-13

18 / 51

So ware Quality . .

Documentation . . . . .

Unit Testing . . . . . . . . . . . . .

Web Drivers . . . . . . . . . .

Behaviour Testing . . . . . . . . . . . . . . . . .

Conclusion . .

Example: Travis CI for GitHub

Martin Schtte

So ware Testing on the Web

2012-10-13

19 / 51

So ware Quality . .

Documentation . . . . .

Unit Testing . . . . . . . . . . . . .

Web Drivers . . . . . . . . . .

Behaviour Testing . . . . . . . . . . . . . . . . .

Conclusion . .

Approaches to Unit Testing

Exploratory testing (e. g. for 3rd party libs) Regression testing (for found & fixed bugs) Spike and Stabilise testing (a er code, before refactoring) Test Driven Development (TDD): test first, then code

Martin Schtte

So ware Testing on the Web

2012-10-13

20 / 51

So ware Quality . .

Documentation . . . . .

Unit Testing . . . . . . . . . . . . .

Web Drivers . . . . . . . . . .

Behaviour Testing . . . . . . . . . . . . . . . . .

Conclusion . .

Program testing can be used to show the presence of bugs, but never to show their absence! Edsger Dijkstra, Notes on Structured Programming

Testing by itself does not improve so ware quality. Test results are an indicator of quality, but in and of themselves, they dont improve it. [] If you want to improve your so ware, dont test more; develop better. Steve McConnell, Code Complete

Martin Schtte

So ware Testing on the Web

2012-10-13

21 / 51

So ware Quality . .

Documentation . . . . .

Unit Testing . . . . . . . . . . . . .

Web Drivers . . . . . . . . . .

Behaviour Testing . . . . . . . . . . . . . . . . .

Conclusion . .

Cost of Testing
Testing (like Documentation) has a cost o en: productivity improvement > cost but ROI depends on type of project/so ware

Martin Schtte

So ware Testing on the Web

2012-10-13

22 / 51

So ware Quality . .

Documentation . . . . .

Unit Testing . . . . . . . . . . . . .

Web Drivers . . . . . . . . . .

Behaviour Testing . . . . . . . . . . . . . . . . .

Conclusion . .

HTTP API
class TumblrAPITest extends PHPUnit_Framework_TestCase { public function testPostDatetime() { $url = http://staff.tumblr.com/api/read; $ch = curl_init($url); curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1); $xmlsrc = curl_exec($ch); $xmlobj = new SimpleXMLElement($xmlsrc); $str_date = date_timestamp_get(date_create( $xmlobj->posts->post[0][date-gmt])); $unix_date = intval( $xmlobj->posts->post[0][unix-timestamp]); $this->assertEquals($str_date, $unix_date); }
Martin Schtte

So ware Testing on the Web

2012-10-13

23 / 51

So ware Quality . .

Documentation . . . . .

Unit Testing . . . . . . . . . . . . .

Web Drivers . . . . . . . . . .

Behaviour Testing . . . . . . . . . . . . . . . . .

Conclusion . .

HTML-Content
class WpCampTest extends PHPUnit_Framework_TestCase { public function testSiteTitle() { $url = http://wpcamp.de/; $ch = curl_init($url); curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1); $xmlsrc = curl_exec($ch); $dom $title = @DOMDocument::loadHTML($xmlsrc); = $dom->getElementsByTagName(title) ->item(0)->textContent;

$this->assertContains(Das WordPress Event, $title); } }

.
Martin Schtte So ware Testing on the Web 2012-10-13 24 / 51

So ware Quality . .

Documentation . . . . .

Unit Testing . . . . . . . . . . . . .

Web Drivers . . . . . . . . . .

Behaviour Testing . . . . . . . . . . . . . . . . .

Conclusion . .

Goutte
a simple PHP Web Scraper

class WpCampTest extends PHPUnit_Framework_TestCase { public function testSiteTitle() { $client = new Client(); $crawler = $client ->request(GET, http://wpcamp.de/); $title = $crawler ->filter(html head title) ->first()->text(); $this->assertContains(Das WordPress Event, $title); } }
Martin Schtte So ware Testing on the Web 2012-10-13 25 / 51

So ware Quality . .

Documentation . . . . .

Unit Testing . . . . . . . . . . . . .

Web Drivers . . . . . . . . . .

Behaviour Testing . . . . . . . . . . . . . . . . .

Conclusion . .

Selenium
a suite of tools to automate web browsers across platforms

class WpTest extends PHPUnit_Extensions_SeleniumTestCase { public function setUp() { $this->setHost(localhost); $this->setBrowser(firefox); $this->setBrowserUrl(http://wpcamp.de/); } public function testSiteTitle() { $this->open(/); $this->assertText(//html/head/title, Das WordPress Event *); }
Martin Schtte So ware Testing on the Web 2012-10-13 26 / 51

So ware Quality . .

Documentation . . . . .

Unit Testing . . . . . . . . . . . . .

Web Drivers . . . . . . . . . .

Behaviour Testing . . . . . . . . . . . . . . . . .

Conclusion . .

Selenium, contd.
public function testSearch() { $this->open(/); $this->type(id=s, sponsoren); $this->click(id=searchsubmit); $this->waitForPageToLoad(30000); $this->assertText(css=#content, regexp:.*Premium-Sponsor Chocri.*); $this->assertText(css=#content, regexp:.*Platin-Sponsor Adobe.*); } }

Martin Schtte

So ware Testing on the Web

2012-10-13

27 / 51

So ware Quality . .

Documentation . . . . .

Unit Testing . . . . . . . . . . . . .

Web Drivers . . . . . . . . . .

Behaviour Testing . . . . . . . . . . . . . . . . .

Conclusion . .

Behat Mink
an acceptance test framework for web applications

class WpCampTest extends PHPUnit_Framework_TestCase { public function setUp() { $driver = new \Behat\Mink\Driver\GoutteDriver(); $this->session = new \Behat\Mink\Session($driver); } public function testSiteTitle() { $this->session->visit(http://wpcamp.de/); $title = $this->session->getPage() ->find(css, html head title)->getText(); $this->assertContains(Das WordPress Event, $title); }
Martin Schtte So ware Testing on the Web 2012-10-13 28 / 51

So ware Quality . .

Documentation . . . . .

Unit Testing . . . . . . . . . . . . .

Web Drivers . . . . . . . . . .

Behaviour Testing . . . . . . . . . . . . . . . . .

Conclusion . .

Behat Mink, contd.

public function testSearch() { $this->session->visit(http://wpcamp.de/); $page = $this->session->getPage(); $page->find(css, #s)->setValue(Sponsoren); $page->find(css, #searchsubmit)->press(); $text = $this->session->getPage() ->find(css, #content)->getText(); $this->assertContains( Premium-Sponsor Chocri, $text); $this->assertContains( Platin-Sponsor Adobe, $text); } }
Martin Schtte So ware Testing on the Web 2012-10-13 29 / 51

So ware Quality . .

Documentation . . . . .

Unit Testing . . . . . . . . . . . . .

Web Drivers . . . . . . . . . .

Behaviour Testing . . . . . . . . . . . . . . . . .

Conclusion . .

The Big Picture

PHPUnit . Mink Selenium Firefox Sahi Opera Goutte Zombie.js

Martin Schtte

So ware Testing on the Web

2012-10-13

30 / 51

So ware Quality . .

Documentation . . . . .

Unit Testing . . . . . . . . . . . . .

Web Drivers . . . . . . . . . .

Behaviour Testing . . . . . . . . . . . . . . . . .

Conclusion . .

Problems with Web Drivers

Dierences in:
web functionality (HTML, CSS, JS, AJAX)

performance
selectors (ID, CSS, XPath, DOM) test functionality and integration with test framework

Recommendation: Select two Drivers, 1. simple and fast one for unit testing and 2. Selenium/Sahi for integration and UX testing.

Martin Schtte

So ware Testing on the Web

2012-10-13

31 / 51

So ware Quality . .

Documentation . . . . .

Unit Testing . . . . . . . . . . . . .

Web Drivers . . . . . . . . . .

Behaviour Testing . . . . . . . . . . . . . . . . .

Conclusion . .

Testing in Production = Monitoring

Example: check webshop availability Possible layers:

ping server check HTTP status 200 OK check homepage title login as user complete order process

Martin Schtte

So ware Testing on the Web

2012-10-13

32 / 51

So ware Quality . .

Documentation . . . . .

Unit Testing . . . . . . . . . . . . .

Web Drivers . . . . . . . . . .

Behaviour Testing . . . . . . . . . . . . . . . . .

Conclusion . .

The next level: Behaviour Tests

Funktionalitt: Selbstprsentation Um etwas ber das WPCamp zu erfahren Als Besucher Mchte ich Infos auf der Website finden Szenario: Sponsoren suchen Angenommen ich bin auf / Und ich suche nach sponsoren Dann ich sollte Premium-Sponsor Chocri sehen Und ich sollte Platin-Sponsor Adobe sehen

Martin Schtte

So ware Testing on the Web

2012-10-13

33 / 51

So ware Quality . .

Documentation . . . . .

Unit Testing . . . . . . . . . . . . .

Web Drivers . . . . . . . . . .

Behaviour Testing . . . . . . . . . . . . . . . . .

Conclusion . .

Next Level: Behaviour Driven Development

TDD with other syntax?

Is BDD the same as TDD? Yes. If youre a programmer, and your entire team is programmers, and all your stakeholders are programmers. Dan North

We dont call them acceptance tests because you cant ask a business person Please help me with my acceptance test. Try Id like to talk to you about the scenario where instead. Or, Can you give me an example? Either of these are good. Liz Keogh
Martin Schtte So ware Testing on the Web 2012-10-13 34 / 51

So ware Quality . .

Documentation . . . . .

Unit Testing . . . . . . . . . . . . .

Web Drivers . . . . . . . . . .

Behaviour Testing . . . . . . . . . . . . . . . . .

Conclusion . .

Unit & Behaviour Testing

Unit Tests
unit testing programmers programming language bottom-up assertXYZ tests derived from user stories

Behaviour Tests
acceptance test scenarios non-developers language of business domain top-down / outside-in X should do Y execute user stories

development tool

communication tool

Martin Schtte

So ware Testing on the Web

2012-10-13

35 / 51

So ware Quality . .

Documentation . . . . .

Unit Testing . . . . . . . . . . . . .

Web Drivers . . . . . . . . . .

Behaviour Testing . . . . . . . . . . . . . . . . .

Conclusion . .

One Notation/Language: Gherkin

Feature: [one line describing the story] In order to [benefit] As a [role] I want [feature]

Scenario: [A scenario specific goal] Given [Something that needs to have happened or be true] When [Some task I must do] And [Another task] Then [Some way I know Ive achieved my goal]

Martin Schtte

So ware Testing on the Web

2012-10-13

36 / 51

So ware Quality . .

Documentation . . . . .

Unit Testing . . . . . . . . . . . . .

Web Drivers . . . . . . . . . .

Behaviour Testing . . . . . . . . . . . . . . . . .

Conclusion . .

I18N
# language: de Funktionalitt: [one line describing the story] Um zu erreichen [benefit] Als [role] Mchte ich [feature] Szenario: [Title] Angenommen [context] Wenn [event] Dann [outcome] Und [another outcome]

Martin Schtte

So ware Testing on the Web

2012-10-13

37 / 51

So ware Quality . .

Documentation . . . . .

Unit Testing . . . . . . . . . . . . .

Web Drivers . . . . . . . . . .

Behaviour Testing . . . . . . . . . . . . . . . . .

Conclusion . .

TDD with other syntax?

Scenario: New lists are empty Given a new list Then the list should be empty. Scenario: Lists with things in them are not empty. Given a new list When we add an object Then the list should not be empty.

Martin Schtte

So ware Testing on the Web

2012-10-13

38 / 51

So ware Quality . .

Documentation . . . . .

Unit Testing . . . . . . . . . . . . .

Web Drivers . . . . . . . . . .

Behaviour Testing . . . . . . . . . . . . . . . . .

Conclusion . .

Implementation / Translation
A look behind the curtain:
framework is clever but not magical some translation needed statements have to become executable code

Mechanism:
plain sentence method name quoted words arguments matching with annotated RegExp methods yield success, exception, or pending exception

Martin Schtte

So ware Testing on the Web

2012-10-13

39 / 51

So ware Quality . .

Documentation . . . . .

Unit Testing . . . . . . . . . . . . .

Web Drivers . . . . . . . . . .

Behaviour Testing . . . . . . . . . . . . . . . . .

Conclusion . .

Behat/Mink FeatureContext.php
use Behat\MinkExtension\Context\MinkContext; class FeatureContext extends BehatContext { public function __construct(array $parameters) { $this->useContext(mink, new MinkContext()); } }

MinkContext adds web-related methods and their I18N translations, e. g.:

visit($page) fillField($field, $value) pressButton($button)
Martin Schtte So ware Testing on the Web 2012-10-13 40 / 51

So ware Quality . .

Documentation . . . . .

Unit Testing . . . . . . . . . . . . .

Web Drivers . . . . . . . . . .

Behaviour Testing . . . . . . . . . . . . . . . . .

Conclusion . .

Behat Output
[mschuett@dagny] ~/wpcamp/code% ./vendor/bin/behat Funktionalitt: Selbstprsentation Um etwas ber das WPCamp zu erfahren Als Besucher Mchte ich Infos auf der Website finden Szenario: Sponsoren suchen Angenommen ich bin auf / Und ich suche nach sponsoren Dann ich sollte Premium-Sponsor Chocri sehen Und ich sollte Platin-Sponsor Adobe sehen 1 scenario (1 undefined) 4 steps (1 passed, 2 skipped, 1 undefined) 0m1.574s You can implement step definitions for undefined steps with these snippets: /** * @Given /^ich suche nach ([^]*)$/ */ public function ichSucheNach($arg1) { throw new PendingException(); }

# features/wpcamp.feature:8 # Behat\MinkExtension\Context\MinkContext::visit

# Behat\MinkExtension\Context\MinkContext::asser # Behat\MinkExtension\Context\MinkContext::asser

Martin Schtte

So ware Testing on the Web

2012-10-13

41 / 51

So ware Quality . .

Documentation . . . . .

Unit Testing . . . . . . . . . . . . .

Web Drivers . . . . . . . . . .

Behaviour Testing . . . . . . . . . . . . . . . . .

Conclusion . .

Edit FeatureContext.php
add method

/** * @Given /^ich suche nach ([^]*)$/ */ public function ichSucheNach($arg) { return array( new When(I fill in \s\ with \$arg\), new When(I press \searchsubmit\), ); }

Martin Schtte

So ware Testing on the Web

2012-10-13

42 / 51

So ware Quality . .

Documentation . . . . .

Unit Testing . . . . . . . . . . . . .

Web Drivers . . . . . . . . . .

Behaviour Testing . . . . . . . . . . . . . . . . .

Conclusion . .

Behat Output

[mschuett@dagny] ~/wpcamp/code% ./vendor/bin/behat Funktionalitt: Selbstprsentation Um etwas ber das WPCamp zu erfahren Als Besucher Mchte ich Infos auf der Website finden Szenario: Sponsoren suchen Angenommen ich bin auf / Und ich suche nach sponsoren Dann ich sollte Premium-Sponsor Chocri sehen Und ich sollte Platin-Sponsor Adobe sehen 1 scenario (1 passed) 4 steps (4 passed) 0m2.59s # # # # #

features/wpcamp.feature:8 Behat\MinkExtension\Context\MinkContext::visit FeatureContext::ichSucheNach() Behat\MinkExtension\Context\MinkContext::asser Behat\MinkExtension\Context\MinkContext::asser

Martin Schtte

So ware Testing on the Web

2012-10-13

43 / 51

So ware Quality . .

Documentation . . . . .

Unit Testing . . . . . . . . . . . . .

Web Drivers . . . . . . . . . .

Behaviour Testing . . . . . . . . . . . . . . . . .

Conclusion . .

The Big Picture (PHP)

Behat . Mink Selenium Firefox

PHPUnit

Sahi Opera

Goutte

Zombie.js

Martin Schtte

So ware Testing on the Web

2012-10-13

44 / 51

So ware Quality . .

Documentation . . . . .

Unit Testing . . . . . . . . . . . . .

Web Drivers . . . . . . . . . .

Behaviour Testing . . . . . . . . . . . . . . . . .

Conclusion . .

Cucumber
original Ruby version, nicer syntax

# Example Webrat step definitions When /I search for (.*)/ do |query| @browser.text_field(:name, s).set(query) @browser.button(:name, searchsubmit).click end Then /I should see/ do |text| @browser.text.should =~ /#{text}/m end

Martin Schtte

So ware Testing on the Web

2012-10-13

45 / 51

So ware Quality . .

Documentation . . . . .

Unit Testing . . . . . . . . . . . . .

Web Drivers . . . . . . . . . .

Behaviour Testing . . . . . . . . . . . . . . . . .

Conclusion . .

The Big Picture (Ruby)

. Cucumber Selenium Firefox Watir Opera Webrat Celerity

Martin Schtte

So ware Testing on the Web

2012-10-13

46 / 51

So ware Quality . .

Documentation . . . . .

Unit Testing . . . . . . . . . . . . .

Web Drivers . . . . . . . . . .

Behaviour Testing . . . . . . . . . . . . . . . . .

Conclusion . .

Codeception unit test

a new full-stack testing PHP framework

$I = new CodeGuy($scenario); $I->wantTo(perform actions and see results); $I->testMethod(DateTime.getTimestamp); $date = new DateTime(2012-10-13T14:45:00+02:00); $I->executeTestedMethodOn($date); $I->seeResultIs(int); $I->seeResultEquals(1350132300);

Martin Schtte

So ware Testing on the Web

2012-10-13

47 / 51

So ware Quality . .

Documentation . . . . .

Unit Testing . . . . . . . . . . . . .

Web Drivers . . . . . . . . . .

Behaviour Testing . . . . . . . . . . . . . . . . .

Conclusion . .

Codeception acceptance test

$I = new WebGuy($scenario); $I->wantTo(find infos on the website); $I->see(Das WordPress Event, html head title); $I->fillField(s, sponsoren); $I->click(searchsubmit); $I->see(Premium-Sponsor Chocri, #content); $I->see(Platin-Sponsor Adobe, #content);

Martin Schtte

So ware Testing on the Web

2012-10-13

48 / 51

So ware Quality . .

Documentation . . . . .

Unit Testing . . . . . . . . . . . . .

Web Drivers . . . . . . . . . .

Behaviour Testing . . . . . . . . . . . . . . . . .

Conclusion . .

Codeception Output
[mschuett@dagny] ~/wpcamp/code% php codecept.phar run --steps Codeception PHP Testing Framework v1.1.4 Powered by PHPUnit 3.6.10 by Sebastian Bergmann. Suite unit started Trying to perform actions and see results (exampleCept.php) Scenario: * I test method DateTime.getTimestamp * I execute tested method on DateTime * I see result is int * I see result equals 1350132300 OK Suite functional started Suite acceptance started Trying to find infos on the website (wpcampCept.php) Scenario: * I am on page / * I see Das WordPress Event,html head title * I fill field s,sponsoren * I click searchsubmit * I see Premium-Sponsor Chocri,#content * I see Platin-Sponsor Adobe,#content OK Time: 4 seconds, Memory: 9.50Mb OK (2 tests, 5 assertions) Martin Schtte So ware Testing on the Web 2012-10-13 49 / 51

So ware Quality . .

Documentation . . . . .

Unit Testing . . . . . . . . . . . . .

Web Drivers . . . . . . . . . .

Behaviour Testing . . . . . . . . . . . . . . . . .

Conclusion . .

Problems

most Web tests require known DB state behaviour testing is slow only run locally use in-memory DB draw the line features in documentation (what) add behaviour test mere implementation (how) only unit tests

Martin Schtte

So ware Testing on the Web

2012-10-13

50 / 51

So ware Quality . .

Documentation . . . . .

Unit Testing . . . . . . . . . . . . .

Web Drivers . . . . . . . . . .

Behaviour Testing . . . . . . . . . . . . . . . . .

Conclusion . .

Summary
Value individuals and interactions over processes and tools The Agile Manifesto

Links:
PHPUnit Selenium Behat & Mink Cucumber Codeception Sebastian Bergmann (PHPUnit) Dan Norths blog (BDD) Liz Keoghs blog (BDD) Geek and Poke Not Invented Here

Martin Schtte

So ware Testing on the Web

2012-10-13

51 / 51