Automated Testing

Value of automated testing

● Automated tests function as business logic documentation by defining constraints

and inputs
● Automated tests are repeatable (for low or zero cost)
● Separation of concerns
○ Engineers can focus more on features
○ Quality engineers can more easily see the bigger picture
● Increased release confidence
○ A production release is much less stressful if engineers and product owners have
confidence it is thoroughly tested
● Easier knowledge transfer
○ Inevitably, engineers leave - a good automated codebase can lead to critical business
knowledge living in the code
Common testing paradigms

● Quality Engineers own all testing ● SWEs write no tests

● Quality Engineers write tests ● SWEs maintain tests
● Quality Engineers suggest testing ● SWEs write and maintain tests
Common problem areas

● A common theme is that engineers often code well for happy path (perfect) scenarios
● Engineers like writing features, not tests
● Common issues I have seen include:
○ Race conditions
■ Saving information in different tabs
■ Two users saving information at the same time
○ Authentication
■ Users having permissions unexpectedly
■ Sessions not requiring new logins after TTL expires
○ Performance
■ Dev / QA environments run scaled-down versions of applications
○ Regression
■ A new feature can touch code that exists, breaking older features
Software Testing Pyramid

UI Tests

Integration Tests

Unit Tests
Reasons for test distribution

● Stability
○ A UI test has more dependencies than an integration test, and so forth
● Maintainability
○ The UI of an application tends to undergo more changes
● Execution time
○ UI testing takes time - especially on a live app
● Extensibility
○ Writing more unit tests is easier than adding additional UI tests
● Value
○ A unit test can more easily held identify issues
Test Methodology

● Use manual ‘exploratory’ testing to fuel automated testing

○ Automated testing is not a direct replacement for manual testing
(except in rare situations)
● When writing tests, plan in advance - it is easier to brainstorm up
front how to make tests extensible than later on in the process
● Focus on code which has important business logic or revenue
impact
● Keep note of areas which have had issues before
Creating Testable Codebases

● Work with engineers to create testable code

○ Engineers are focused on features, not making them testable
○ A good example is adding selectors for UI elements, which can
make UI testing more robust
● Participate in feature planning
○ If a feature is hard to understand, it is also probably hard to
test