www.praqma.

com
 Testing Microservices
Techniques for Automation
European Testing Conference 2018
Emily Bache
@emilybache

www.praqma.com
Emily Bache

Practice Lead for Test Automation

Consultant

Author of

“The Coding Dojo Handbook”

@emilybache
emily.bache@praqma.com

www.praqma.com
Pagero Online
Cloud Service

Invoice Invoice
Issuer Recipient
Some numbers…

• We were around 45 developers, in 7 teams

• We had around 60 microservices

• Our monolith was in production for nearly 10

years before we started with microservices,
3-4 years ago

www.praqma.com
Pagero Online’s architecture

www.praqma.com
A Microservice

• Provides one business

capability

• Independently Deployable

• Lightweight API

• Not too big: fits in my head Docker is used to containerise the service

www.praqma.com
Testing a Microservices
Architecture

In my experience

www.praqma.com
Test Pyramid: Mike Cohn

End-To-End tests
End-to-End exercise the whole
system
“Service Tests”

Unit
Unit tests are for
individual functions

https://www.mountaingoatsoftware.com/blog/the-forgotten-layer-of-the-test-automation-pyramid
www.praqma.com
Testing Strategy for the Monolith

Unit tests and end-to-end

GUI tests still valid for
Microservices
architecture

www.praqma.com
Testing Strategy

Testing a monolith Testing a bunch of microservices

www.praqma.com
Testing a Microservice

www.praqma.com
Microservice Tests

µSUT
• Deploy your service on localhost (and
the database if it has one) Test Case

• Test uses the public API to trigger

functionality

• Mock a response to any calls made to

other services or queues

µSDB
collaborator µS

www.praqma.com
Faster execution
Deploy in Same Process

µSUT
• In-process testing: Deploy your
service with an in-memory database Test Case Internal api

and in-memory api

• ‘internal api’ to set up test data and
query internals

µSDB
collaborator µS

www.praqma.com
More production-like
Deploy in Different
Processes

µSUT
• Out-of-process testing: Deploy your
service in one process, and the test Test Case
case in another
• Security needed for ‘internal api’ so
only test/non-production code can
access
• Easier to performance test

µSDB
collaborator µS

www.praqma.com
Business-Facing Feature
Tests

Test a whole scenario across several microservices

www.praqma.com
Testing a whole Feature

Test Case

This is where it starts getting really complicated…

www.praqma.com
Example End-to-End Test Strategy

• Test crucial workflows, not every detail

• Use APIs instead of the GUI for most tests

• Techniques:
Keep testing costs
as low as possible
• Selective Deployment

• Approval Testing

• Event Monitoring

www.praqma.com
 Service-layer tests in a monolith

Monolith
• Test cases are for
different business-
Test Case A
facing workflows

Test Case B
• Will exercise different
components within the
monolith

Database

www.praqma.com
 Feature tests in microservices

Microservices • Test cases are for

different business-facing
Test Case A workflows

• Will exercise different

Test Case B groups of microservices

• Need not deploy

everything for every test
case

www.praqma.com
 Selective Deployment

• Need not deploy

everything for every test
Test Case A case

• But: all test cases

needed a few essential
infrastructure services.

www.praqma.com
Pagero Online
Cloud Service

Invoice Invoice
Issuer Recipient
Parts of a test case

Invoice Document Invoice Recipient

Test Workflow
Issuer Recipient Presentation

Arrange Act Assert

(Almost) all the tests involve these elements

www.praqma.com
Approval Testing

Invoice Document Invoice Recipient

Test Workflow
Issuer Recipient Presentation

Arrange Act Assert

Approval Testing specifies the Assert: compare against an ‘Approved’

result

www.praqma.com
The “Approved” Result as Text

Recipient
Recipient Presentation
Presentation in plain text

pdf-to-text utility

• Find the important outputs. Convert them to plain text.

• Use textual diff to decide if the actual output matches the approved version

www.praqma.com
 Elements of a Test Case

Invoice Document Invoice Recipient Approved

Test Workflow
Issuer Recipient Presentation Output

common to to determine
for reference
Input data varies by test case many tests pass/fail

• Minimise test creation work

• Minimise test maintenance work
• Maximise serendipity - find bugs you didn’t anticipate

www.praqma.com
Debugging a failing test?

Test Case

Which part broke?

www.praqma.com
Correlation ID

Pick out all events with ‘123’

www.praqma.com
 Event log recorded in a test case

Helps you to debug what happened when the test fails

www.praqma.com
 Test Case Elements

Invoice Document Invoice Test Workflow Event Logs Approved

Issuer Recipient Output

common to to debug to determine

Input data varies by test case many tests the test pass/fail

• Recorded traffic is stored with the Recipient

Presentation
approved output for reference

www.praqma.com
Independent Microservices
& Teams

Organizing Testing efforts

www.praqma.com
 Testing in the pipeline

version
unit
microservice
system
manual
production
control test test test test

Pair

Programming,

TDD
more production-like environment

test larger pieces of code together

monitoring
Test costs increase
www.praqma.com
 Multi-team development

Component B

Infrastructure & Architecture

Component A

Components C&D
Components E&F
 Testing in the pipeline

version
unit
component
system
manual
production
control test test test test

Pair

Programming,

TDD

monitoring

www.praqma.com
 Multi-Team CD
Team A

micro- system
manual

version
unit
production
service
test test
control test
test

These tests are

often broken!

monitoring

Team B
 Team pipelines
Team A

micro- team-
manual

version
unit
system

service
system
test
production
control test test
test test

One environment
shared by all teams
monitoring
Team B
 Teams Deploy Independently
Team A incremental

micro- team-
team-
roll-out to

version
unit

service
system
manual
production
control test
test test test

monitoring &

manual testing
Team B
 Summary

@emilybache
monitoring
Testing in a
Microservices Architecture team-

system

test

System Tests
 Summary

@emilybache
monitoring
Testing in a
Microservices Architecture team-

system

test

System Tests
The Continuous Delivery and DevOps Company

www.praqma.com