‎PRODUCT ELEMENTS 60

‎Capability ‎Can it perform the required functions?

‎ ata Integrity: the data in the system is

D
‎protected from loss or corruption.

‎ rror handling: the product resists failure in

E
‎ ill it work well and resist failure in all
W
‎the case of errors, is graceful when it fails,
‎Reliability ‎required situations?
‎and recovers readily.

‎ afety: the product will not fail in such a way

S
‎as to harm life or property.

‎ earnability: the operation of the product can

L
‎be rapidly mastered by the intended user.

‎ perability: the product can be operated

O
‎ ow easy is it for a real user to use the
H
‎with minimum effort and fuss.
‎Usability ‎product?
‎ o you know who your customers are?
D
‎Whose opinions matter? Who benefits or ‎ ccessibility: the product meets relevant
A
‎suffers from the work you do? ‎accessibility standards and works with O/S
‎accessibility features.
‎ o you have contact and communication
D
‎ ecurity holes: the ways in which the system
S
‎with your customers? Maybe they can help
‎cannot enforce security (e.g. social
‎you test.
‎Anyone who is a client of the test project. ‎Customers ‎engineering vulnerabilities)

‎ aybe your customers have strong ideas

M
‎ uthorization: the rights that are granted to
A
‎about what tests you should create and run.
‎authenticated users at varying privilege
‎ ow well is the product protected against
H ‎levels.
‎ aybe they have conflicting expectations.
M
‎Security ‎unauthorized use or intrusion?
‎You may have to help identify and resolve
‎ uthentication: the ways in which the system
A
‎those.
‎verifies that a user is who she says she is.
‎ re there any engineering documents
A
‎available? User manuals? Web-based -‎ Privacy: the ways in which customer or
‎OPERATIONAL CRITERIA
‎materials? ‎employee data is protected from
‎unauthorized people.
‎ oes this product have a history? Old
D
‎ ow well does the deployment of the
H
‎problems that were fixed or deferred?
‎Scalability ‎product scale up or down?
‎Pattern of customer complaints?

‎Performance ‎How speedy and responsive is it?

‎ o you need to familiarize yourself with the
D
I‎nformation about the product or project that
‎product more, before you will know how to
‎is needed for testing. ‎Information ‎ pgrades: Can new modules or versions be
U
‎test it?
‎added easily? Do they respect the existing
‎configuration?
I‎s your information current? How are you
‎apprised of new or changing information?
‎ ninstallation: When the product is
U
‎uninstalled, is it removed cleanly?
I‎s there any complex or challenging part of
‎the product about which there seems ‎ ow easily can it be installed onto its target
H
‎Installability ‎platform(s)? ‎ onfiguration: What parts of the system are
C
‎strangely little information?
‎affected by installation? Where are files and
‎resources stored?
‎ ubris: Does the development team seem
H
‎overconfident about any aspect of the
‎product? ‎ ystem requirements: Does the product
S
‎recognize if some necessary component is
‎missing or insufficient?
‎ efensiveness: Is there any part of the
D
‎product the developers seem strangely
‎ esource Usage: the product doesn’t
R
‎opposed to having tested?
‎unnecessarily hog memory, storage, or other
‎system resources.
‎ apport: Have you developed a friendly
R
‎How you get along with the programmers. ‎Developer Relations
‎working relationship with the programmers?
‎ ackward Compatibility: the products works
B
‎with earlier versions of itself.
‎ eedback loop: Can you communicate
F
‎quickly, on demand, with the programmers?
‎ ardware Compatibility: the product works
H
‎ ow well does it work with external
H
‎with particular hardware components and
‎ eedback: What do the developers think of
F ‎Compatibility ‎components & configurations?
‎configurations.
‎your test strategy?
‎ perating System Compatibility: the product
O
‎Do you know who will be testing? ‎works with a particular operating system.

‎ eople who’ve tested similar products before

P ‎ pplication Compatibility: the product works
A
‎and might have advice? ‎in conjunction with other software products.

‎Programmers?
‎ re there people not on the “test team” that
A
‎ ow economical will it be to provide support
H
‎Designers? ‎might be able to help?
‎Supportability ‎to users of the product?

‎Users? ‎Testability ‎How effectively can the product be tested?

‎Anyone who will perform or support testing. ‎Test Team
‎Writers? ‎ ow economical is it to build, fix or enhance
H
‎Maintainability ‎the product?
‎ o you have enough people with the right
D ‎ euristic Testing
H
‎skills to fulfill a reasonable test strategy? ‎ ow economical will it be to port or reuse the
H
‎Strategy Model ( ‎Portability ‎technology elsewhere?

‎Master)
‎ re there particular test techniques that the
A
‎ ow economical will it be to adapt the
H
‎team has special skill or motivation to
‎perform? ‎PROJECT ENVIRONMENT ‎DEVELOPMENT CRITERIA ‎product for other places?

‎ egulations: Are there different regulatory or

R
‎Is any training needed? Is any available?
‎reporting requirements over state or national
‎borders?
‎ ardware: Do we have all the equipment you
H
‎need to execute the tests? Is it set up and
‎ready to go? ‎ anguage: Can the product adapt easily to
L
‎longer messages, right-to-left, or
‎ideogrammatic script?
‎ utomation: Are any test automation tools
A ‎Localizability
‎needed? Are they available? ‎ ardware, software, or documents required
H
‎ oney: Must the product be able to support
M
‎to administer testing. ‎Equipment & Tools
‎multiple currencies? Currency exchange?
‎ robes: Are any tools needed to aid in the
P
‎observation of the product under test?
‎ ocial or cultural differences: Might the
S
‎customer find cultural references confusing
‎ atrices & Checklists: Are any documents
M
‎or insulting?
‎needed to track or record the progress of
‎testing?

‎ est Design: How much time do you have?

T
‎PERCEIVED QUALITY
‎Are there tests better to create later than
‎sooner?

‎ est Execution: When will tests be executed?

T I‎dentify things that the product can do (
‎Are some tests executed repeatedly, say, for ‎functions and sub- functions).
‎regression purposes? ‎ he sequence, duration, and synchronization
T
‎of project events. ‎Schedule ‎ etermine how you’d know if a function was
D
‎ evelopment: When will builds be available
D ‎capable of working.
‎for testing, features added, code frozen, ‎Function Testing ‎Test what it can do
‎etc.?
‎Test each function, one at a time.

‎ ocumentation: When will the user

D
‎ ee that each function does what it’s
S
‎documentation be available for review?
‎supposed to do and not what it isn’t
‎supposed to do.
‎ cope: What parts of the product are and are
S
‎not within the scope of your testing
‎ ook for any data processed by the product.
L
‎responsibility?
‎Look at outputs as well as inputs.

‎Availability: Do you have the product to test?

‎

‎ olatility: Is the product constantly

V
‎boundary values
‎changing? What will be the need for
‎retesting?
‎typical values
‎Decide which particular data to test with.
‎ ew Stuff: What has recently been changed
N ‎Domain Testing ‎Divide and conquer the data
‎or added in the product? ‎convenient values
‎The product to be tested. ‎Test Items
‎ estability: Is the product functional and
T ‎invalid values
‎reliable enough that you can effectively test
‎it? ‎best representatives

‎ uture Releases: What part of your tests, if

F ‎ onsider combinations of data worth testing
C
‎any, must be designed to apply to future ‎together.
‎releases of the product?
‎ ook for sub-systems and functions that are
L
‎ edia: How will you record and
M ‎vulnerable to being overloaded or “broken” in
‎communicate your reports? ‎the presence of challenging data or
‎constrained resources.
‎ tandards: Is there a particular test
S
‎documentation standard you’re supposed to I‎dentify data and resources related to those
‎follow? ‎sub-systems and functions.
‎Stress Testing ‎Overwhelm the product
‎ urpose: Are your deliverables provided as
P ‎The observable products of the test project. ‎Deliverables ‎large or complex data structures
‎part of the product? Does anyone else have
‎to run your tests?
‎high loads
‎ elect or generate challenging data, or
S
‎ ontent: What sort of reports will you have
C
‎resource constraint conditions to test with ‎long test runs
‎to make? Will you share your working notes,
‎or just the end results?
‎many test cases

‎low memory conditions

‎ efine test procedures or high level cases

D
‎that incorporate multiple activities connected
‎end-to-end.

‎Don’t reset the system between tests.

‎Flow Testing ‎Do one thing after another
‎ ary timing and sequencing, and try parallel
V
‎threads.

‎ hink about everything going on around the

T
‎product.

‎TEST TECHNIQUES ‎ esign tests that involve meaningful and

D
‎complex interactions with the product.
‎Scenario Testing ‎Test to a compelling story
‎ good scenario test is a compelling story of
A
‎how someone who matters might do
‎something that matters with the product.

I‎dentify reference materials that include

‎claims about the product (implicit or explicit).

‎ nalyze individual claims, and clarify vague

A
‎claims.

‎Claims Testing ‎Verify every claim ‎ erify that each claim about the product is
V
‎true.

I‎f you’re testing from an explicit

‎specification, expect it and the product to be
‎brought into alignment.

‎Identify categories and roles of users.

‎ etermine what each category of user will

D
‎do (use cases), how they will do it, and what
‎they value.

‎ et real user data, or bring real users in to

G
‎test.
‎User Testing ‎Involve the users
‎ therwise, systematically simulate a user (be
O
‎careful—it’s easy to think you’re like a user
‎even when you’re not).

‎ owerful user testing is that which involves a

P
‎variety of users and user roles, not just one.

‎ hat kinds of problems could the product

W
‎have?

‎Which kinds matter most? Focus on those.

‎ ow would you detect them if they were

H
‎there?
‎Risk Testing ‎Imagine a problem, then look for it.
‎ ake a list of interesting problems and
M
‎design tests specifically to reveal them.

I‎t may help to consult experts, design

‎documentation, past bug reports, or apply
‎risk heuristics.

‎ ook for opportunities to automatically

L
‎generate a lot of tests.

‎ evelop an automated, high speed

D
‎Automatic Testing ‎Run a million different tests ‎evaluation mechanism.

‎ rite a program to generate, execute, and

W
‎evaluate the tests.