Reliability Test Kit

Build it. Test it. Ship it.

1
 A note from Anna-Katrina Shedletsky,
CEO and Founder of Instrumental

Great product engineers know that reliability testing is key to

building amazing products, and making them better with each
generation. On my first day as a mechanical engineer at Apple, I
received a ream of paper listing every reliability test that might
be run, and a big part of my job was analyzing test results and
understanding their implications for how the product would
perform in the field.

As the product design lead on the first-generation Apple Watch,

it became my team’s responsibility to invent novel reliability tests
for a product that would be subjected to totally new scenarios in
the field.

This testing-first approach left a huge impression on me. In part,

this experience eventually led to the founding of Instrumental,
whose mission is to help engineers build better by enabling them
to anticipate and resolve problems through better data and AI.

I believe every product engineer needs a firm understanding of

the basic types of reliability tests that apply to any product, and
that’s why we created this Reliability Test Kit. I hope you find
the following content informative and tactical as you work to
#buildbetter.

1

About Instrumental
Electronics brands need to meet higher customer expectations
while delivering more innovation, faster than ever in order to stay
competitive and maintain or capture new market share. Speed of
product development is a competitive advantage in the instant
economy, but most development teams still expect it to take months
to find and fix design or process issues before they’re ready for
mass production.
Instrumental is the first manufacturing optimization platform that
supports AI-powered novel issue discovery, failure analysis, ongoing
monitoring, and defective unit interception so you can ramp sooner
with higher yields and less rework.
2
Find, track, and intercept issues automatically
Instrumental’s proprietary AI analyzes data from every unit at every
key assembly stage to automatically surface new anomalies for
investigation. From our simple cloud-based application, program
managers can immediately understand and prioritize issues, and
product designers can quickly find root cause and validate solutions.
Accelerate NPI
Instrumental eliminates manual analysis, spreadsheet jockeying,
and factory visits, so engineering time is spent designing
solutions, not gathering data from siloed data stores. Typical
customers discover their first new issue within 3 days of deploying
Instrumental and shave weeks off their development timelines.
Achieve continuous optimization in MP
Transferring data from development to production teams has
always been a challenge. When kept on your lines through mass
production, Instrumental automatically provides operational teams
complete Paretos monitoring the frequency of new or existing
issues, providing the most complete picture of yield loss and risk to
support continuous optimization.
 “Instrumental makes failure analysis
a snap during reliability testing”
Many reliability tests are destructive, so it can be difficult to
retroactively analyze failed units. Instrumental offers unique
coverage and traceability by capturing a complete data record of
every unit produced. This enables engineers to turn back the clock
to learn more about why a unit failed a particular test.
Discover a loose screw after a set of test failures? Instrumental can
narrow down root cause by showing whether the screw was loose
before testing began.

3
 Introduction
Reliability testing enables engineers to catch issues that may
degrade a product’s field performance, and characterize risk under
different failure conditions. Reliability tests also catch issues that
can’t be found through other methods, such as discovering that
you picked the wrong plating on an electrical contact. A unit may
work fine in the factory, but may fail once exposed to the harsher
environment of the real world. Something as common as sweat
could cause plating corrosion and would only be discovered through
reliability testing. Reliability tests provide unique insight into
product failings and allow teams to build better.

This kit is the result of combined decades of experience with

top manufacturers and deep familiarity with regulatory testing
requirements, and is intended as a comprehensive guide
to reliability testing. Our team of mechanical engineers and
manufacturing experts has included detailed test definitions, setup
parameters, and instructions for a wide range of reliability tests,
demonstrating best practices for some of the most common testing
scenarios.

The kit covers environmental, mechanical, and coating tests, as

well as instructive diagrams to help any product team develop a
common testing framework and build products better.

Making the most out of this kit

Review the tests in this kit and then decide if and how your
particular product can benefit from each one. Determine whether
you need to make modifications to make the tests more suitable for
your product — or which tests might be missing. After reviewing
this document, you’ll have a great understanding of how to create
repeatable tests that will ensure that you ship a high quality, robust
product.

4
 Contents
I. Environmental Tests...........................................................................6
● Seal Tests.....................................................................................6
○ IPX4 Splashing Water.........................................................8
○ Submersion.........................................................................9
○ Salt Spray.......................................................................... 10
● Thermal Tests.......................................................................... 12
○ Temperature Humidity Cycling (Operational)............. 14
○ 65/90 Heat Soak (Operational)...................................... 14
○ Three Corners (Operational).......................................... 15
○ 85/90 (Non-Operational)................................................ 15
○ Storage (Non-Operational)............................................ 16
● Chemical Tests......................................................................... 17
○ Basic Chemical Exposure................................................ 18
● Ultra-Violet (UV) Test.............................................................. 19
○ Basic UV............................................................................ 19
● Altitude Test............................................................................. 20
○ Basic Altitude................................................................... 20
● Other Environmental Tests.................................................... 21

II. Mechanical Tests........................................................................... 22

● Controlled Orientation Drop.................................................. 23
● Randomized Orientation Drop.............................................. 25
● Retail / Shipping Packaging Box Drop.................................. 26
● Ball Drop................................................................................... 27
● Vibration................................................................................... 28
● Button Cycling......................................................................... 29
● Connector Cycling................................................................... 30
● Connector Lateral Loading.................................................... 31
● Cable Tug (Cable Part Level Test).......................................... 32
● Connector Conical Loading (Cable Part Level Test)............ 33
● Coating Tests........................................................................... 34
○ Cross-Hatch Adhesion.................................................... 35
○ Abrasion........................................................................... 36

Conclusion......................................................................................... 37

5
 Environmental Tests
Environmental testing is a large category that can include thermal
and humidity testing, as well as chemical exposure, corrosion,
ultra-violet (UV) light exposure, water-resistance testing, and many
others. The environmental testing that your product needs will
depend on how it’s going to be used by customers in the field.
An indoor, internet-connected speaker would likely need the
following kinds of validation:
• Thermal, including normal indoor ranges and outdoor ranges
experienced during boat / air /truck shipment
• Chemical, with a focus on household chemicals and products
people put on their skin
• UV, in case the product is put in a sunny window
An outdoor, IOT home product would need:
• Thermal, including extended temperature and humidity ranges
applicable to the countries where the product will be sold
• Chemical, similar to the above, potentially with a slightly
expanded range to include chemicals like gasoline, depending
on where the product is typically used
• Salt-spray testing, in the event the product might be installed on
a house near the ocean
• UV
• Water resistance — likely at least IPX6, potentially including IPX8
6
Seal Tests
In the past few years, expectations for product resilience have
consistently increased, in particular when it comes to water
resistance. You may have noticed terms like ‘weather resistant’ and
‘splash proof’ and wondered what these marketing terms actually
mean. Why don’t they just say ‘water proof’? Does ‘spill resistant’
include drinks like coffee and soda? In the section below we will
outline the most common seal tests along with a few insider tips to
take the mystery out of ingress testing.
Common Ratings:
IP44: common for indoor devices designed for kitchens or
bathrooms
IP65: common for weather proof outdoor devices like security
cameras
IP67: common for top-line cell phones since 2016
IP68: common for rugged wearables like smart watches
 Table of IPX Ratings
The most common system for electronics is the IPx rating outlined
below for dust and water protection.

Degree of Protection Against Degree of Protection Against

IP Rating IP Rating
Solid Foreign Objects Water

IP0x Not Protected IPx0 Not Protected

IP1x Solid object >50 mm IPx1 Dripping water

IP2x Solid object >12.5 mm IPx2 Dripping water when tilted up to 15°

IP3x Solid object>2.5 mm IPx3 Spraying water

IP4x Solid object>1 mm IPx4 Splashing water

IP5x Dust protected IPx5 Water jets

IP6x Dust tight IPx6 Powerful water jets

IPx7 Immersion up to 1 m

IPx8 Immersion beyond 1 m

IPx9 High-pressure / temperature water jets

7
 IPX4 Splashing Water Test
Purpose:
To simulate rain, shower, hand washing, or other accidental liquid
spills that may occur on your device

Required Equipment:
Oscillating Tube water jet

Pass/Fail Criteria:
Unit must pass all functional tests, and experience no other harmful
effects to be considered a pass. Unit should show no signs of water
ingress (see Condensation check)

Test Parameters:
Setup: Unit is turned off during test
Water Volume: 10 liters per min
Pressure: 80-100 kN/m^2
Time: 10 minutes

Procedure:

1. Program the water jets to the current angles, flow rate, and
pressure

2. Place the bare unit in either operational or non-operational

state, depending on test.

3. Close chamber and start test.

4. When program finishes, complete functional and cosmetic

test suite on all unit

8
 Submersion test
Purpose:
To simulate light submersion due to serious spills or shallow water
like sinks

Required Equipment:
Water tank or high pressure water chamber

Pass/Fail Criteria:
Unit must pass all functional tests, and experience no other harmful
effects to be considered a pass. Unit should show no signs of water
ingress (see Condensation check)

Test Parameters:
Setup: Unit is turned off during test
Tank depth: 1m for IPx7 OR deeper for IPx8 (ex: 2m)
Submersion time: 30 minutes
Water temperature: typically 25˚C, but can be varied to simulate hot
tubs, oceans, lakes, etc
Water quality: typically local tap water, but chemicals such as salt or
chlorine can be added to simulate pools, oceans, etc

Procedure:

1. Check the water temperature and chemical content.

2. Place unit in a basket and lower into tank.

3. Set timer for 30 minutes.

4. When timer finishes, complete functional test suite on all units.

9
 Salt Spray
Purpose: Procedure:
To simulate exposure to salty moisture in environmental conditions
such as ocean air or sweat often to test for corrosion of coatings or 1. Place units in the chamber in a position such that condensation
other external materials. cannot pool on a the device

Required Equipment: 2. Close chamber and start test.

Salt spray chamber
Pass/Fail Criteria:
Unit must pass all functional tests, and experience no other harmful
effects to be considered a pass. Some corrosion may be acceptable
depending on the materials used and design.
3. The chamber should remain closed for the entire test period or
failure is obvious
4. When testing finishes, remove the units and dry them carefully.
Visually inspect for corrosion, discoloration, staining, or flaking.

Test Parameters:
Setup: Unit is turned off during test
Temperature: 35˚C
Salinity: 3 quarts of the salt solution per 10 cubic feet of box volume
(4% to 6% sodium chloride concentration)
Exposure time: 24 hrs, uninterrupted
(tests can be run longer, but obvious issues generally appear in the
first 24 hrs)
Nozzle: Orifice from 0.5-0.75 mm in diameter, pressure from 12-18
psi

10
 Test Parameters:
Setup: Unit is turned off during inspection. Unit is wiped down and

Condensation Check left to air dry for 15-30 minutes.

Hot plate temperature: 40˚C
Cold water temperature: <25˚C
The easiest way to detect a leak is to take a unit apart and inspect
sensitive parts like PCBs for corrosion or liquid contact stickers
that change color. Always make sure to fully dry the unit before Procedure:
disassembly or liquid trapped in the seals will easily spread and
confuse your results. 1. Place unit with transparent window facing up on the hot plate.

The problem with disassembly is that it is often destructive to seals,
especially those made with adhesives. This means you can only
check for leaks once, which really limits how much you can learn
from a test suite. If there are any transparent surfaces on your
product, here’s a pro tip on how to do a non-destructive check.
Note: this check only works if there is a transparent surface to look
into the product like a sensor window or camera lens cover.
Purpose:
Non-destructively inspect a unit for water ingress after water
testing.
2. Allow unit to heat up for 10 minutes.
3. Using an eye dropper, place a single drop of cold water on the
transparent window
4. Wait 10 seconds then wipe off the water drop
5. Inspect the transparent window for condensation. If there is
liquid water inside the unit, you will see small droplets form on the
inside of the window. A microscope may be required to see the
small droplets.

Required Equipment:
Hot plate

Pass/Fail Criteria:
The window must be clean of any condensation or water droplets.

11
 Thermal Tests
Thermal testing is one of the most basic and highest bang-for-your-
buck testing that you can do. Every product should meet some basic
standard against reasonable operating temperature and humidity
ranges at the system level.
Thermal testing also does double duty. Not only does it validate
that the product can withstand normal and strife temperature and
humidity profiles, thermal testing goes a step further. High heat
actually has the affect of rapidly aging many materials, including
plastics and circuitry. This aging can be beneficial in understanding
how the performance of the product will degrade over Time.
It’s best practice to run the same unit through multiple tests, in
a series called a waterfall. This is usually done to maximize the
reliability test data for the number of units built. In a reliability
waterfall, it’s very common for thermal testing to be the first step
of testing, so that components can be aged before they enter more
mechanical tests like drop testing.
A THERMAL TEST CHAMBER SOURCE ENVITESTS.COM
12
Thermal Testing Equipment
A thermal chamber is one of the most basic pieces of test
equipment. If you are interviewing a manufacturer that does not
even have a basic thermal chamber capable of running the tests
below, this is a major red flag.
All of the tests specified below will use the same piece of
equipment, called a thermal chamber, to control different profiles
of temperature, humidity, and time. Some tests are operational
(meaning they are conducting with the device on or charging),
and some are non-operational (the device is off). In deciding what
is right for your product — consider the use cases. For example,
products with batteries should not be expected to work over
65 degrees Celsius, as this is typically beyond the safety circuit
allowance for heat — but it may be reasonable to test a device with
a battery that is off up for 85 degree Celsius for short periods of
time to ensure it can withstand being left out on a hot tarmac (if you
are air-shipping), or on the dashboard of a hot car.
For all tests, the chamber should start at ambient temperature,
ramp up or down, and then return to ambient before the chamber
door is opened to avoid causing a thermal shock. Should you want
to test thermal shock or condensation, you can set up different tests
for those.
As a tip, when you write your own product’s specification, be sure to
include the exact make and model number of the machine that will
be used. This provides a little protection should the factory decide
to switch without informing you — you’ll have something to point to
showing them the testing was not to the specification.
 Checkpoints
Since thermal tests tend to be long (most are three days), it is best
practice to check on the status of the units at regular intervals. In
the event that there is an early failure, the unit can be pulled for
failure analysis without waiting for the entire test to complete.
Usually these checkpoints are basic in nature:
● does the unit power on (if it was off)?
● is it still charging normally?
● do all of the buttons function?
● does a basic command work as expected?
● is there any notable cosmetic change, such as discoloration
or paint cracking?

We have included our recommendations for checkpoint timing in

the tests below — and any trade-offs that should be considered.

Criteria
For our fictitious indoor, internet-connected speaker, the passing
criteria is:
● Unit must function and pass all performance tests from
the assembly line (for example, Wifi/BT, Acoustic, button
functionality, etc.)
● Unit should have no visible cosmetic damage or coloration
change.

13
 Temperature Humidity Cycling 65/90 Heat Soak
(Operational) (Operational)
Purpose:
To identify issues resulting from thermal shocks or Purpose:
thermal expansion. To verify operation in a sustained high temperature environment.

Upper Limit: Condition:

65 C, 90% RH 65 C, 90% RH

Lower Limit: Ramp Profile:

-20 C, uncontrolled RH 3 hours up, 72 hour dwell, 3 hours down

Ramp Profile: Number of Cycles:

5 hours up, 1 hour dwell, 5 hours down, 1 hour dwell 1

Number of Cycles: Test Duration:

6 78 hours

Test Duration: Checkpoints:

approximately 72 hours In order to check this at 24 hour intervals, you’ll need to insert
ramp up and downtime to get to ambient. This will increase the test
Checkpoints: length and decrease the validation of sustained operation — which
Typically checked every 24 hours at a time when the chamber will be is a trade-off that your team will have to decide on.
at or near ambient. Any changes observed should be recorded via
photograph.

14
 Three Corners 85/90
(Operational) (Non-Operational)
Purpose: Purpose:
To test a combination of thermal and humidity stresses. To verify resistance to temperature strife for a short period of time.
This test can also be used as a quick preconditioning / aging test for
Phase 1: the unit before mechanical testing.
Hot/Dry
Condition: Condition:
60 C, 5% RH 85 C, 90% RH
Ramp: Ramp:
3 hours up, 24 hour dwell, 3 hours down 15 minutes up, 6 hour dwell, 30 minutes down
Number of Cycles: Number of Cycles:
1 1

Phase 2: Test Duration:

Hot/Humid approximately 7 hours
Condition: Checkpoints:
40 C, 95% RH After test completion.
Ramp:
3 hours up, 24 hour dwell, 3 hours down
Number of Cycles:
1

Phase 3:
Cold/Uncontrolled
Condition:
-20 C, uncontrolled RH
Ramp:
3 hours down, 24 hour dwell, 3 hours up
Number of Cycles:
1

Test duration:
approximately 90 hours

Checkpoints:
After each Phase.

15
 Storage (Non-Operational)
Purpose:
To verify that the unit can be stored at extreme warehousing or
travel conditions.

Phase 1:
Hot
Condition:
65 C, uncontrolled RH
Ramp:
1 hour up, 72 hour dwell, 1 hour down
Number of Cycles:
1

Phase 2:
Hot/Humid
Condition:
40 C, 90% RH
Ramp:
1 hour up, 72 hour dwell, 1 hour down
Number of Cycles:
1

Phase 3:
Cold
Condition:
-40 C, uncontrolled RH
Ramp:
1 hour down, 72 hour dwell, 1 hour up
Number of Cycles:
1

Test duration:
approximately 222 hours
Checkpoints:
After each Phase.

16
 Chemical Tests
While it might not seem like it, most devices come into contact with
a lot of chemicals over their lifetimes. Humans not only secrete
harsh chemicals like sweat and oleic acid, but they also put a lot of
chemicals on their skin (lotions, perfumes, sunblock). Many of these
chemicals can degrade adhesives, coatings, and plastics — which is
one of the reasons water-resistant wearables are challenging.
Chemicals can bubble, craze, and crack plastics and coatings. They
can embrittle plastics and cause discoloration (mustard and denim
are both particularly harsh for this). You’ll want to ensure that your
test plan will consider both.
Similar to thermal tests, chemical testing can be a pre-conditioning
step for other follow on tests. For example, if your product has an
embossed foil logo, after chemical testing you may want to do a
cross-hatch adhesion test to make sure that the logo is not prone to
peeling after chemical exposure. If your product is water-resistant,
you may want to do water-resistance testing after chemical
exposure to ensure the seal has not degraded.
Chemical Testing Equipment
You’ll need a thermal chamber, which you will run at 65/90 Heat
Soak specifications. You’ll also need ample supply of the chemical in
question.
For chemical testing, consider if you can do the test at the part level,
instead of the system level. If you are just trying to validate that
the enclosure plastic will not discolor, part level testing is sufficient.
However, if you are trying to validate that chemical exposure at the
unit level will not degrade water-resistance, you will need to test at
the system level.
17
The test specification outlined in this document is for the evaluation
of a fictitious household device, so the list of chemicals to test are
mostly those that customers may use to “clean” a dirty device.
We’ve also considered some “strife” chemicals which could exist on
customers’ hands and rub off on the device, but we acknowledge
that these chemicals are particularly aggressive.
You should specify the specific chemical you are interested in
testing. Don’t just specify “sunscreen”, as you will get uncontrolled
results. Figure out what are the most aggressive or most popular
brands and configurations of a given chemical, and choose one to
use. For example, Banana Boat Sport Waterproof Sunscreen, SPF 30.
Keep in mind who your target audience is. Are you building a truly
global product? If so, consider the chemicals that are prevalent in
cultures and locations outside of your own. For example, DEET bug
spray such as OFF! is popular in the United States, but other regions
of the world do not often use bug sprays and have alternatives you
may want to consider as well.
As another consideration: if this testing is going on at your factory,
you’ll need to be able to get an ample supply of that chemical
in-region. This can be tricky for more hazardous materials. For
these reason, some companies decide to run these tests at their
Headquarters or with a research lab nearby.
Checkpoints
Check every 24 hours for all chemical tests. We recommend plaque
or part level testing. Keep an untested part as a control, and
photograph the part under test next to the control at each check
point (to monitor and track cosmetic changes, if any).
Criteria
For our fictitious indoor, internet-connected speaker, the passing
criteria on the part level for the plastic enclosure parts should be to
have no visible cosmetic damage or coloration change.
 Basic Chemical Exposure
Purpose: Procedure:
To confirm that the device and its exposed components are resistant
to chemicals the device will likely encounter during normal use. 1. Set aside a control part that will not be tested.

Approved Equipment: 2. On a test part, apply a small amount of material to the testing
● Thermal chamber, including make and model number. area. Measure the weight of the application and use for future
● Windex with ammonia application of that chemical.
● 93% isopropyl alcohol
● Fantastik 3. Allow material to “dry” for 10 minutes.
● Vinegar
● Banana Boat Sport Waterproof Sunscreen, SPF 30 4. Put sample in thermal chamber set at the 65/90 Heat Soak
● Oleic A specification (see Thermal Tests in this document for reference).

5. Check sample every 24 hours. Take a photograph of the sample

Test Setup: next to the control part.

6. One the full set is completed, remove any trace material with a
dry microfiber cloth.

Compare the chemically-exposed area to the area around the

chemical exposure and to the control.

Recommended Sample Size:

1 part / unit per chemical, per build

18
 Ultra-Violet (UV) Test Basic UV
Purpose:
Any product that will be exposed to sunlight, should undergo UV To verify the color-fastness of the resins and coatings used in the
testing — at least at the part level. UV notoriously yellows white product. 50% of UVa (the component that causes yellowing) gets
plastics and can also influence the appearance of displays or other through windows.
components that you might not expect.
Approved Equipment:
1. Xenon-Arc test chamber
2. Spectrometer
UV Testing Equipment
Test Parameters:
You’ll need a Xenon-Arc test chamber. These are not commonly ● 0.8 +/- 0.05 W/m2 at 420 nm wavelength
found at manufacturers, but might be available at your plastics ● black panel temperature 55 +/- 2 C
or resins supplier. Alternatively, you can contract with an outside ● relative humidity 55 +/- 5%
testing lab to complete this test.
Recommended Sample Size:
1 sample to receive exposure, 1 control for comparison
Checkpoints
UV testing takes 100s of hours. We recommend spacing checkpoints Procedure:
evenly through your exposure period.
1. Set aside a control part that will not be tested.

Criteria 2. Measure the color the part using the spectrometer in at least
four places.
For our fictitious indoor, internet-connected speaker, the
specification is 300 hours of UV 3. Put the part in the chamber.
exposure and the passing criteria on the part level for the plastic
enclosure parts should be a 4. Check sample every 100 hours and measure the color in the
coloration change of less than one LAB delta-E (a just noticeable same locations with the spectrometer. Also compare to the
difference). reference part for the delta-E variation.

19
 Altitude Test
All products need some degree of altitude testing since they may be Pass/Fail Criteria:
shipped on an airplane or used at high altitude. Unit must pass all functional tests to be considered a pass.

Test Parameters:
Altitude Testing Equipment
Non-operational (Airplane)
You’ll need an altitude chamber. These are not commonly found at Setup: unit is not plugged in during test
manufacturers, so you will likely need to use an outside testing lab. Height: 19000 ft.
Ramp up time: 30 minutes
Dwell at altitude: 60 minutes
Criteria Ramp down time: 30 minutes

For our fictitious indoor, internet-connected speaker, the passing Operational

criteria is for there to be no functional or cosmetic difference after Setup: unit is plugged in and operating during test
the test. Make sure to test your speakers, microphones, and any Height: 12000 ft
other components of the product that have portions that may be Ramp up: 30 minutes
“sealed off” from the outside environment. Dwell at altitude: 60 minutes
Ramp down: 30 minutes

Basic Altitude Procedure:

1. Program the altitude chamber to the correct profile as described

Purpose: above.
To simulate non-operational airplane shipment and to simulate
operation in a high altitude geography. This is particularly important 2. Place the bare unit in either operational or non-operational
for products with diaphragm’s like speakers and microphones. state, depending on test.

Approved Equipment: 3. Close chamber and start test.

Altitude chamber
4. When program finishes, complete functional test suite on all
units.

20
 Other Environmental Tests
Additional environmental tests that you may want to consider, but
are not detailed in this guide would be:

● Mixed Flowing Gas — MFG evaluates corrosion resistance for

gases in the atmosphere, particularly relevant for connectors.
ASTM B827-97 is a good place to look for a specification.

● Wet Finger Testing — If you are building a product that

customers will use with wet fingers, even if it’s not meant to
be water resistant, this is a useful test where you wipe artificial
sweat around button, connector, and other openings on the
product multiple times, usually followed by either a heat soak or
charging to encourage dendritic growth.

● Overpressure Testing — The IP Code will be insufficient should

you be designing an underwater drone or submersible product.
IPX8 doesn’t mean much more than “better than IPX7”. The WR
Code, which is most often found on watches, uses overpressure
to test products in water or air for leaks. Consumers have a
common misperception that WR-30 (which means the product
has been tested to 30 meters of depth) means that they can
swim with the device and even free dive. This is not actually
the case. Review ISO2281 and ISO6425 and select your test
conditions carefully.

21
 Mechanical Tests
Mechanical testing is a large category that validates the product’s
resistance to impulse, pressure, and mechanical fatigue. The
mechanical testing that your product needs will depend on how it
will be used by customers in the field:
Going back to our examples, an indoor, internet-connected speaker
would likely need the following kinds of validation:
● Basic drop testing, with a focus on household surfaces like
wood and granite and probably not more than waist height
(1 meter) which would be reasonable for an “unboxing
fumble”.
● Shipment strife testing, including vibrations and package
drop, to make sure the product isn’t damaged in transit
● Component cycling and overloading for all buttons or
connectors
An outdoor, IOT home product would need to consider:
● Additional heights for drop testing, should the product
typically be mounted 2 meters or higher (such as with a
security camera)
As discussed in the Thermal Testing section, mechanical tests often
follow thermal preconditioning in the waterfall.
{Caption: Drop robot from Shinyei Testing and Machinery Co.}
Mechanical Testing Equipment
The most basic mechanical testing equipment is a free fall drop
robot (which most manufacturers will have in their laboratory)
and a vibration table (which will be more rare). Almost everything
else required will need to be custom, or at least customized, since
mechanical testing very much depends on the features of the
product being tested.
22
As for thermal tests, when you write your own product’s
specification, be sure to include details as to the make and model
number of the equipment, or if it’s custom, the revision of the
equipment allowed. Also be very specific about setup, placement,
and orientation to maximize the repeatability of the tests.
Checkpoints
Mechanical tests are pretty short. Best
practice is to do a quick cosmetic check
and rough functional check after each
impact or set of cycles. This increases
the test time, but leaves your team
clues as to what might be happening
to cause the failure. Some things to
consider:
● does the unit power on?
● do all of the buttons function?
● does a basic command work as
expected?
● is there any notable cosmetic
change, such as a dent or crack?
We have included our
recommendations for checkpoint
timing in the tests below — and any
trade-offs that should be considered.
DROP ROBOT
FROM SHINYEI TESTING
AND MACHINERY CO.
 Controlled Orientation Drop
Purpose:
To validate that the product can withstand mild and medium
severity impacts and identify the weaknesses in the design. This is
achieved through a controlled orientation and a controlled surface.
Those weaknesses can then be bolstered to improve performance.
This test is the workhorse of the mechanical test suite. This test can
be modified for different heights and drop surfaces, all using the
same test setup and procedure. Some common ones are included
below.
Height Variations:
● 0.5m Drop - 1.0m Drop
● 1.5m Drop
Drop Surface Variations: (listed in order of aggressiveness)
● Wood particle board
● Polished granite, ohs hardness 7
● Sidewalk concrete - Rough asphalt (parking lot)
Note:
The specific part number and source of material must be the same
for all builds. Buy several 50 cm x 50 cm pieces up front to ensure
supply for the duration of the program. If the surface is damaged
during testing, which commonly happens with wood, keep moving
the surface around to present a virgin area for each subsequent
test.
23
What to Test and Pass / Fail Specification Considerations:
● For consumer products that get installed and then cannot
be dropped (such as a thermostat or a lightbulb) you’re
really just trying to understand the “unboxing fumble” user
scenario where the user drops it while unboxing. While we
would still do 1.0m Granite Drop for this product, a non-zero
failure rate may be acceptable.
● For a product that is hand-held and carried around, such as
a test meter or smart phone case, more aggressive surfaces
and lower failure rates may be required.
● Most consumer products test at 0.5m and 1.0m on Wood and
Granite.
○ 100% of units should be functional after 0.5m and
1.0m on Wood, and 0.5m on granite.
○ Dents and cracking may occur at 0.5m and higher
on granite, concrete, and asphalt — and may be
deemed acceptable as long as they don’t affect device
usability. For those cosmetic failures that can impact
usability, such as cracked display glass, there may be
a tolerable rate of failure depending on the scenario
and the company’s warranty return plan.
Approved Equipment:
Free fall drop tester, include make and model number in your
specification.
 Unit Orientations:

Procedure:

1. Set drop robot to correct testing height. If you are testing 0.5m,
ensure that the distance between the unit and the surface is
correct.

2. Mount unit in first test orientation.

3. Use a dry microfiber cloth to wipe the entire drop surface to

remove dust or sand particles.

4. Drop the unit with the robot. Capture high-speed video

during this step if you are trying to debug a failure that may be
depending on the drop mechanics.

Note:
Depending on product, you may decide that product “edges” should
also be included, increasing the number of orientations and drops.
Drop Order:
Should be reviewed depending on product. Put your most fragile
orientation first to save time, but do not change orientation order
once you set it, otherwise the test may not be repeatable. Here is a
generic order.
5. Inspect the unit for cosmetic damage. If damage is noted, take a
photo and record SN and drop # when damage occurred.
6. Complete functional test suite. If any tests fail, record the failure
item, SN, and drop # when damage occurred.
7. Repeat steps 2-6 for the remaining of the drop orientations in
the order specified.

Drop Order Orientation

1 Top Face
2 Bottom Face
3 Left Face
4 Right Face
5 Front Face
6 Back Face
7 Top Left Corner
8 Top Right Corner
9 Bottom Right Corner
10 Bottom Left Corner

24
 Randomized Orientation Drop
Purpose:
As an extension to the Controlled Orientation Drop, this test
provides additional confidence that there are no vulnerabilities
to any orientations that were not covered. The same heights and
surfaces from Controlled Orientation Drop should be used.

Procedure:

1. Mark drop height on a wall with the drop surface underneath.

2. Use a dry microfiber cloth to wipe the entire drop surface to

remove dust or sand particles.

3. Using the height mark as a reference, a technician should drop

the unit in a randomized, unspecified orientation. This should be
done by starting the unit in a random orientation and allowing
it to “roll off of the fingers” from that orientation. The goal is to
not add additional significant spin. Capture high-speed video
during this step if you are trying to debug a failure that may be
depending on the drop mechanics.

4. Inspect the unit for cosmetic damage. If damage is noted, take a

photo and record SN and drop # when damage occurred.

5. Complete functional test suite. If any tests fail, record the failure
item, SN, and drop # when damage occurred.

6. Repeat steps 2-5 for the remaining of the drop orientations in

the order specified.

25
 Retail / Shipping Packaging Procedure:

Box Drop 1. Set drop robot to correct testing height.

2. Mount packaging in first test orientation.

Approved Equipment:
Free fall drop tester with box-drop attachment 3. Use a dry cloth to clean drop surface.

Drop Surface Specification: 4. Drop the box with the robot.

polished granite, mohs hardness 7
Unit Mounting Height: 0.75m 5. Repeat steps 2-4 for the remaining drop orientations in the
order specified.
Pass / Fail Criteria:
Unit must pass all functional tests and cosmetic specifications after 6. When complete, unbox all units. Keep track of which units were
unboxing to be considered a pass. in which location in the box.

Box Drop Orientations and Order: 7. Inspect units for cosmetic damage. If damage is noted, take a
As recommended by FedEx. photo and record SN and location in the box.

8. Complete functional test suite on all units. If any tests fail,

record the failure item, SN, and location in the box.

26
 Ball Drop Ball Drop Test Setup:

Purpose:
To simulate an object being dropped on the unit, or the unit being
dropped on another object. You may choose the weight of the ball
that is most relevant to the use case. What is a reasonable object
that could get dropped on the device? How heavy is the device
itself? An acrylic ball will be more forgiving than a steel ball — and
which to use will depend on the use case.

Approved Equipment:
● Ball drop tester (see diagram below)
● Solid acrylic ball, 15 cm diameter, approximately 2 kg (as an
example)

Test Height:
● 0.5m
● 1.0m

Pass/Fail Criteria:
Unit must pass all functional tests after 0.5m ball drop test
sequence. Procedure:

Impact Orientation and Order:
To be defined by your engineers. Usually, you’d pick several
locations on the front and back of the unit. If there are particularly
vulnerable features, you may want to consider those as well. An
example for a home IOT device is shown.
1. Place unit on top of an unyielding surface with correct
orientation presented to ball impact tube. Leave unit
unconstrained.
2. Use a dry microfiber cloth to clean the ball.

3. Drop ball from indicated test height.

4. Catch ball when it bounces -- it should only hit the unit once.

5. Observe and photograph any cosmetic damage, noting drop

number and SN.

6. Test functional tests.

7. Repeat 1-6 for remaining locations.

27
 Vibration
Purpose:
To simulate vibrations that occur during shipment via truck and
airplane.

Approved Equipment:
Vertical electrohydraulic vibration machine (Vibration table)
Procedure:
Standards:
ASTM 4728 (Truck and Air) 1. Program the vibration table to reproduce three consecutive
sequences of random vibration profiles: truck vibration at 0.52
Pass/Fail Criteria: Grms for 30 minutes, air vibration at 1.06 Grms for 30 minutes,
Unit must pass all functional tests and cosmetic specifications after repeat truck vibration at 0.52 Grms for 30 minutes.
unboxing to be considered a pass.
2. Place the test package (packed in retail packaging inside
Profile Definition: shipment packaging) on the vibration table. Fixtures may be used
Truck profile at 0.52 Grms to prevent the package from moving off of the table or other
Airplane profile at 1.06 Grms unsafe conditions, but may not restrict the package’s vertical
movement.

3. Start the test.

4. When complete, unbox all units. Keep track of which units were
in which location in the box.

5. Inspect units for cosmetic damage. If damage is noted, take a

photo and record SN and location in the box.

6. Complete functional test suite on all units. If any tests fail,

record the failure item, SN, and location in the box.

IMAGES FROM FEDEX’S “TESTING PACKAGED PRODUCTS WEIGHING UP

TO 150 LBS.” SEE RESOURCES FOR COMPLETE CITATION.

28
 Button Cycling
Purpose:
To simulate life-cycle usage of each button to understand click feel
or other degradation.

Approved Equipment:
● Operator with finger cots
● Button cycling machine with set force profile

Pass / Fail Criteria: Procedure:

Unit button much still have click feeling, tactility, and pass all
functional tests. 1. Verify click feeling, tactility, sound, and function of button.

Pressure:
Pressing not to exceed 400 gf (your product may require different
specifications)
Cycles:
● If button interaction is critical to user interface, calculate the
number of times a heavy user would use it each day, multiply
by product lifetime (typically 2-4 years), and add a safety
factor based on the consequences of failure. Typically cycles
may be in the mid-to-high thousands.
● Low usages buttons, such as reset buttons, could be tested
much less. Typically hundreds of cycles.
2. Put unit into diagnostic program that will record the number of
button presses.
3. Fixture unit naturally for use if hand-pressing, or securely in the
cycling robot if automating.
4. Complete one cycle: 1 second press, then release. 2 second
(minimum) dwell.
5. Repeat step 4 for number of cycles specified.
6. Record SN, pass/fail, and number of cycles registered by
diagnostic program.

29
 Connector Cycling
Purpose:
To simulate life-cycle usage of each connector port.

Approved Equipment: Testing Cycles:

● Cycling robot with set force profile and a little XY give on the ● Set it and forget it connectors: 500 total (no more than 50%
plug side. may be done by robot)
● Plugs: ● Moderate use connectors: 1000 total (no more than 50% may
○ Must be the same for all builds be done by robot)
○ Before testing, replace cable for a new one if it does ● Frequent use connectors: 3000 total (no more than 50% may
not fit snugly into fresh ports and/or shows visible be done by robot)
signs of wear on the connectors.
○ Snap features must be removed from any ethernet Procedure:
test cables before use.
1. If cycling manually, operator holds unit in one hand, and plugs
Pass / Fail Criteria: the appropriate cable using the other hand. If cycling by robot, the
After cycling, plug must still fit securely into the connector port and unit and plug are fixtured in the robot, but remember to have a
connector must be fully functional. little XY slack on the plug to prevent an over-constrained system
and unrealistic damage.
Test Setup:
2. Complete one cycle: plug cable in, dwell for one second, remove,
dwell for one second.

3. Repeat step 2 for number of cycles specified. Check functionality

after every 500 cycles.

30
 Connector Lateral Loading Test Setup:

Purpose:
To simulate static loading on the plug in order to test solder joints
and internal conductors in the connectors.

Approved Equipment:
● Instron
● Plugs:
○ Must be the same for all builds
○ Before testing, replace cable for a new one if it does
not fit snugly into fresh ports and/or shows visible
signs of wear on the connectors.
○ Snap features must be removed from any ethernet
test cables before use.

Pass / Fail Criteria:

After cycling, plug must still fit securely into the connector port and
connector must be fully functional.
Procedure:
Test Force:
5 kgf 1. Insert cable into connector under test. All other connectors
should be disconnected.
Test Application Location:
5mm from the enclosure along the length of the cable’s connector 2. Fixture unit solidly so that it cannot move.

Test Direction Order: 3. Use instron to apply up to 5 kgf (no faster than 1 second ramp)
force applied towards Front, force applied towards Back, force to the cable-side connector housing in the direction being tested.
applied towards Left, force applied towards Right Dwell for 2 seconds. Remove load.

4. Repeat step 3 (loading cycle) for a total of 5 times in the same

direction.

5. Check full functionality. If fail, record SN, port, and test direction.

6. Repeat steps 3-5 for each test direction in the order specified.

31
 Cable Tug
(Cable Part Level Test) Test Setup:

Purpose:
To simulate pulling on the cable when it is plugged into the device.

Approved Equipment:
Test receptacle for cable being tested -- receptacle should be tested
and tight before starting test, otherwise a new one should be used.

Pass / Fail Criteria:

No visible change in the strain relief area of the cable. Cable remains
functional, even when bent during function test.

Weight: Procedure:
● For USB testing: 3 kgf
● For power cable testing: 2 kgf 1. Plug connector under test into test receptacle that is firmly
● For wall outlet testing: 5 kgf mounted. Set up appropriate weight on the end of the cable
depending on which cable is being tested. Start with 0 degree test.
Recommended Sample Size: During setup verify that the cable is not slack but does not have
5 cables per build additional tension being applied by the weight.

Test Angles: 2. Remove the weight and allow cable to become unplugged.
● 0 degree, 200 cycles
● +45 degree, 200 cycles 3. Repeat step 2 for the appropriate number of cycles.
● -45 degree, 200 cycles
4. Repeat steps 1-3 with the same cable for the +45 degree and -45
degree cases.

32
 Connector Conical Loading
(Cable Part Level Test)
Purpose:
To verify the connector robustness to a dynamic 360 degree load. Procedure:

Approved Equipment: 1. Insert cable into connector under test.

Test receptacle for cable being tested -- receptacle should be tested
and tight before starting test, otherwise a new one should be used. 2. Fixture unit solidly so that it cannot move.

Test Force: 3. Fixture cable into test fixture, as shown in the test setup
1 kgf or depending on connector type diagram.

Test Setup: 4. Complete 500 cycles each: 360 degrees CW, 360 degrees CCW.

5. Check full functionality of both cable and unit. If fail, record SN,
port, and test direction.

33
 Other Mechanical Tests
Other mechanical tests that you may want to consider, but are not
outlined in detail in this guide:

● Short-Range Repetitive Drop — This is relevant if the product

will repeatably get thrown down on a surface, similar to how
you might drop your keys onto a counter. Test heights vary,
but 15 cm is a typical height. Impact resistance used to be
part of the IP Code. Older versions of the code still include a
rating scale for various short distance heights with products
of different weights.

● Tumble Testing — This is relevant is the product will live in a

backpack, purse, or other enclosed space with other objects
that might get “tumbled” around with it. Simulate this test
by filling a rock tumbler with appropriate items for the user
scenario.

Coating Tests
Coating testing validates coatings, paints, foils, or other cosmetic
treatments that are chemically or otherwise bonded to a substrate.
This testing ensures that the added materials will not degrade and
start to de-bond, resulting in cracking or flaking.

Coating tests are usually used after thermal preconditioning, to

determine whether the coating can be considered passing or failing.

34
 Cross-Hatch Adhesion
Purpose:
To validate the adhesion of the coated area after a pre-conditioning
test (most often thermal, could also be mechanical). This test can
be used to confirm time zero adhesion as well as degradation of
adhesion. Typically this test is done on flat surfaces, but can be
adapted for rounded areas.

Approved Equipment:
● cross-hatch adhesion kit
● cross-hatch knife
● single-sided tape with carrier layer, PA-280630 (use the same
tape for all builds)

Pass / Fail Criteria: 5B is a pass.

Recommended Sample Size:

3 pieces control, 1 piece for each checkpoint to check performance
over time.

Procedure:
1. Prepare the sample by wiping it with ethanol to make sure it is
clean of any oils or particles.

2. Cross-hatch the sample using the array of blades. The second

set of hatches should be 90 +/- 5 degrees of the first hatch.

3. Remove two layers of tape from the roll and discard. Then
remove a small piece for testing.

4. Apply to the surface of the test piece, rubbing it in well with a

fingernail to remove any air bubbles and to get good adhesion.

5. Pull tape off at a 180 degree peel (ASTM) in one clean motion
with no jerking.

6. Evaluate the performance against the classification chart. THIS CROSS-HATCH ADHESION
GRADING CHART IS FROM GARDCO.COM

35
 Abrasion
Purpose:
To validate the abrasion resistance of the coating.

Approved Equipment:
● Taber Linear Abraser 5750
● CS-17 Wearaser

Pass/Fail Criteria:
● 10 cycles: no damage
● 25 cycles: no more than 10% logo removal by area and still
readable
● 50 cycles: no more than 25% logo removal by area and still
readable

Test Conditions:
● 1000 gram weight
● Rubbing speed 25 cycles / minute
● 1 inch stroke length

Recommended Sample Size:

2 units per build

Procedure:

1. Set sample on the testbed with pad print in the testing area.
Setup test parameters.

2. Start the test.

3. Inspect at 10, 25, and 50 cycles. Photograph at each interval.

36
 Conclusion
Setting up the right, repeatable product reliability tests can be the
difference between a smooth launch with happy customers and a
full-blown product recall. While it is highly likely that approximately
80% of your test plan may come from this list of specifications, make
sure to consider what tests are missing for your product and its use
case. It would be reasonable to expect to develop a few additional
tests to cover your product specific scenarios.

Hopefully the Instrumental Reliability Test Kit has given you many
ideas for what types of tests to consider, how to set them up, and
how to customize them for your product. Whether you catch issues
in reliability testing or on the line with Instrumental, Instrumental is
here to help your team reach new heights in quality and
#buildbetter.

Contact us at sales@instrumental.com

37
 Resources
There are tons of publicly available references for thermal profiles,
mechanical tests, and even performance testing that are in use in
different industries. For inspiration, you may want to review the
following resources.

● Hardware Development Whitepaper. AT&T. 2011.

● Device Hardware Reliability Test Plan. CTIA. 2015.

● Testing Packaged Products Weighing up to 150 Lbs. FedEx.

● Effects of High Altitude on Packaging Integrity. S. Paul Singh.

2002.

● Steel Ball Drop / Impact Test UL 60950: Promated Diamond

Series Touch Display. Promate Solutions. Video. 2015.

Suppliers:
● Oleic Acid

● Cross-Hatch Adhesion Test Kit

Instrumental’s Manufacturing Intelligence blog is a great resource

for best practices in designing and building hardware products.

38