1.1 What Constitutes a Success?

 Much of a test’s success or failure is founded on the goals and objectives stated at the
onset of the test.
o Without planning and some form of goal, there is little chance of determining
what was actually accomplished.
 Yes, you will have the results of the test and can use them in some fashion to perform
tactical remediation, but what of the broader picture?
 It ultimately comes down to:
o Do you want to run a test to generate work, or
o Do you want to run a test to generate meaningful security activities?
 Again, without clarity in goals supported by insightful planning that takes into
consideration inherent and imposed limitations, and the methodologies and tools
employed, you will get exactly what you planned for—a list of vulnerabilities.
 It seems far more logical to employ investments in testing wisely, taking advantage of the
process to expose meaningful information that you can act on tactically and strategically.

1.1.1 Technical Exploitation

 There are many metrics that can be employed to rate the success of a test, but the most
predominant one is technical exploitation.
 Having a tester penetrate an online application and gain access to a database of credit
card numbers has significant tangible characteristics, which are therefore easy to
measure.

1.1.2 Management of the test

 Another aspect of a success can be the management of the test.
 For example, how well was the test conducted?
 Many organizations establish operating parameters to protect systems, employees, and
customers from any potential threat that may come from hacking systems.
 The most obvious is downtime.
 Bringing a business-critical system down in the middle of the business day can be a
costly mistake.
 How the information collected about the target handled (e.g., protected) during the test
will certainly be scrutinized.
 If the list of vulnerabilities and how they were exploited were to become public, the test
would move quickly from success to damage control.

1.1.3 Deliverables
 Some organizations base the success of the test on the deliverable.
 The quality of the deliverable is paramount to many, understandably so, and even in cases
of total technical failure, the deliverable quality alone can substantiate a success.
 The interchange of value and success will occur in every test.
 Typically, the definition of success will be associated with meeting a set of specific goals.
 More often than not, these goals are those vulnerabilities that are identified and
successfully exploited.
 This should come as no surprise because the foundation of the test is typically to hack a
target!
 However, even the exploitation of a vulnerability does not constitute a success.
o In fact, in some cases, exploiting a hole is exactly what the target does not want
and success is founded on what can be identified—not necessarily broken.
 On the other hand, there are companies that insist on evaluating the exposure to attack
and that are only satisfied if the vulnerability is exploited.
 Typically, this demand is associated with a specific target, such as a new application,
change in the infrastructure, or the addition of new untested technology.
 Nevertheless, there are many situations where the goal is simple—gain access—and not
to accommodate the demand is grounds for failure no matter how well the test was
managed, the deliverable quality, or the execution.
