SecDevOps

DESCRIPTION AND PRIMER

Distrust and Caution
“Distrust and caution are the
parents of security”
◦ Benjamin Franklin
Traditional Roles - Silos
Development Operations Security
• Ship code • Keep code stable • Lock up the code
• Get around blocks • Block changes • Block changes
• Do minimal • Get paged because • Prescribe fixes
Operations and of other peoples • Work in a locked
Security code room
requirements • Attend funny
sounding
conferences
• Obsessed with hats
Traditional Roles – Silos v2
DevOps Security
• Optimized for Fast • Optimized for fewer
iterations incidents
• Automated • Adds API certifications,
Deployments Architecture review
gates
• Only approved tools
What is (Sec)DevOps
IF Operations + Development Automation = Devops
THEN Security + Development Automation = SecDevOps?

SecDevOps is
◦ Security Automation Security
◦ Security at Scale
◦ Discussing security and business tradeoffs

Operations Development
SecDevOps is not
◦ Only there to Audit code
◦ Ivory tower security
Why Security Automation?
Why Security Automation?
Reduce risk of human error
◦ Automation is effective
◦ Automation is reliable
◦ Automation is scalable
Typical Enterprise Ratio
Developers : Operations : Security
?: ?: ?
Typical Enterprise Ratio
Developers : Operations : Security
100 : 10 : 1
Typical Enterprise Ratio + Security
Champions
Developers : Operations : Security
100 : 10 : (+10) 1

Security Champion: One developer from each team, assuming ten teams, spending a small amount of
time to gain proficiency and lead their team in automating and implementing security
Answer: Security Champions
Best Practice: Teams of 10 or less with all the skills needed to push to production

This person should be the security champion within the team. They should represent the voice
of security while still performing some duties as an application developer.

tldr:
Security Champion developers still ship code
Security Champion developers automate security
Security Champion developers watch for the common security gotchas
Security Champion benefits
◦ Understanding and empathy with the security team aka Trust between teams
◦ Higher level of security within the application
◦ Security in the design phase and throughout the whole lifecycle
◦ Top Risks (severity and estimated % chance of occurring) are identified early and kept top of mind
◦ Higher, more productive discussion with the security team (tradeoffs, etc)
Devs developing security automation
Infrastructure as code
Devops is generally a trend to automate traditional operation tasks such as deploying code and
increasing the availability and uptime of that running code.
Security as code
(Sec)devops is enforcing good security patterns and automating traditional security checks
(ports, sql inject inputs, etc)
How to be a Security Champion?
Learn the basics! OWASP Top 10 application vulnerabilities
How to be a Security Champion?
Learn the basics!
8 secure design principles from Saltzer and Schroeder: 
◦ Economy of mechanism: Keep the design simple
◦ Fail-safe defaults: Fail towards denying access
◦ Complete mediation: Check authorization of every access request
◦ Open design: Assume attack knows the system internals
◦ Separation of privilege: require two separate keys or other ways to check authorization (2 factor auth)
◦ Least privilege: Give only necessary rights
◦ Least common mechanisms: Ensure failures stay local
◦ Psychological acceptability: design security mechanism that are easy to use
Basics? Really?
Do apps still get hacked with SQL injection? You betcha!
Web Hacking Incident Database lists these and other public incidents
How to be a Security Champion?
◦ You can likely add
value with OWASP
& Secure Design
Principles
Could a Security Champion defeated the
Equifax attack?
Facts
◦ 143 million breach victims had their names, social security numbers, birth
days, addresses and driver licenses stolen by attackers
◦ The site was hacked in mid-May, and attacked continued to access the data
until late July (breach discovered)
◦ The Equifax breach notification page was using a free shared CloudFlare
SSL, causing many browsers to think it was a phishing site (not terrible, but
not helping)
◦ Signing up for Equifax’s identity theft protection forces uses to accept a
terms of service that waive ability to sue Equifax. (Also not helping)
◦ Finally, an employee portal in Argentina had admin/admin as the
username/password. (indicates a lax security posture)
Action: Equifax Tech Stack
What was the tech stack?
◦ Apache Struts
◦ IBM WebSphere
◦ Java

Apache Struts was compromised by an

announced vulnerability 2 months prior (Apache
Struts CVE-2017-5638)
◦ Failure to track and update vulnerabilities is a common OWASP issue (A9)

The personal information of the entire US adult

population was within reach of this vulnerable
web server
◦ Over reliance on perimeter security is absolutely suspect
 Action: A layered architecture (Microservices)
would have likely defeated this attack
Apache Software Foundation Response
◦ “Establish security layers. It is good software engineering practice to
have individually secured layers behind a public-facing presentation
layer”
◦ “A breach into the presentation layer should never empower
access to significant or even all back-end information resources. ”
Principle of Least Privilege
◦ Implementing rate limiting between services would have been effective
◦ Normal use case for data retrieval required retrieving a single record per session
and retrieving the entire database should have set off alarms
 Action: Monitoring between layers
would have likely detected the attack
Apache Software Foundation
◦ “Establish monitoring for unusual access patterns to your public Web
resources ”
◦ Queries could have been monitored looking for suspicious behavior
◦ Many open and commercial tools will look for common attach patterns and alert
◦ A firewall could have been placed between services
◦ Basically, don’t put all of America’s PII behind one compromised web
server
◦ And if you do, try to monitor and halt the breach before several months pass
Result: Security basics for Microservices
and beyond
1. OWASP Top 10
2. 8 Secure Design Principles from Saltzer and Schroeder
3. Understand and keep updated libraries and frameworks used. Know any known security
vulnerabilities. Roll out updates rapidly.
4. Monitor for intrusions and run scenarios if something you imagine cannot be hacked was
indeed exploited, what would happen?
5. Have a layered security defense when possible
Implementing
Experience OWASP ‘Juice Shop’
’Juice shop’ is an example of an intentionally insecure app that has one of each of the OWASP
top 10 vulnerabilities
◦ It makes a great team exercise, as it gamifies the finding of vulnerabilities and can be done over a lunch
break
◦ It is incredibly enlightening to experience how intruders can exploit common vulnerabilities such as sql
injection and cross site scripting
Devops Fundamentals assist Security
The ability to rapidly deploy code is crucial
◦ What if you need to be able to deploy a patch quickly to protect against attacks?

Valid backups that are regularly used

◦ What if your database and application got seized with ransomware tomorrow? Could you wipe it clean
and redeploy?

Detailed logging of all access

◦ Typically a low hanging fruit when it comes to establishing service uptime. Spotting attacks and intrusion
can be much, much harder without decent logging practices in place
Open Design Principle Applied
Share your code internally
◦ Encourage sharing of components, finding of vulnerabilities and bugs
by peers

Security through obscurity is usually a bad idea

◦ Somehow Google shares their whole code base internally and don’t fall
over due to hackers

Share code internally and encourage vulnerability submissions

◦ Fix reported vulnerabilities, responding within 0-14 days.
◦ Extra credit for bug bounties and other internal security efforts
Test Driven Security
Write security tests that are triggered in the Continuous
Delivery pipeline alongside unit tests.
◦ Same benefits as unit tests, ensuring a baseline security in the
application
◦ Ensures specific vulnerabilities are not opened in the future

Encourages developer ownership of application security

◦ Security Vulnerability and remediation are tracked and tackled like
other tasks
Automated vulnerability scans (black box)
Fuzzer – Quality assurance technique that involves providing invalid, unexpected, or random
data as inputs to a computer program (wikipedia)
◦ Popular tools
◦ Honggfuzz, peach, fuzzer, radamsa

Web (vulnerability) Scanner - program which communicates with a web application through the
web front-end in order to identify potential security vulnerabilities in the web application and
architectural weaknesses (Wikipedia)
◦ Popular tools
◦ Zaproxy, Cenzic, Appscan, Webinspect, nikto

* Hyperlinks are open source

Automated vulnerability scans (white box)
Static Code Scanner - attempt to highlight possible vulnerabilities within 'static' (non-running)
source code by using techniques such as Taint Analysis and Data Flow Analysis. (wikipedia)
◦ Popular tools
◦ HP Fortify, Vericode, Bandit, Brakeman, Contrast
◦ False Positives
◦ These tools can highlight some pretty frustrating false positives, made worse by external teams who demand resolution of said
‘vulnerabilities’ at all costs.

Contrast detecting an Equifax exploit

* Hyperlinks are open source
Risk Summary Table
The team understands the risks their product is exposed to

Name Severity Chance of Occuring Mitigation

Zombie Outbreak Extreme Very Low Zombie plan created last
sprint in repo
SQL Injection Med High Input validation
enforced. Security unit
tests run with
deployments.
Incidents
Incidents
Incidents
◦ Attacks are less a matter of IF and more a matter of WHEN
◦ Attacks help reinforce security investment or highlight new areas that need it
◦ Attacks should help the application become stronger over time with effective hardening of
targeted vulnerability
◦ Nothing like real incidents to test the ruggedness of a service
◦ Hope is a strategy.. Preparation, Continuous Improvement and Team Ownership is a better
strategy
Security Checklists
Shieldify Security Checklist (https://github.com/shieldfy/API-Security-Checklist)
CI & CD
Audit your design and implementation with unit/integration
tests coverage.
Use a code review process and disregard self-approval.
Ensure that all components of your services are statically
scanned by AV software before push to production, including
vendor libraries and other dependencies.
Design a rollback solution for deployments.
Other Security Topics (Beyond the
basics)
◦ Security Monitoring
◦ Threat Modeling
◦ Secure Networking
◦ Public Key
◦ Certificates
◦ etc
$WHOAMI
◦ Peter Lamar
◦ Opensource & Golang fanatic
◦ Software Engineering Director, UnitedHealth Group
◦ @ptlamar
Appendix
PCI
Build and maintain a secure network
Protect userdata
Maintain a vulnerability management program
Implement strong access control measures
Regularly monitor and test networks
Maintain an information security policy
• Install and maintain firewall
• Do not use vendor-supplied defaults for system
passwords & parameters
• Encrypt data acros networks
• Protect stored user data
• Develop and maintain secure
systems/applications
• Use and update antivirus software
• Assign and track each person with unique ID
• Restrict access to data by need-to-know
• Track, monitor and log all access to network
resources
• Regularly test security systems
• Maintain policy that addresses information
security