Version 12

 What is DevOps
 Why DevOps
 About Chef
 Chef Architecture
 Classroom Environment
 Chef Recipe
 Chef Cookbook
 Manage a Node
 Resource Notifications
 Chef Data Bags
 Chef Exercises with AWS & Azure
What is DevOps?
“DevOpsis “DevOpsis
development treating your
and operations infrastructure
collaboration” as code”

“DevOps “DevOps
is using is feature
automation” switches”

“Kanban
“DevOps for Ops?”
is small
deployments”
Welcome to the 21st century!
CALMS
• Culture
• Automation
• Lean
• Measurement
• Sharing
HIGH TRUST
PERFORMANCE
• Unified mission
• Aligned incentives
• Little fear/failure/blame
• High quality of work life
IT capabilities = strategic
assets, not cost centers
• Projects, features and
work flow through fast
cycles times
• Systems are “anti-
fragile”
• IT processes &
capabilities are aligned
with overarching
organizational needs
Automated Mature
Deployment pipeline
• Technical phases of
projects supported by
common tools and
automation processes
• Collaboration replaces
handoffs
• IT infrastructure is agile
Continuous delivery of
software and IT value
• Features, projects and IT
work follow a regular,
iterative flow
• Cycle time is short
• Workflow favors small
frequent changes
Continuous Learning
& improvement
• Disciplined feedback
loops quickly travel back
upstream for inclusion
• Tools for monitoring,
measurement and
alerting implemented &
effective.
• Shared knowledge
repositories.
Concept / ideation
Value

DevOps is not about IT problems: DevOps is about

business problems.
 DevOps is culture for collaboration, Integration
and Communication between different cross
functional teams (including ops) for Continuous
delivery.
 DevOps encourages Operations to participate
early in the Development cycle so that products
are designed to be easily deployable and
maintainable.
 DevOps emphasizes on keeping WIP/Inventory
low and go to production ASAP.
 Automating for faster delivery with DevOps and cloud
Traditional Infrastructure Platform
On-Premises as a Service as a Service
Applications Applications Applications
(SaaS)
Deploy Data Deploy Data Data

Mid Config Mid Config Mid Config

Middleware Middleware Middleware

Man/Auto Man/Auto
O/S O/S O/S

Virtualization Virtualization Virtualization

PureApplication
System
Servers Servers Servers
Manual SmartCloud
Storage Storage Orchestrator Storage
SoftLayer
Networking Networking Networking

Customization; higher costs; slower time to value

Standardization; lower costs; faster time to value

“DevOps is a methodology with both a technological and collaborative
component. DevOps is an outgrowth of the Agile method and relies on
the Agile process for developing code. It also aims to improve the first
steps of creating a component, such as developing stories and
requirements, and the last stage of the process, which is releasing the
code. One of its basic DevOps tenets is to break down the barriers
between infrastructure and code and to blur or even eradicate the
boundaries between development and operations.”
 Wastage
Issues

Work
Delay
Cultural
Issues
Issues
OI’s

Managem
ent & Quality
Control Issues
Issues
 Integration
 Collaboration
 Communication
The pace of change in our economy is
accelerating remorselessly
San Francisco taxi trips down 65% over past 15 months
Uber 3X Taxi Revenues in 5 Years
 “There's a new trend called "devops" that is sweeping
the enterprise IT world and its become a life-or-death career
situation for many IT departments…
Like the manufacturers were in the 1970s and 1980s were
fighting for their lives,
today's IT departments are going to fight for
their survival.”
 Configuration Management
 Continuous Integration
 Automated Testing
 Metrics
 Collaboration
 Making smart use of smart people
Why DevOps?
DevOps – Business Agility

 Time-to-Market
Acceleration
Business  Experimentation
Agility  Rapid Prototyping
 Flexible Partnering
 Effective Support
DevOps – Technical Innovation

 Polyglot Enablement
 DevOps Automation
Technical  API Support
Innovation  Micro services Architecture
 Application Scaling and
Elasticity
 PaaS
DevOps – Technical Innovation

 Docker Foundation
 Language and Stack Neutral

Infrastructure  Lightweight Hybrid Cloud

Choice with AWS, VMware, &
OpenStack
 Common Application
Design and Operations
DevOps – Technical Innovation

Business
Agility

DevOps
Technical Infrastructure
Innovation Choice
DevOps – Culture is Yoghurt
 Culture is an output, not
an input
 Common organization
with a nearby boss
 Shared metrics foundation
of collaboration
 High-level sponsorship
DevOps – Don’t Build New Legacy

 Smart person + 3 years:

“How’s that system work?”
 Future-proof your
application pipeline
 Seek commercial product
DevOps – Innovation Lab

 Separate Organization
 Accelerate Adoption of
New Technologies
 Exploration via New Skills
and Employees
 Integration Process
Defined
DevOps – Benefits
IT Domain Before DevOps After DevOps

Application lifecycle Months Weeks

Polyglot support Java only 10+ languages/stacks

Integration of
Innovation Lab with Limited Structured
mainstream IT

Attractiveness to new
Minimal Significant
developers
  Efficiency - Faster time
to market
 Predictability - Lower
failure rate of new
releases
 Reproducibility –
Version everything
 Maintainability - Faster
time to recovery in the
event of a new release
crashing or otherwise
disabling the current
system

“Break down the wall between development and operations”

About Chef
 IT Automation Software:
◦ Founded in 2005
◦ First commercial product release in 2009
◦ Previously known as Opscode.
◦ 25,000+ Users
◦ 10,000+ Nodes managed in the largest
deployments
◦ Support for Red Hat, CentOS, Ubuntu, Debian, SUSE,
Solaris 10, Windows, Mac OS X, AIX, FreeBSD.
◦ Chef Supermarket as common platform.
 Manually Configure
◦ Literally logging in to every node to configure it
 Golden Images
◦ Creating a single copy of a node’s software and
replicating that across nodes
 Custom One-off Scripts
◦ Custom code written to address a specific, tactical
problem
 Software Packages
◦ Typically all or nothing approach
 Difficult to scale
 Impossible, for all intents and purposes, to
maintain consistency from node-to-node.
 Need separate images for different
deployment environments, eg, development,
QA, production, or different geo locations.
 As number of images multiply it becomes
very difficult to keep track and keep
consistent.
 Since they’re monolithic copies, golden
images are rigid and thus difficult to update
as the business needs change.
 No leverage – effort typically cannot be re-
used for different applications or
deployments.
 Brittle – as needs change, the entire script
must be often be re-written.
 Difficult to maintain when the original author
leaves the organization.
 These packages typically require that all
resources be placed under management –
cannot selectively adopt and scale
automation.
 As a result, longer deployments times.
 Dated technology developed before
virtualization and cloud computing – lacks
responsiveness to changing requirements.
 Automation for System Administrators
 Web Scale IT
 Enable DevOps
 Continuous Delivery
 Analytics Platform
◦ Get visibility into your Chef servers, verify compliance
and keep up with changes, all with the Chef analytics
platform.
 Management Console
◦ Use the web-based management console to manage
RBAC, edit and delete nodes, and reset private keys.
Keep up to date with what’s happening during chef
client runs across an entire organization or on specific
nodes.
 Reporting
◦ Capture and visualize what happens during the
execution of chef-client runs across all of your Chef-
managed infrastructure.
 High Availability
◦ Ensure that your Chef service is uninterrupted
within your data center or AWS region, even if a
Chef server fails.
 Replication
◦ Maintain a single worldview across multiple
locations and ensure consistency across your
network, no matter how many data centers or cloud
availability zones you use in your enterprise.
 Push Jobs
◦ Execute commands across hundreds or even
thousands of nodes in your Chef-managed
infrastructure.
Chef Architecture
 Chef is a powerful automation platform that
transforms complex infrastructure into code,
bringing your servers and services to life.
 Whether you’re operating in the cloud, on-
premises, or a hybrid, Chef automates how
applications are configured, deployed, and
managed across your network, no matter its size.
 Chef is built around simple concepts:
◦ Achieving desired state
◦ Centralized modeling of IT infrastructure
◦ Resource primitives that serve as building blocks
 Simplifies installation and configuration.
 Chef Stack Elements include:
◦ Chef Server
◦ Chef Client
◦ Chef Workstation
◦ Chef Supermarket
◦ Chef Analytics
 The Chef server acts as a hub for configuration
data.
 The Chef server stores cookbooks, the policies
that are applied to nodes, and metadata that
describes each registered node that is being
managed by the chef-client.
 Nodes use the chef-client to ask the Chef
server for configuration details, such as
recipes, templates, and file distributions.
 Cookbooks and policy settings are uploaded to
the Chef server by users from workstations.
 The chef-client accesses the Chef server from
the node on which it’s installed to get
configuration data, perform searches of
historical chef-client run data, and then pull
down the necessary configuration data.
 After the chef-client run is finished, the chef-
client uploads updated run data to the Chef
server (as the updated node object), uploads
data generated by audit-mode (for additional
rules processing by Chef Analytics), and
generates reporting data.
 Chef management console is a web-based
interface for the Chef server that provides
users a way to manage the following objects:
◦ Nodes
◦ Cookbooks and recipes
◦ Roles
◦ Stores of JSON data (data bags), including encrypted
data
◦ Environments
◦ Searching of indexed data
◦ User accounts and user data for the individuals who
have permission to log on to and access the Chef
server
 A chef-client is an agent that runs locally on
every node that is under management by
Chef.
 RSA public key-pairs are used to authenticate
the chef-client with the Chef server every
time a chef-client needs access to data that is
stored on the Chef server. This prevents any
node from accessing data that it shouldn’t
and it ensures that only nodes that are
properly registered with the Chef server can
be managed.
 When a chef-client is run, it will perform all of
the steps that are required to bring the node
into the expected state, including:
◦ Registering and authenticating the node with the Chef
server
◦ Building the node object
◦ Synchronizing cookbooks
◦ Compiling the resource collection by loading each of the
required cookbooks, including recipes, attributes, and
all other dependencies
◦ Taking the appropriate and required actions to
configure the node
◦ Looking for exceptions and notifications, handling each
as required
 Ohai is a tool that is used to detect attributes
on a node, and then provide these attributes to
the chef-client at the start of every chef-client
run.
 Ohai is required by the chef-client and must be
present on a node. (Ohai is installed on a node
as part of the chef-client install process.)
 Attributes that are collected by Ohai are
automatic attributes, in that these attributes
are used by the chef-client to ensure that these
attributes remain unchanged after the chef-
client is done configuring the node.
 The types of attributes Ohai collects include
(but are not limited to):
◦ Platform details
◦ Network usage
◦ Memory usage
◦ CPU data
◦ Kernel data
◦ Host names
◦ Fully qualified domain names
◦ Other configuration details
 Chef Analytics provides real-time visibility
into what is happening on the Chef server,
including what’s changing, who made those
changes, and when they occurred.
 Details are tracked by the chef-client during
the chef-client run.
 These details are uploaded to the Chef server
at the end of the chef-client run.
 This data is used to build reports, run rules
against the output of audit-mode, generate
notifications based on the results of auditing,
and visibility into messages that were
generated during the chef-client run.
 Chef analytics includes:
◦ actions
◦ reports
 One (or more) workstations are configured to
allow users to author, test, and maintain
cookbooks.
 A workstation is a computer that is
configured to run knife, to synchronize with
the chef-repo, and interact with a single Chef
server.
 The workstation is the location from which
most users will do most of their work,
including:
◦ Developing cookbooks and recipes (and authoring them
using Ruby)
◦ Keeping the chef-repo synchronized with version source
control
◦ Using knife to upload items from the chef-repo to the
Chef server
◦ Configuring organizational policy, including defining
roles and environments and ensuring that critical data is
stored in data bags
◦ Interacting with nodes, as (or when) required, such as
performing a bootstrap operation
 Commands
◦ Chef includes two important command-line tools.
◦ Use the knife command-line tool when interacting with
nodes or when working with objects on the Chef server.
◦ Use the chef command line tool when working with the
chef-repo, which is the repository structure in which
cookbooks are authored, tested, and maintained.
 Chef Supermarket is the location in which
community cookbooks are authored and
maintained.
 Cookbooks that are part of the Chef
Supermarket may be used by any Chef user.
 How community cookbooks are used varies
from organization to organization.
 A cookbook is the fundamental unit of
configuration and policy distribution. A
cookbook defines a scenario and contains
everything that is required to support that
scenario:
◦ Recipes that specify the resources to use and the
order in which they are to be applied
◦ Attribute values
◦ File distributions
◦ Templates
◦ Extensions to Chef, such as libraries, definitions,
and custom resources
 The chef-client uses Ruby as its reference
language for creating cookbooks and
defining recipes, with an extended DSL for
specific resources.
 Some important components of cookbooks
include:
◦ attributes
◦ recipes
Classroom Environment
 Pre-installation
◦ Assign a hostname to your machine(Master or
Agent) and make that name persist across reboot.
◦ Ensure time is synced between Chef Server &
Nodes.
 Chef Server would be installed on Training
Environment as demo LAB.
 Chef uses SSL to facilitate secure Node – Server
communication.
 Once the Node has to get SSL certificate from Chef
Server to enable trusted communication.
 Chef Workstation would be installed on Training
Environment as demo LAB.
 Chef Node would be installed on Training
Environment as demo LAB.
Chef Recipe
 A Chef resource describes some piece of
infrastructure, such as a file, a template, or a
package.
 A Chef recipe is a file that groups related
resources, such as everything needed to
configure a web server, database server, or a
load balancer.
 Resources describe the what, not the how.
 Resources have actions.
 Recipes organize resources.
 Chef applies resources in the order you specify.
 Chef enforces in an idempotent way.
 The property of certain operations in
mathematics or computer science is that they
can be applied multiple times without further
changing the result beyond the initial
application.
 Able to be applied multiple times with the
same outcome.
 Chef resources are idempotent, since they
describe a desired final state rather than a
series of steps to follow.
 Create the chef-repo directory under your
home directory, ~/
mkdir ~/chef-repo
cd ~/chef-repo

 Create a file “hello.rb” with following

contents:
file 'motd‘ do
content 'hello world'
end
 Apply the recipe
chef-apply hello.rb

 Create a file “goodbye.rb” with following

contents:
file 'motd‘ do
action :delete
end

 Apply the recipe

chef-apply goodbye.rb
 From your ~/chef-repo directory, add this
recipe to a file named “webserver.rb”.
package 'httpd'

 Apply the recipe

sudo chef-apply webserver.rb

 Note, here that “install” is default action, so

we don’t need to specify action.
 Now, modify “webserver.rb” to look like
below:
package 'httpd‘
service 'httpd' do
action [:enable, :start]
end

 Apply the recipe

sudo chef-apply webserver.rb
 Now, modify “webserver.rb” to look like below:
package 'httpd'
service 'httpd' do
action [:enable, :start]
end
file '/var/www/html/index.html' do
content '<html>
<body>
<h1>hello world</h1>
</body>
</html>'
end
 Apply the recipe
sudo chef-apply webserver.rb
Chef Cookbook
 A cookbook provides structure to your
recipes and enables you to more easily
reference external files, such as our web
server's home page.
 In essence, a cookbook helps you stay
organized
 First, from your ~/chef-repo directory,
create a cookbooks directory and cd there.
mkdir cookbooks
cd cookbooks

 Now run the ”chef” command to generate a

cookbook named ”learn_chef_httpd”.
chef generate cookbook learn_chef_httpd
 Here, is directory structure that command
created.
# tree .
learn_chef_httpd
|- Berksfile
|- chefignore
|- metadata.rb
|- README.md
|- recipes
|- default.rb

 Now, “default.rb” is recipe file.

 Ruby’s built-in templating language.
 Templates are mostly plain text files.
 Inserting ERB tags allows you to:
◦ Display or act on the contents of variables.
◦ Alter the flow of logic.
◦ Include Ruby code to perform calculations or
iterate.
 Now run the ”chef” command to generate a
template for our home page.

chef generate template learn_chef_httpd index.html

 The file ”index.html.erb” gets created

under ”learn_chef_httpd/templates/default”.
 Now, your “default.rb” file should look like
below:

package 'httpd'
service 'httpd' do
action [:enable, :start]
end
template '/var/www/html/index.html' do
source 'index.html.erb'
end
service 'iptables' do
action :stop
end
 Now, your “index.html.erb” file should look
like below:

<html>
<body>
<h1>hello world</h1>
</body>
</html>
 Now, we will run the cookbook using below
command:

sudo chef-client --local-mode --runlist ‘recipe[learn_chef_httpd]‘

 In this example, recipe[learn_chef_httpd] is the same

as specifying recipe[learn_chef_httpd::default],
meaning we want to run learn_chef_httpd cookbook's
default recipe, default.rb.
Manage a Node
 Typically, Chef is comprised of three elements – your
workstation, a Chef server, and nodes.
 Your workstation is the computer from
which you author your cookbooks and
administer your network.
 It's typically the machine you use everyday.
 Although you'll be configuring a Red Hat
Enterprise Linux server, your workstation
can be any OS you choose – be it Linux, Mac
OS, or Windows.
 Chef server acts as a central repository for
your cookbooks as well as for information
about every node it manages.
 For example, the Chef server knows a
node's fully qualified domain name (FQDN)
and its platform.
 A node is any computer that is managed by
a Chef server.
 Every node has the Chef client installed on
it.
 The Chef client talks to the Chef server.
 A node can be any physical or virtual
machine in your network.
 Chef server gives you a persistent location
to store your cookbooks and information
about your nodes.
 The ”knife” command enables you to
communicate with the Chef server.
 Run this command from anywhere under
your~/chef-repo directory on workstation.

knife cookbook upload learn_chef_httpd

 You ran knife bootstrap to associate your
node with the Chef server and do an initial
check-in.
 Bootstrapping is a one-time process.
 During the bootstrap process, your node
downloaded and installed chef-client,
downloaded the latest cookbooks, and
executed the run-list.
 Chef provides information about your node
that you can access from your cookbooks.
 From your workstation, run this command
to bootstrap your node.

knife bootstrap {{address}} --ssh-user {{user}} --ssh-password

'{{password}}' --sudo --use-sudo-password --node-name
node1 --run-list 'recipe[learn_chef_httpd]‘

 Replace {{address}} with your remote node's

external address, {{user}} with your
username, and {{password}} with your
password.
 You can see bootstrap result at following
locations:
◦ Chef Management Console
◦ knife node [list|show node_name]
 Now, change index.html.erb file to look like
below:
<html>
<body>
<h1>hello from <%= node['fqdn'] %></h1>
</body>
</html>
 Now, change index.html.erb file to look like
below:

knife cookbook upload learn_chef_httpd

 Run knife ssh to run your cookbook on your
node.

knife ssh {{address}} 'sudo chef-client' --manual-list --ssh-

user {{user}} --ssh-password '{{password}}'

 Replace {{address}}, {{user}}, and

{{password}} with your values.
Chef Data-Bags
 A data bag is a global variable that is stored
as JSON data and is accessible from a Chef
server.
 A data bag is indexed for searching and can
be loaded by a recipe or accessed during a
search.
 A data bag can be created in two ways:
using knife or manually.
 In general, using knife to create data bags
is recommended, but as long as the data
bag folders and data bag item JSON files are
created correctly, either method is safe and
effective.
 knife can be used to create data bags and
data bag items when the knife data bag sub-
command is run with the create argument.
For example:
knife data bag create DATA_BAG_NAME (DATA_BAG_ITEM)
 One or more data bags and data bag items can
be created manually under the data_bags
directory in the chef-repo. Any method can be
used to create the data bag folders and data bag
item JSON files. For example:

mkdir data_bags/admins
 A data bag item can be created manually in the
same way as the data bag, but by also specifying
the file name for the data bag item (this example
is using vi, a visual editor for UNIX):

data_bags/admins/charlie.json
 data_bags/admins/charlie.json
{
"id": "charlie",
"shell": "/bin/zsh",
"comment": "Crazy Charlie"
}

 data_bags/admins/stuart.json
{
"id": "stuart",
"shell": "/bin/zsh",
"comment": "smart stuart"
}
 As long as a file is in the correct directory
structure, knife will be able to find the data
bag and data bag item with only the name of
the data bag and data bag item. For example:

Knife data bag create BAG_NAME

knife data bag from file BAG_NAME ITEM_NAME.json
 cookbooks/admins/recipes/default.rb
admins = data_bag('admins')

admins.each do |login|
admin = data_bag_item('admins', login)
home = "/home/#{login}"

user(login) do
shell admin['shell']
comment admin['comment']
home home
supports :manage_home => true
end
end
Chef(Knife) Exercises
 Exercise to create EC2 instance using Chef.
 You must have AWS account to create minimum
on EC2 instance.
 Execute below command on new VM:

yum install gcc-c++ patch readline readline-devel zlib zlib-devel

yum install libyaml-devel libffi-devel openssl-devel make
yum install bzip2 autoconf automake libtool bison iconv-devel sqlite-devel
rpm -ivh ruby-2.3.3-1.el6.x86_64.rpm --force
gem install chef
gem install knife-ec2
 Add following lines in /root/.chef/knife.rb
knife[:aws_access_key_id] = "AKIAJQHVCIJKLC3KHJC"
knife[:aws_secret_access_key] = "khAIC7665SQWRBCCTmPUS2lZy8Rd3HadO8eymJu"
knife[:region] = "us-west-2“
knife ec2 flavor list
knife ec2 server list
knife ec2 server create -I ami-5ec1673e --ssh-key awsfirst -f t2.micro -
-ssh-user ubuntu --identity-file ~/.ssh/your-private-key
knife ec2 server delete
 Exercise to create Azure instance using Chef
knife.

yum install gcc-c++ patch readline readline-devel zlib zlib-devel

yum install libyaml-devel libffi-devel openssl-devel make
yum install bzip2 autoconf automake libtool bison iconv-devel sqlite-devel
rpm -ivh ruby-2.3.3-1.el6.x86_64.rpm --force
gem install chef
gem install knife-azure
 Download Azure Publish Settings File.

https://manage.windowsazure.com/PublishSettings/index?Client=&SchemaVers
ion=&DisplayTenantSelector=true
 Add following lines in .chef/knife.rb
knife[:azure_subscription_id] = "e1fe473-e94b-45d6-88d7-ae17e7bc5b4d"
knife[:azure_api_host_name] = "https://management.core.windows.net"
knife[:azure_mgmt_cert] = "management-certificate.pem"

 Get “azure_subscription_id” from publish

settings under “Id”.
 Get “azure_api_host_name” from publish settings
under “ServiceManagementUrl”.
 Generate “azure_mgmt_cert” by using
“ManagementCertificate” from publish settings
as follows:
 Open the downloaded xml file and copy the contents of of
managementcertificate tag. Copy the contents between the
double quotes and paste it to a new file called cert.pfx

 Use the following command to decode the pfx file.

base64 -d cert.pfx > decoded_cert.pfx

 Convert the decoded file to .pem file using the following

commands.

openssl pkcs12 -in decoded_cert.pfx -out management-certificate.pem -nodes

#after executing the above command it will ask for password.Just hit enter without
typing anything.

 Copy the created pem file to .chef folder and add the below
given credentials to the knife.rb file.
knife azure server list
knife azure server create
--azure-dns-name 'knifeazuredemo'
--azure-source-image
"a699494373c04fc0bc8f2bb1389d6106__Windows-Server-
2012-Datacenter-201306.01-en.us-127GB.vhd"
--winrm-password 'p@ssw0rd!'
--azure-service-location "East Asia"
 Contact @ http://www.atgensoft.com/
 Linkedin: @atgensoft-solutions-llp
 Twitter: @skedautomation
 FaceBook: @atgensoftsolutions
 Register: http://www.myguruzone.com/
 Our Products & Services:
◦ DevOps Training, Consulting & Implementation
◦ Sked Automation Software
◦ Ajar DBMS
◦ Automation Design & Solutions
Thank You !!