Learning Puppet

Table of Contents

Learning Puppet

Credits

About the Author

About the Reviewers

www.PacktPub.com

Support files, eBooks, discount offers, and more

Why subscribe?

Free access for Packt account holders

Preface

What this book covers

What you need for this book

Who this book is for

Conventions

Reader feedback

Customer support

Downloading the example code

Errata

Piracy

Questions

1. Puppet Development in Isolation

Puppet Manifests

Downloading Oracle VirtualBox

Downloading the Puppet Learning VM

Importing the Puppet Learning VM into VirtualBox

Virtual machine snapshots

Snapshot of the virtual machine

Puppet on command line

Puppet version

The open source Puppet

The Puppet Enterprise edition

Puppet resources

Managing resources from the command line

Puppet dry run

Use Puppet to examine the current state of resources

Puppet is run as a user root

Puppet DSL and manifests

Managing resources with the puppet apply command

Creating Puppet manifests

Idempotency

Puppet command line versus Puppet manifests

Managing files and directories with a file resource

Puppet configuration

Summary

2. Managing Packages in Puppet

Restoring a snapshot

Introducing VirtualBox Guest Additions

Shared folders

Host-only networking

Configuring shared folders

Configuring the host-only network interface

Testing shared folders

Using the file resource to configure network interface on the virtual machine

Restart networking

Creating the puppet-agent node

Take a snapshot of the virtual machine

Branch the virtual machine by creating a clone from the snapshot

Purging package resources

Summary

3. My First Puppet Module

Introducing the Puppet module

The Puppet module structure

The Puppet module name format

Generating a Puppet module

A Puppet class

Resources inside the Puppet class

Rename the module directory

Applying a Puppet class

Installing a module from Puppet Forge

Installing Apache HTTP Server

Testing the Apache HTTP Server

Customizing a web server with facts and templates

Puppet templates

Creating a template file

Creating a file resource for the template file

Facts by the facter

Facter on the command line

Accessing facts from the Puppet template

Accessing facts from Puppet manifests

A simple for loop in the Puppet template

Testing repeatable deployment

Shut down the virtual machine

Revert the machine state to the previous snapshot

Reduce memory allocation for the virtual machine

Create a linked virtual machine clone from the snapshot

Power on both the virtual machines and apply the webapp class

Summary

4. Monitoring Your Web Server

Monitoring the architecture

Creating a Nagios module for the client and server

A recap on the state of virtual machines and snapshots

Cloning the virtual machine for Nagios module development

Generating the Nagios module

Puppetize the Nagios Server installation

Configuring the Nagios Server web interface

Applying the nagios::server class

Verifying Nagios Server installation

Creating nagios::client class

Testing the nagios::client class

Enable monitoring on the web server

Configuring the web server host and checks on the Nagios Server

Creating a Nagios host

Summary

5. Load Balancing the Cluster

The parameterized class

Calling a class with parameters

Creating a parameterized class

The defined type

Calling the defined type

Creating the defined type

The load balancing architecture

Building the load balancer node

Cloning a new virtual machine for the load balancer

Reducing the virtual machine memory allocation

Creating a snapshot and starting the virtual machine

Creating a load balancer module

Installing the load balancer using class parameters

Deploying the load balancer

Verifying the load balancer deployment

Testing end-to-end functionality

Adding parameters to the loadbalancer class

Load balancing web server nodes

Enabling load balancing on the loadbalancer class

Applying and testing the load balancer

Launching the second web server node

Summary

6. Scaling Up the Puppet Environment

Puppet Master

The Puppet Master components

Certificate Authority for authorization

Mcollective for orchestration

PuppetDB for exported resources, PuppetDB queries, and reporting

Connecting Puppet Agent with Puppet Master

Creating the bootstrap module for Puppet Master and Puppet Agent

Configuring static IP address on Puppet Master

Defining resource processing order with the arrow notation

Creating class bootstrap::master

Referencing an out-of-scope variable from Puppet template

Conditional statements

The if statement

Creating site.pp file for node classification

Applying bootstrap class on Puppet Master

A first look at the Puppet Enterprise Console

Bypassing the certificate warning message

Logging on to the Puppet Enterprise Console

503 Service Temporarily Unavailable?

Creating a node group

Bootstrapping Puppet Agent

Applying the bootstrap::agent class via the bootstrap class

Signing the certificate on the Puppet Enterprise Console

Adding nodes to the node group

Deploying the Web Server node against Puppet Master

Bootstrapping Load Balancer and Nagios Server nodes

Summary

7. Making the Configuration Dynamic

An introduction to PuppetDB and exported resources

Exported resources

Exporting and importing resources

Exporting resources

Importing resources

Testing exported resources

Purging resources

Purging resources with the nagios::purge class

The PuppetDB query

Installing the dalen-puppetdbquery module

Adding puppetdbquery into the RUBYLIB environment variable

Examples of Puppet query commands on the command line

Querying certname with action nodes

Querying facts with action facts

Using the puppetdbquery functions

The query_nodes function

The query_facts function

Creating a custom type for testing PuppetDB queries

Using the PuppetDB query to configure the load balancer

Testing the PuppetDB query manifests on the load balancer node

Summary

8. Extending Puppet

Puppet functions

Creating a Puppet module for custom functions

Writing a function

Test-driving the Puppet function

Testing a Puppet function on the Puppet Agent node

Testing a Puppet function against Puppet Master

Distributing SSH keys with a Puppet function

Creating a public and private key pair

Writing a Puppet function to distribute a public key

Calling the custom function from the file resource

Testing the password-less SSH session

Creating custom facts

External facts

Creating facts that return structured data

Writing custom facts in Ruby

Creating a custom fact to extract certname

Distributing certname records across the cluster

Summary

9. The Puppet Enterprise Console

Role-based Access Control

Creating a user and assigning a role

Creating a user account

Enabling login for a user account

Assigning a role to the account

Creating a node group

Signing the Puppet agent's certificate

Adding a node to the node group

Classifying nodes in the Puppet Enterprise Console

Moving the site.pp file temporarily out of the modulepath

Using Live Management

Accessing reports via the Puppet Enterprise Console

Searching nodes with Inventory Search

Summary

10. Troubleshooting Puppet

Prerequisites

Troubleshooting node definition issues

Making the node definition file and ENC work concurrently

Diagnosing duplicate declaration errors

Using the defined() function to avoid duplicate declarations

Using the $name variable in custom types

Getting around dependency cycle errors

Troubleshooting missing resources

Diagnosing template errors

Diagnosing missing source file errors

Rectifying certificate errors

Listing certificates on the Puppet Master

Removing Puppet certificate on Puppet Master

Regenerating Certificate Signing Request

Signing a certificate on the command line

Finding help online

Summary

Index

Learning Puppet

Learning Puppet

Copyright © 2015 Packt Publishing

All rights reserved. No part of this book may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, without the prior written permission of the publisher, except in the case of brief quotations embedded in critical articles or reviews.

Every effort has been made in the preparation of this book to ensure the accuracy of the information presented. However, the information contained in this book is sold without warranty, either express or implied. Neither the author nor Packt Publishing, and its dealers and distributors will be held liable for any damages caused or alleged to be caused directly or indirectly by this book.

Packt Publishing has endeavored to provide trademark information about all of the companies and products mentioned in this book by the appropriate use of capitals. However, Packt Publishing cannot guarantee the accuracy of this information.

First published: August 2015

Production reference: 1270815

Published by Packt Publishing Ltd.

Livery Place

35 Livery Street

Birmingham B3 2PB, UK.

ISBN 978-1-78439-983-2

www.packtpub.com

Credits

Author

Jussi Heinonen

Reviewers

Vlastimil Holer

Ashish Jaiswal

Amar Krishna

Eric Stonfer

Commissioning Editor

Kartikey Pandey

Acquisition Editor

Nikhil Karkal

Content Development Editor

Anish Sukumaran

Technical Editors

Dhiraj Chandanshive

Pramod Kumavat

Copy Editors

Janbal Dharmaraj

Rashmi Sawant

Project Coordinator

Izzat Contractor

Proofreader

Safis Editing

Indexer

Hemangini Bari

Graphics

Sheetal Aute

Production Coordinator

Shantanu N. Zagade

Cover Work

Shantanu N. Zagade

About the Author

Jussi Heinonen is a seasoned systems developer and an open source enthusiast who lives in Cambridge, UK. He has been working for various software businesses in the media and telecommunications sectors since 1998. During this period, he acquired a few Linux certifications, and more recently, in 2014 and 2015, he passed the Puppet Certified Professional exam. He currently works at the Financial Times as a senior integration engineer. His role revolves around building and designing software delivery pipelines that enable developers to create high-quality software quickly and frequently. In his spare time, he likes to spend time with his family and loves to watch games at Arsenal Football Club.

There are many people who have contributed to this book, and I'd like to take this opportunity to say thanks to them.

First and foremost, a big thank you to the folks at Packt Publishing, especially Nikhil and Anish, for giving me the opportunity to write this book. Your support and guidance throughout the writing process has been invaluable.

Secondly, a huge credit to the reviewers, in particular, Vlastimil Holer, who have helped me streamline the content of this book and improve the reading experience.

Thirdly, I'd like to thank my lovely wife, Thury, and my children, Markus, Jakob, and Elisa, for allowing me to take occasional breaks from daddy duties to concentrate on writing this book.

Finally, I would like to thank my colleagues at the Financial Times, namely the members of the Integration Engineering team and Team CMS. In the past years, I've been working with them in various projects involving Puppet. This has enabled me to hone my Puppet skills and learn how to use this tool to solve specific business problems.

About the Reviewers

Vlastimil Holer is a systems engineer who focuses on automation. He has worked with Unix-like systems for more than a decade, and he first used Puppet in 2008 while preparing and managing the growing deployment of the GoodData cloud BI on Amazon EC2. Currently, he is working on the CERIT Scientific Cloud project at Masaryk University, where he manages and automates the computing, cloud, and storage infrastructures.

Ashish Kumar Jaiswal has been working for the past 4 and a half years and has worked on Puppet for almost 4 years. Puppet was the root cause for the growth of his technical career.

He is currently working on a project called obmondo.com—an Operations as a service project—using Puppet to automate server configuration and management. This project sets up the whole Puppet infrastructure without a Puppet server, and it's just far off to click on the profile you want your server to have.

I would like to thank Corey Ralph, an Aussie guy who was my manager at my previous organization, and my wife, Dhara Jaiswal. She is just too kind at heart. I would also like to thank my beautiful family, which includes my mom, dad, and two sisters.

Amar Krishna is a DevOps professional and loves to automate everything that comes his way. He has used Puppet and scripting for automation. He started his career with Linux and PHP and moved on to high-performance computing, where he worked with one of the largest clusters in India. Then, he moved on to the cloud computing world, where he worked on tools such as CloudStack and OpenStack. He was involved in one of the biggest cloud projects in India. Currently, he is working at Reliance Jio Infocomm.

This was his first book as a reviewer and he loved it. He would like to review more books in the future.

I would really like to thank all my colleagues for helping me.

www.PacktPub.com

Support files, eBooks, discount offers, and more

For support files and downloads related to your book, please visit www.PacktPub.com.

Did you know that Packt offers eBook versions of every book published, with PDF and ePub files available? You can upgrade to the eBook version at www.PacktPub.com and as a print book customer, you are entitled to a discount on the eBook copy. Get in touch with us at for more details.

At www.PacktPub.com, you can also read a collection of free technical articles, sign up for a range of free newsletters and receive exclusive discounts and offers on Packt books and eBooks.

https://www2.packtpub.com/books/subscription/packtlib

Do you need instant solutions to your IT questions? PacktLib is Packt's online digital book library. Here, you can search, access, and read Packt's entire library of books.

Why subscribe?

Fully searchable across every book published by Packt

Copy and paste, print, and bookmark content

On demand and accessible via a web browser

Free access for Packt account holders

If you have an account with Packt at www.PacktPub.com, you can use this to access PacktLib today and view 9 entirely free books. Simply use your login credentials for immediate access.

Preface

This book is a step-by-step guide to get started with Puppet development, and use Puppet modules as the building blocks to deploy production-ready application clusters in the virtual environment.

The journey begins with the installation of the development environment on the VirtualBox hypervisor and the installation of the Puppet Learning VM that will be used platforms to test and develop Puppet modules.

You will learn how to manage virtual machines and snapshots effectively and enhance the developer's experience with advanced VirtualBox features.

Once the development environment is up and running, this book will focus on Puppet module development in detail. You will be guided through the process of how to utilize the existing modules that are available in the public module repository, write your own modules, and use modules to deploy a real-world web application that includes features such as monitoring and load balancing. When an application cluster is deployed, the focus shifts to how to scale the environment and turn the static configuration into a dynamic one through stored configurations and PuppetDB. The latter part of the book will provide you with practical advice on Puppet troubleshooting, and how to manage your environment with a wealth of features provided by the Puppet Enterprise Console. Starting from the basics, this step-by-step guide will walk you through the process of becoming the master of your own Puppets.

What this book covers

Chapter 1, Puppet Development in Isolation, teaches you how to set up the local Puppet development environment quickly and start experimenting with Puppet on the command line in a matter of minutes.

Chapter 2, Managing Packages in Puppet, shows you how to restore the virtual machine snapshot and set up shared folders and host-only networking to enhance the developer's experience. You can also learn how to purge a software package using Puppet.

Chapter 3, My First Puppet Module, explains the concept of Puppet modules. You will learn how to install third-party Puppet modules from Puppet Forge and how to create and use your own modules.

Chapter 4, Monitoring Your Web Server, teaches you how to use Puppet to install the Nagios monitoring server and how to add a web server to monitor using Puppet.

Chapter 5, Load Balancing the Cluster, introduces you to parameterized classes and defined types and teaches you to use these to configure load balancing in the cluster.

Chapter 6, Scaling Up the Puppet Environment, introduces you to the Puppet Master, which enables you to centrally manage a large number of Puppet Agent nodes. You will learn how to sign Puppet Agent certificates and join agents to the Puppet environment.

Chapter 7, Making the Configuration Dynamic, teaches you how to use Puppet's Exported Resources to pass Puppet resources between nodes. This chapter also introduces you to the PuppetDB queries that are used to discover services in the cluster.

Chapter 8, Extending Puppet, teaches you how to extend Puppet beyond its built-in functionality. This chapter introduces you to custom facts and functions.

Chapter 9, The Puppet Enterprise Console, explores the Puppet Enterprise Console, which is a web-based management console that runs on the Puppet Master node. In this chapter, we will learn about Role-based Access Control and how to classify nodes using the External Node Classifier.

Chapter 10, Troubleshooting Puppet, teaches you how to identify the most common issues in Puppet and how to tackle them. This chapter provides you with basic troubleshooting skills.

What you need for this book

A computer that runs a Windows, Mac, or Linux operating system. The computer should have a minimum of 4 GB of memory and 10 GB of free hard drive space.

Who this book is for

This book is aimed at people who are new to configuration management and IT automation processes. You may have a background in software development, and you may have set yourself a goal of learning how to take full control of the software deployment process; or perhaps, you are more experienced in the system administration field, and are looking for better ways to manage system configuration changes at scale. Although previous experience in IT is helpful, it is not a requirement. This book will get you up to speed with Puppet development quickly and effortlessly.

Conventions

In this book, you will find a number of styles of text that distinguish between different kinds of information. Here are some examples of these styles, and an explanation of their meaning.

Code words in text, database table names, folder names, filenames, file extensions, pathnames, dummy URLs, user input, and Twitter handles are shown as follows: Experiment with the Puppet command-line commands puppet describe, puppet resource, and puppet apply.

A block of code is set as follows:

file {

  '/etc/sysconfig/network-scripts/ifcfg-eth1':

    content =>

'DEVICE=eth1

BOOTPROTO=dhcp

ONBOOT=yes',

}

Any command-line input or output is written as follows:

puppet describe --list | less

New terms and important words are shown in bold. Words that you see on the screen, in menus or dialog boxes for example, appear in the text like this: Start the Oracle VM VirtualBox Manager and select Import Appliance from the File menu.

Note

Warnings or important notes appear in a box like this.

Tip

Tips and tricks appear like this.

Reader feedback

Feedback from our readers is always welcome. Let us know what you think about this book—what you liked or may have disliked. Reader feedback is important for us to develop titles that you really get the most out of.

To send us general feedback, simply send an e-mail to <feedback@packtpub.com>, and mention the book title via the subject of your message.

If there is a topic that you have expertise in and you are interested in either writing or contributing to a book, see our author guide at www.packtpub.com/authors.

Customer support

Now that you are the proud owner of a Packt book, we have a number of things to help you to get the most from your purchase.

Downloading the example code

You can download the example code files from your account at http://www.packtpub.com for all the Packt Publishing books you have purchased. If you purchased this book elsewhere, you can visit http://www.packtpub.com/support and register to have the files e-mailed directly to you.

Errata

Although we have taken every care to ensure the accuracy of our content, mistakes do happen. If you find a mistake in one of our books—maybe a mistake in the text or the code—we would be grateful if you could report this to us. By doing so, you can save other readers from frustration and help us improve