Linux in Action

Copyright

For online information and ordering of this and other Manning books, please visit www.manning.com. The publisher offers discounts on this book when ordered in quantity. For more information, please contact

       Special Sales Department

       Manning Publications Co.

       20 Baldwin Road

       PO Box 761

       Shelter Island, NY 11964

       Email: 

orders@manning.com

©2018 by Manning Publications Co. All rights reserved.

No part of this publication may be reproduced, stored in a retrieval system, or transmitted, in any form or by means electronic, mechanical, photocopying, or otherwise, without prior written permission of the publisher.

Many of the designations used by manufacturers and sellers to distinguish their products are claimed as trademarks. Where those designations appear in the book, and Manning Publications was aware of a trademark claim, the designations have been printed in initial caps or all caps.

Recognizing the importance of preserving what has been written, it is Manning’s policy to have the books we publish printed on acid-free paper, and we exert our best efforts to that end. Recognizing also our responsibility to conserve the resources of our planet, Manning books are printed on paper that is at least 15 percent recycled and processed without the use of elemental chlorine.

Development editor: Frances Lefkowitz

Review editor: Ivan Martinović

Technical development editor: John Guthrie

Project manager: Deirdre Hiam

Copyeditor: Frances Buran

Proofreader: Tiffany Taylor

Technical proofreader: Reka Horvath

Typesetter: Gordan Salinovic

Cover designer: Marija Tudor

ISBN 9781617294938

Printed in the United States of America

1 2 3 4 5 6 7 8 9 10 – DP – 23 22 21 20 19 18

Brief Table of Contents

Copyright

Brief Table of Contents

Table of Contents

Preface

Acknowledgments

About this book

About the author

About the cover illustration

Chapter 1. Welcome to Linux

Chapter 2. Linux virtualization: Building a Linux working environment

Chapter 3. Remote connectivity: Safely accessing networked machines

Chapter 4. Archive management: Backing up or copying entire file systems

Chapter 5. Automated administration: Configuring automated offsite backups

Chapter 6. Emergency tools: Building a system recovery device

Chapter 7. Web servers: Building a MediaWiki server

Chapter 8. Networked file sharing: Building a Nextcloud file-sharing server

Chapter 9. Securing your web server

Chapter 10. Securing network connections: Creating a VPN or DMZ

Chapter 11. System monitoring: Working with log files

Chapter 12. Sharing data over a private network

Chapter 13. Troubleshooting system performance issues

Chapter 14. Troubleshooting network issues

Chapter 15. Troubleshooting peripheral devices

Chapter 16. DevOps tools: Deploying a scripted server environment using Ansible

 Conclusion

 Appendix. A chapter-by-chapter, command-line review

 Linux in Action topics

 Networking troubleshooting workflow

Index

List of Figures

List of Tables

List of Listings

Table of Contents

Copyright

Brief Table of Contents

Table of Contents

Preface

Acknowledgments

About this book

About the author

About the cover illustration

Chapter 1. Welcome to Linux

1.1. What makes Linux different from other operating systems

1.2. Basic survival skills

1.2.1. The Linux file system

1.2.2. Getting around: Linux navigation tools

1.2.3. Getting things done: Linux file management tools

1.2.4. Keyboard tricks

1.2.5. Pseudo file systems

1.2.6. Showing ’em who’s boss: sudo

1.3. Getting help

1.3.1. Man files

1.3.2. Info

1.3.3. The internet

Summary

Key terms

Security best practices

Command-line review

Test yourself

Answer key

Chapter 2. Linux virtualization: Building a Linux working environment

2.1. What is virtualization?

2.2. Working with VirtualBox

2.2.1. Working with Linux package managers

2.2.2. Defining a virtual machine (VM)

2.2.3. Installing an operating system (OS)

2.2.4. Cloning and sharing a VirtualBox VM

2.3. Working with Linux containers (LXC)

2.3.1. Getting started with LXC

2.3.2. Creating your first container

Summary

Key terms

Security best practices

Command-line review

Test yourself

Answer key

Chapter 3. Remote connectivity: Safely accessing networked machines

3.1. The importance of encryption

3.2. Getting started with OpenSSH

3.3. Logging in to a remote server with SSH

3.4. Password-free SSH access

3.4.1. Generating a new key pair

3.4.2. Copying the public key over a network

3.4.3. Working with multiple encryption keys

3.5. Safely copying files with SCP

3.6. Using remote graphic programs over SSH connections

3.7. Linux process management

3.7.1. Viewing processes with the ps command

3.7.2. Working with systemd

Summary

Key terms

Security best practices

Command-line review

Test yourself

Answer key

Chapter 4. Archive management: Backing up or copying entire file systems

4.1. Why archive?

4.1.1. Compression

4.1.2. Archives: Some important considerations

4.2. What to archive

4.3. Where to back up

4.4. Archiving files and file systems using tar

4.4.1. Simple archive and compression examples

4.4.2. Streaming file system archives

4.4.3. Aggregating files with find

4.4.4. Preserving permissions and ownership...and extracting archives

4.5. Archiving partitions with dd

4.5.1. dd operations

4.5.2. Wiping disks with dd

4.6. Synchronizing archives with rsync

4.7. Planning considerations

Summary

Key terms

Security best practices

Command-line review

Test yourself

Answer key

Chapter 5. Automated administration: Configuring automated offsite backups

5.1. Scripting with Bash

5.1.1. A sample script for backing up system files

5.1.2. A sample script for changing filenames

5.2. Backing up data to AWS S3

5.2.1. Installing the AWS command-line interface (CLI)

5.2.2. Configuring your AWS account

5.2.3. Creating your first bucket

5.3. Scheduling regular backups with cron

5.4. Scheduling irregular backups with anacron

5.4.1. Running the S3 sync job

5.5. Scheduling regular backups with systemd timers

Summary

Key terms

Security best practices

Command-line review

Test yourself

Answer key

Chapter 6. Emergency tools: Building a system recovery device

6.1. Working in recovery/rescue mode

6.1.1. The GRUB bootloader

6.1.2. Using recovery mode on Ubuntu

6.1.3. Using rescue mode on CentOS

6.1.4. Finding command-line rescue tools

6.2. Building a live-boot recovery drive

6.2.1. System rescue images

6.2.2. Writing live-boot images to USB drives

6.3. Putting your live-boot drive to work

6.3.1. Testing system memory

6.3.2. Damaged partitions

6.3.3. Recovering files from a damaged file system

6.4. Password recovery: Mounting a file system using chroot

Summary

Key terms

Security best practices

Command-line review

Test yourself

Answer key

Chapter 7. Web servers: Building a MediaWiki server

7.1. Building a LAMP server

7.2. Manually setting up an Apache web server

7.2.1. Installing the Apache web server on Ubuntu

7.2.2. Populating your website document root

7.3. Installing an SQL database

7.3.1. Hardening SQL

7.3.2. SQL administration

7.4. Installing PHP

7.4.1. Installing PHP on Ubuntu

7.4.2. Testing your PHP installation

7.5. Installing and configuring MediaWiki

7.5.1. Troubleshooting missing extensions

7.5.2. Connecting MediaWiki to the database

7.6. Installing the Apache web server on CentOS

7.6.1. Understanding network ports

7.6.2. Controlling network traffic

7.6.3. Installing MariaDB on CentOS

7.6.4. Installing PHP on CentOS

Summary

Key terms

Security best practices

Command-line review

Test yourself

Answer key

Chapter 8. Networked file sharing: Building a Nextcloud file-sharing server

8.1. Enterprise file sharing and Nextcloud

8.2. Installing Nextcloud using snaps

8.3. Installing Nextcloud manually

8.3.1. Hardware prerequisites

8.3.2. Building a LAMP server

8.3.3. Configuring Apache

8.3.4. Downloading and unpacking Nextcloud

8.4. Nextcloud administration

8.5. Using AWS S3 as the primary Nextcloud storage

Summary

Key terms

Security best practices

Command-line review

Test yourself

Answer key

Chapter 9. Securing your web server

9.1. The obvious stuff

9.2. Controlling network access

9.2.1. Configuring a firewall

9.2.2. Using nonstandard ports

9.3. Encrypting data in transit

9.3.1. Preparing your website domain

9.3.2. Generating certificates using Let’s Encrypt

9.4. Hardening the authentication process

9.4.1. Controlling file system objects with SELinux

9.4.2. Installing and activating SELinux

9.4.3. Applying SELinux policies

9.4.4. System groups and the principle of least privilege

9.4.5. Isolating processes within containers

9.4.6. Scanning for dangerous user ID values

9.5. Auditing system resources

9.5.1. Scanning for open ports

9.5.2. Scanning for active services

9.5.3. Searching for installed software

Summary

Key terms

Command-line review

Test yourself

Answer key

Chapter 10. Securing network connections: Creating a VPN or DMZ

10.1. Building an OpenVPN tunnel

10.1.1. Configuring an OpenVPN server

10.1.2. Configuring an OpenVPN client

10.1.3. Testing your VPN

10.2. Building intrusion-resistant networks

10.2.1. Demilitarized zones (DMZs)

10.2.2. Using iptables

10.2.3. Creating a DMZ using iptables

10.2.4. Creating a DMZ using Shorewall

10.3. Building a virtual network for infrastructure testing

Summary

Key terms

Command-line review

Test yourself

Answer key

Chapter 11. System monitoring: Working with log files

11.1. Working with system logs

11.1.1. Logging with journald

11.1.2. Logging with syslogd

11.2. Managing log files

11.2.1. The journald way

11.2.2. The syslogd way

11.3. Consuming large files

11.3.1. Using grep

11.3.2. Using awk

11.3.3. Using sed

11.4. Monitoring with intrusion detection

11.4.1. Setting up a mail server

11.4.2. Installing Tripwire

11.4.3. Configuring Tripwire

11.4.4. Generating a test Tripwire report

Summary

Key terms

Security best practices

Command-line review

Test yourself

Answer key

Chapter 12. Sharing data over a private network

12.1. Sharing files through Network File System (NFS)

12.1.1. Setting up the NFS server

12.1.2. Setting up the client

12.1.3. Mounting an NFS share at boot time

12.1.4. NFS security

12.2. Sharing files with Windows users using Samba

12.2.1. Testing your Samba configuration

12.2.2. Accessing a Samba server from Windows

12.3. Sharing files with yourself using symbolic links

Summary

Key terms

Security best practices

Command-line review

Test yourself

Answer key

Chapter 13. Troubleshooting system performance issues

13.1. CPU load problems

13.1.1. Measuring CPU load

13.1.2. Managing CPU load

13.1.3. Making trouble (simulating CPU load)

13.2. Memory problems

13.2.1. Assessing memory status

13.2.2. Assessing swap status

13.3. Storage availability problems

13.3.1. Inode limits

13.3.2. The solution

13.4. Network load problems

13.4.1. Measuring bandwidth

13.4.2. Solutions

13.4.3. Shaping network traffic with tc

13.5. Monitoring tools

13.5.1. Aggregating monitoring data

13.5.2. Visualizing your data

Summary

Key terms

Security best practices

Command-line review

Test yourself

Answer key

Chapter 14. Troubleshooting network issues

14.1. Understanding TCP/IP addressing

14.1.1. What’s NAT addressing?

14.1.2. Working with NAT addressing

14.2. Establishing network connectivity

14.3. Troubleshooting outbound connectivity

14.3.1. Tracking down the status of your network

14.3.2. Assigning IP addresses

14.3.3. Configuring DNS service

14.3.4. Plumbing

14.4. Troubleshooting inbound connectivity

14.4.1. Internal connection scanning: netstat

14.4.2. External connection scanning: netcat

Summary

Key terms

Security best practices

Command-line review

Test yourself

Answer key

Chapter 15. Troubleshooting peripheral devices

15.1. Identifying attached devices

15.2. Managing peripherals with Linux kernel modules

15.2.1. Finding kernel modules

15.2.2. Manually loading kernel modules

15.3. Manually managing kernel parameters at boot time

15.3.1. Passing parameters at boot time

15.3.2. Passing parameters via the file system

15.4. Managing printers

15.4.1. Basics of lp

15.4.2. Managing printers using CUPS

Summary

Key terms

Security best practices

Command-line review

Test yourself

Answer key

Chapter 16. DevOps tools: Deploying a scripted server environment using Ansible

16.1. What deployment orchestrators can do for you

16.2. Ansible: Installation and setup

16.2.1. Setting up passwordless access to hosts

16.2.2. Organizing Ansible hosts

16.2.3. Testing connectivity

16.3. Authentication

16.4. Ansible playbooks

16.4.1. Writing a simple playbook

16.4.2. Creating multi-tiered, role-powered playbooks

16.4.3. Managing passwords in Ansible

Summary

Key terms

Security best practices

Command-line review

Test yourself

Answer key

 Conclusion

What you’ve learned

Virtualization

Connectivity

Encryption

Networking

Image management

System monitoring

What’s next

Resources

 Appendix. A chapter-by-chapter, command-line review

1. Welcome to Linux

2. Linux virtualization: Building a Linux working environment

3. Remote connectivity: Safely accessing networked machines

4. Archive management: Backing up or copying entire file systems

5. Automated administration: Configuring automated offsite backups

6. Emergency tools: Building a system recovery device

7. Web servers: Building a MediaWiki server

8. Networked file sharing: Building a Nextcloud file-sharing server

9. Securing your web server

10. Securing network connections: Creating a VPN or DMZ

11. System monitoring: Working with log files

12. Sharing data over a private network

13. Troubleshooting system performance issues

14. Troubleshooting network issues

15. Troubleshooting peripheral devices

16. DevOps tools: Deploying a scripted server environment using Ansible

 Linux in Action topics

 Networking troubleshooting workflow

Index

List of Figures

List of Tables

List of Listings

Preface

No matter what you do or how long you’ve been doing it in the IT or programming world, if you’re not learning new stuff, you’re probably not doing it right. It’s not that the platforms and paradigms are constantly changing. Nor is it that new business demands require fresh thinking. Or that the bad guys are constantly coming up with new ways to attack your servers. It’s all of those things and more. You can’t afford to stop learning. The trick is finding a way to learn the high-priority skills without turning the experience into a major detour.

It’s my intention and desire that you should be able to read even a single chapter from this book, Linux in Action, and walk away feeling confident enough to take on something challenging and productive—something you wouldn’t previously have even considered. If you hang around until the bitter end, you’ll learn to work with critical and current technologies powering virtualization, disaster recovery, infrastructure security, data backups, web servers, DevOps, and system troubleshooting.

But why Linux? Because Linux powers most of the internet, most scientific research, and most commerce—in fact, most of the world’s servers. Those servers need to be provisioned, launched, secured, and managed effectively by smart and well-trained people. Smart is what you bring to the table, and I think I can help with well trained.

Not sure you know enough about Linux to embark on such an ambitious project? Chapter 1 will quickly fill in the holes. After that, fasten your seat belt and prepare for a serious learning curve.

Acknowledgments

It’s impossible to reach the end of a book’s long and sometimes tortured production cycle without reflecting on what it took to make it through. In the case of Linux in Action—as with my Learn Amazon Web Services in a Month of Lunches—survival required the talent and dedication of every part of the Manning team’s deep bench.

Once again Frances Lefkowitz, as development editor, added significant clarity and purpose to each chapter, relentlessly keeping me focused and on track. Both Reka Horvath and John Guthrie patiently tested all the book’s projects and added valuable operational insights along the way. The copy editor, Frances Buran, seems never to have met an adverb she approves—at least not when used by me. But the accuracy and grace of the text in its current form clearly indicate the quality of her judgment.

In her role as project manager, Deirdre Hiam effectively shepherded us through the last mile, successfully keeping all the many moving parts in sync. Each of the book’s peer reviewers has left an important mark. They may not realize it, but all of their valuable observations were carefully noted, weighed, and, where possible, applied. Many thanks, therefore, to Angelo Costo, Christopher Phillips, Dario Victor Duran, Flayol Frederic, Foster Haines, George L. Gaines, Gustavo Patino, Javier Collado, Jens Christian B. Madsen, Jonas Medina de los Reyes, Maciej Jurkowski, Mayer Patil, Mohsen Mostafa Jokar, and Tim Kane.

This book is about more than just Linux administration skills. It also tries to impart the larger sense of responsibility successful administrators have for the servers and systems under their care. I was lucky to have benefited from a great mentor at the start of my career as a Linux system administrator. Peter Fedorow’s attention to both fine operational details and the big global picture make him an especially effective admin. His dragging me kicking and screaming into the world of Linux virtualization hooked me on containers long before containers were cool. When everything’s said and done, at least some of Peter’s guidance is, no doubt, reflected here.

And finally, none of my professional (or private) projects would get off the blocks without the cheerful and helpful participation of my dear wife. We fully share the hard work, but the successes are mostly her doing.

About this book

Looking to learn to administer Linux computers? Excellent choice. While it can hold its own in the consumer desktop space, where Linux absolutely dominates is the world of servers, especially virtual and cloud servers. Because most serious server administration these days takes place remotely, working through a GUI interface of one sort or another just adds unnecessary overhead. If you want to manage the servers and network architectures that are currently attracting all the attention, you’re going to have to learn your way around the Linux command line.

The good news is that the core Linux command set is going to work for you across geographic and corporate lines, just about anywhere computers and business intersect. The better news is that, relatively speaking, Linux skills have staying power. Because it’s such a mature and stable operating system, most of the tools used a quarter century ago are still just as effective as ever, and most of the tools used today will probably still be actively used after another quarter century. Learning Linux, in other words, is a lifelong investment.

But you’re busy and you’ve got deadlines. Well, I can’t promise you that mastering Linux will be as simple as learning to tie your shoes. But I can help you focus like a laser so you can leave all the stuff you don’t need lying on the highway, choking on your exhaust fumes (assuming you’re not driving a Tesla, of course).

How am I going to pull that one off? Linux in Action turns technology training sideways. That is, while other books, courses, and online resources organize their content around categories (Alright boys and girls, everyone take out your slide rules and charcoal pencils. Today we’re going to learn about Linux file systems.), I’m going to use real-world projects to teach.

So, for example, I could have built an entire chapter (or two) on Linux file systems. But instead, you’ll learn how to build enterprise file servers, system recovery drives, and scripts to replicate archives of critical data. In the process, you’ll pick up the file system knowledge as a free bonus.

Don’t think I’m going to cover every Linux administration tool. That’s impossible: there are literally thousands of them out there. But don’t worry. The core skills and functionality needed through the first years of a career in Linux administration will be covered, and covered well, but only when needed for a practical, mission-critical project. When you’re done, you’ll have learned no less than what you would have from a traditional source, but you’ll also know how to complete more than a dozen major administrative projects, and be comfortable tackling dozens more.

Are you in? I thought so.

Who should read this book

This book is designed to help you acquire a solid range of Linux administration skills. Perhaps you’re a developer who wants to work more directly with the server environment within which your applications will live. Or maybe you’re ready to make your move in the server administration or DevOps worlds. Either way, you belong with us.

What should you already know? At the very least, you should be comfortable working with the files, networks, and basic resources of a modern operating system. Experience with system administration, network management, and programming languages definitely won’t hurt, but are not required. Most of all, you should be unafraid of exploring new environments and enthusiastic about experimenting with new tools. One more thing: you’re expected to know how to perform a simple and straightforward installation of a Linux operating system.

How this book is organized: A roadmap

Just a few words about the way the book is built. Each chapter of Linux in Action covers one or two practical projects—except chapter 1. Chapter 1, because it’s designed to fill in any very basic gaps that might exist in your Linux knowledge, will be different from all the others. Don’t need the basics? I’m absolutely sure you’ll find lots of fun new toys to play with in chapter 2.

Along with the book’s projects, I’ll also introduce you to the individual skills and tools that you’ll need. In addition, each chapter’s projects usually build on the skills you’ve learned previously in the book. Just to show you that I mean business, here’s a fairly complete list of the main projects (under the Chapter heading), skill domains, and tools you’ll meet through the course of the book:

About the code

This book contains many examples of source code both in numbered listings and in line with normal text. In both cases, source code is formatted in a fixed-width font like this to separate it from ordinary text.

In many cases, the original source code has been reformatted; we’ve added line breaks and reworked indentation to accommodate the available page space in the book. In rare cases, even this was not enough, and listings include line-continuation markers ( ). Additionally, comments in the source code have often been removed from the listings when the code is described in the text. Code annotations accompany many of the listings, highlighting important concepts.

Linux distributions

There are currently dozens of actively maintained Linux distributions. Even though most of the Linux basics are common to all distros (distributions), there’ll always be little things that’ll work here but not there. For practicality’s sake, I’m going to concentrate mostly on two distributions: Ubuntu and CentOS. Why those two? Because each represents an entire family of distributions. Ubuntu shares its roots with Debian, Mint, Kali Linux, and others, while CentOS enjoys the company of Red Hat Enterprise Linux (RHEL) and Fedora.

That’s not to say I don’t value other distros like Arch Linux, SUSE, and Gentoo, or that what you’ll learn in this book won’t help you work with those environments. But fully covering the Ubuntu and CentOS families means grabbing the largest single slice of the Linux pie that I could reach using just two distributions.

Book forum

Purchase of Linux in Action includes free access to a private web forum run by Manning Publications where you can make comments about the book, ask technical questions, and receive help from the author and from other users. To access the forum, go to https://forums.manning.com/forums/linux-in-action. You can also learn more about Manning’s forums and the rules of conduct at https://forums.manning.com/forums/about.

Manning’s commitment to our readers is to provide a venue where a meaningful dialogue between individual readers and between readers and the author can take place. It is not a commitment to any specific amount of participation on the part of the author, whose contribution to the forum remains voluntary (and unpaid). We suggest you try asking the author some challenging questions lest his interest stray! The forum and the archives of previous discussions will be accessible from the publisher’s website as long as the book is in print.

Other online resources

Stuck? Web search is your best friend, as it can quickly connect you with a wealth of existing Linux guides and troubleshooting expertise. But you shouldn’t forget the StackExchange family of sites and, in particular, serverfault.com. If something’s gone wrong with some system configuration or the network has disappeared, then the odds are high that someone else has experienced the same thing, asked about it on ServerFault, and received an answer already. Nothing yet? Then ask the question yourself. LinuxQuestions.org and ubuntuforums.org can also be helpful.

And those who enjoy video training will find a good range of Linux courses on Pluralsight.com, including more than a dozen of my own courses.

About the author

DAVID CLINTON is a system administrator, teacher, and writer. He has administered, written about, and created training material for many important technology subjects including Linux systems, cloud computing (AWS in particular), and container technologies like Docker. He’s the author of Learn Amazon Web Services in a Month of Lunches (Manning, 2017). Many of his video training courses can be found on https://www.pluralsight.com/, and links to his other books (on Linux administration and server virtualization) can be found at https://bootstrap-it.com.

About the cover illustration

The figure on the cover of Linux in Action is captioned Habit of an Armenian native of Persia in 1700. The illustration is taken from Thomas Jefferys’ A Collection of the Dresses of Different Nations, Ancient and Modern (four volumes), London, published between 1757 and 1772. These are hand-colored copperplate engravings, heightened with gum arabic. Thomas Jefferys (1719–1771) was called Geographer to King George III. He was an English cartographer who was the leading map supplier of his day. His work as a map maker sparked an interest in local dress customs of the lands he surveyed and mapped, which are brilliantly displayed in this collection.

Fascination with faraway lands and travel for pleasure were relatively new phenomena in the late 18th century, and collections such as this one were popular. The diversity of the drawings in Jefferys’ volumes speaks vividly of the uniqueness and individuality of the world’s nations some 200 years ago. Dress codes have changed since then, and the diversity by region and country, so rich at the time, has faded away. It is now often hard to tell the inhabitant of one continent from another. Perhaps, trying to view it optimistically, we have traded a cultural and visual diversity for a more varied personal life. Or a more varied and interesting intellectual and technical life.

At a time when it is hard to tell one computer book from another, Manning celebrates the inventiveness and initiative of the computer business with book covers based on the rich diversity of regional life of two centuries ago, brought back to life by Jeffreys’ pictures.

Chapter 1. Welcome to Linux

This chapter covers

What makes Linux different

Basic survival skills

Getting help

This book turns technology training sideways. Although other books, courses, and online resources organize their content around skills, I’m going to use real-world projects as teaching tools. Each of the core skills and the functionality of Linux systems will be covered—and covered well—but only when needed for a project. When you’re done, you’ll have learned everything you would have from a traditional source, but you’ll also know how to perform more than a dozen vital and sophisticated administration tasks and be comfortable tackling dozens more.

The first two or three chapters will quickly guide you through your initial introduction to the world of Linux servers. After that, you’ll work through and adapt practical hands-on projects, and nothing but practical hands-on projects. Through those projects, you’ll learn more than just commands and skills. Get ready to dive deeper and to eventually create solutions to your own business problems.

No single book can anticipate all the challenges that you’ll face throughout your career. But, by demonstrating how to approach real-world problems using real-world tools, this book will make it a whole lot easier for you to use the vast resources available through both inline documentation and the internet. If your prior Linux experience is limited, this chapter introduces some basic command-line survival skills and points to places you can go for help when things don’t work.

Note

As you’ll see, a command line is an interface provided by an operating system (OS) that permits you to type text commands to control the OS or to query data it manages.

I should note that in this and every chapter, you’re strongly encouraged to try everything out for yourself. There’s no better way to really get to the core of an IT skill than actually doing it, realizing that it’s not working the way you expected, and playing with it until it becomes yours forever. Good luck and have fun!

1.1. What makes Linux different from other operating systems

Linux is free, which means it’s a lot easier than other OSs to install exactly where and when needed for any use you can imagine. Not having to worry about purchasing site licenses and jumping through Digital Rights Management hoops makes testing all kinds of hardware combinations and server configurations much more straightforward.

Linux makes it possible to do various really useful and creative things. For instance, you can load a Linux live boot image on a USB stick, boot a PC whose own hard disk has been corrupted, and then troubleshoot and fix the problem. (You’ll learn how to do that in chapter 6.) Or, because Linux is a true multiuser OS, whole teams can concurrently log in to work locally or remotely, confident in the privacy and stability of the system.

Linux was built with some of the same technology and comes with most of the same tools as the deeply mature UNIX OS. This adds a great deal of stability and security. Linux distributions also provide sophisticated software package management systems that reliably install and maintain any of the thousands of free software applications available through online curated repositories.

But beyond free, Linux is open source, which means anyone can take the code base and reshape it into anything they want. Practically, this has spawned a vast ecosystem of specialty Linux distributions. A distribution (sometimes shortened to distro) is a customized stack of software that’s packaged along with the Linux kernel and distributed with tools for installing a working version of Linux of user computers. Table 1.1 provides a very incomplete distro list to illustrate the kinds of things that are available.

Table 1.1. Some of the many available Linux distros

Can’t find what you’re after? Create your own. Need help? There’s a large and active community online where, if someone hasn’t already solved your problem, they’ll know where to go to get it done. More than anything else, I’d say it’s the community-based resources that really make Linux so powerful.

1.2. Basic survival skills

Before beginning with the enterprise-ready projects that make up the rest of the book, it’s worthwhile to make sure we’re starting on the same page. This chapter covers the Linux basics: the UNIX Filesystem Hierarchy Standard (including pseudo file systems), navigation (ls, pwd, and cd), file management tools (cat, less, touch, mkdir, rmdir, rm, cp, and mv), some tricks (like tab completion and file globbing), sudo, and where to turn for help (man, info, and journalctl).

It’s possible you have enough experience already that you won’t need any of that material. Feel free to skip this chapter altogether. Don’t worry about the rest of us. We’ll catch up.

Installing Linux

I’m not going to spend time talking about how to install Linux on your PC. It’s not because installation is so ridiculously simple; it can sometimes get quite complicated. Rather, it’s because the approach you choose depends on your specific circumstances. Describing one possibility or even half a dozen would do nothing more than annoy the 75% of you for whom those scenarios won’t work.

Need some help getting started with installation? Check out Learn Linux in a Month of Lunches (Manning, 2016). Encountering a particular installation issue? Take a minute to write a brief but detailed description, and then use it to search the internet for help. Looking for a laptop or a desktop with Linux preinstalled? Search the internet for pc with Linux preinstalled. Have some unused hardware and a USB stick? Search for install Linux from usb. Prefer to install Linux as a virtual machine? Smart move. Stick around for chapter 2.

1.2.1. The Linux file system

It’s often said that everything in Linux works through plain text files, so it probably makes the most sense to start by understanding the Linux file system. But before we can get to Linux, what’s a file system? You can think of it as a data table (or an index) that creates apparent connections between individual files and groups of files with identifiable locations on a disk. Figure 1.1 can help you visualize how data spread across a disk partition can be exposed to system users within a directory structure.

Figure 1.1. Raw data on storage devices can be visually represented by the OS as organized directory hierarchies.

Why would you need an index? A digital storage device like a hard drive or USB device isn’t divided into physical divisions that can be used as organizing folders (or directories, as they’re known in Linux circles). One particular file can reside in a location on the actual media that’s a great distance away from another, nearly identical file created minutes or seconds apart, and all the parts of a single file might not be contiguous. Not only that, a file’s geographic location on the disk won’t necessarily remain static over time.

If you want your data to be reliably retrievable, you’ll need some kind of index that can consistently point you to the resources you’re after. A file system uses such an index to provide the appearance of an organized set of directories and files within a single disk division known as a partition.

Note

In case you need to dig deeper on your own some time, it’ll be useful to know that these days, the most commonly used Linux file system is ext4. But Linux can also work with storage drives that were formatted using file systems from other platforms like FAT32 and NTFS.

All the files in a disk partition are kept in directories beneath the root directory, which is represented by the / (forward slash) character. The way these directories are arranged is largely governed by the UNIX Filesystem Hierarchy Standard (FHS). You’re going to see pretty much the same basic layout whether you’re using a Linux distribution, UNIX, or even macOS. Figure 1.2 shows some of the most used, top-level directories.

Figure 1.2. Common top-level directories as defined by the UNIX FHS

Top-level directories—those directories located directly beneath the root—include /etc/, which contains configuration files that define the way individual programs and services function, and /var/, which contains variable files belonging to the system or individual applications whose content changes frequently through the course of normal system activities. You’ll also want to know about the /home directory where individual users are given directories for their private files.

1.2.2. Getting around: Linux navigation tools

Here’s where you’ll learn the five most basic, must-have Linux navigation commands (ls, pwd, cd, cat, and less). Because the command line is not a particularly visual environment no matter what you’re trying to do, you’re going to rely a great deal on these five tools to orient yourself.

Note

I hope it’s obvious that you should be trying each of these tools out for yourself on your own computer. That’s the only way you’ll learn.

The rest of the book requires a command-line terminal of one sort or another. Unfortunately, there’s no single way to open a terminal window that’ll work in all Linux distributions. For example, the terminal’s location in the Ubuntu menu system won’t necessarily match that of Fedora or Mint. And Ubuntu itself? Well, that depends on which version you’re running.

The Ctrl-Alt-t keyboard combination should work in at least most environments, as will looking through the application menus searching for an item with terminal in the name. By default, once your terminal opens, your home directory (/home/yourname/) will be active.

ls (list)

No point hanging around the terminal if you can’t see what’s there. You can list the names of the files and subdirectories in your current directory using ls. The ls command with the l flag (l stands for long) lists not only the object names, but their file permissions, owner, group, file size, and time stamp. Adding a directory designation like /var/ displays the contents of that directory:

$ ls -l /var

total 40

drwxr-xr-x  2 root root  4096 May  3 06:25 backups

drwxr-xr-x 11 root root  4096 Jan 17 21:16 cache

drwxr-xr-x 39 root root  4096 Jan 17 21:16 lib

drwxrwsr-x  2 root staff  4096 Apr 12  2016 local

lrwxrwxrwx  1 root root      9 Aug 12  2016 lock -> /run/lock

drwxrwxr-x  7 root syslog 4096 May  3 06:25 log

drwxrwsr-x  2 root mail  4096 Aug 12  2016 mail

drwxr-xr-x  2 root root  4096 Aug 12  2016 opt

lrwxrwxrwx  1 root root      4 Aug 12  2016 run -> /run

drwxr-xr-x  5 root root  4096 Jan 17 21:16 spool

drwxrwxrwt  2 root root  4096 Nov  7  2016 tmp

drwxr-xr-x  3 root root  4096 Sep 11  2016 www

The h argument when added to ls -l displays file sizes in a human-readable format—kilobytes, megabytes, and gigabytes, rather than bytes, which tend to involve a great many hard-to-count digits:

$ ls -lh /var/log

total 18M                                                     

1

-rw-r--r-- 1 root  root    0 May  3 06:25 alternatives.log

drwxr-xr-x 2 root  root 4.0K May  3 06:25 apt

-rw-r----- 1 syslog adm  265K Jun  9 00:25 auth.log

-rw-r--r-- 1 root  root 312K Aug 12  2016 bootstrap.log

-rw------- 1 root  utmp    0 May  3 06:25 btmp

-rw-r----- 1 root  adm    31 Aug 12  2016 dmesg

-rw-r--r-- 1 root  root  836 May 21 14:15 dpkg.log

-rw-r--r-- 1 root  root  32K Nov  7  2016 faillog

drwxr-xr-x 2 root  root 4.0K Aug 12  2016 fsck

-rw-r----- 1 syslog adm  128K Jun  8 20:49 kern.log

-rw-rw-r-- 1 root  utmp 287K Jun  9 00:25 lastlog

-rw-r----- 1 syslog adm  1.7M Jun  9 00:17 syslog

-rw-rw-r-- 1 root  utmp 243K Jun  9 00:25 wtmp

1 The total disk space (in MB) consumed by files in this directory

Note

In general, you add arguments to Linux commands in one of two ways: a dash followed by a single