Microservices with Azure

Microservices with Azure

Software architecture patterns to build enterprise-grade Microservices using Microsoft Azure Service Fabric

Namit Tanasseri

Rahul Rai

       BIRMINGHAM - MUMBAI

Microservices with Azure

Copyright © 2017 Packt Publishing

All rights reserved. No part of this book may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, without the prior written permission of the publisher, except in the case of brief quotations embedded in critical articles or reviews.

Every effort has been made in the preparation of this book to ensure the accuracy of the information presented. However, the information contained in this book is sold without warranty, either express or implied. Neither the authors, nor Packt Publishing, and its dealers and distributors will be held liable for any damages caused or alleged to be caused directly or indirectly by this book.

Packt Publishing has endeavored to provide trademark information about all of the companies and products mentioned in this book by the appropriate use of capitals. However, Packt Publishing cannot guarantee the accuracy of this information.

First published: June 2017

Production reference: 1240617

Published by Packt Publishing Ltd.

Livery Place

35 Livery Street

Birmingham 

B3 2PB, UK.

ISBN 978-1-78712-114-0

www.packtpub.com

Credits

About the Authors

Namit Tanasseri is a certified Microsoft Cloud Solutions Architect with more than 11 years of work experience. He started his career as a Software Development Engineer with Microsoft Research and Development Center in 2005. During the first five years of his career, he had opportunities to work with major Microsoft product groups such as Microsoft Office and Windows. The experience working with the largest development teams within Microsoft helped him strengthen his knowledge of agile software development methodologies and processes. He also earned a patent award for a technology invention during this tenure.

Namit joined Microsoft Services chasing his passion for delivering solutions on cloud technologies and for gaining better exposure to technology consulting. As a technology consultant with Microsoft, Namit worked with Microsoft Azure Services for four years. Namit was a worldwide subject matter expert in Microsoft Azure, and he actively contributed to the Microsoft cloud community, apart from delivering top quality solutions for Microsoft customers. Namit also led the Windows Azure community in Microsoft Services India. Namit currently serves as a Microsoft Cloud Solutions Architect from Sydney, Australia working on large and medium sized enterprise engagements.

To my mom, Leena, and my wife, Poonam, for their patience during the writing of this book. 

I would like to portray my gratitude to Microsoft for cultivating the curiosity about technology and innovation within me.

Special thanks to my friends for all the good days, the memories of which help me sail through difficult times.

Finally, I cannot forget to thank God and my grandpa (who sits beside him in Heaven) for gifting me the skills and patience for completing this book.

Rahul Rai is a technology consultant based in Sydney, Australia with over nine years of professional experience. He has been at the forefront of cloud consulting for government organizations and businesses around the world.

Rahul has been working on Microsoft Azure since the service was in its infancy, delivering an ITSM tool built for and on Azure in 2008. Since then, Rahul has played the roles of a developer, a consultant, and an architect for enterprises ranging from small start-ups to multinational corporations.

Rahul has worked for over five years with Microsoft Services, where he worked with diverse teams to deliver innovative solutions on Microsoft Azure. In Microsoft, Rahul was a worldwide Subject Matter Expert in Microsoft cloud technologies. Rahul has also worked as a Cloud Solution Architect for Microsoft, for which he worked closely with some established Microsoft partners to drive joint customer transformations to cloud-based architectures. He loves contributing to community initiatives and speaks at some renowned conferences such as Tech Days and Ignite. Rahul is a contributor to Azure Open Source initiatives and has authored several MSDN articles and publications on Azure.

I dedicate this book to my mother and my father, Indra and Ghanshyam Rai, who have always been there for me.

I would like to thank my wife, Neha, without whose support this book would not have been possible. I would also like to thank my lovely sister, Mohini, my brother-in-law, Ajay and, my friends who have supported me in this authorial journey.

Thank you, God, for taking care of me every day and giving me the gift of writing.

Finally, I would like to thank the brilliant technical community which has never failed to inspire and drive me with its incredible knowledge and insatiable thirst for learning.

About the Reviewers

Roberto Freato has been an independent IT consultant since he started to work for small software factories while he was studying, after the MSc in computer science. With a thesis about Consumer Cloud Computing, he got specialization on Cloud and Azure. Today, he works as a freelance consultant for major companies in Italy, helping clients to design and kick-off their distributed software solutions. He trains for the developer community in his free time, speaking in many conferences. He is a Microsoft MVP since 2010.

Manish Sharma works with Microsoft as a solution architect as part of Microsoft Services. As a solution architect, he is responsible for leading large enterprise transformational engagements defining technology/solution roadmaps, conducting architecture and technical evaluations of enterprise engagements and architecting mutli-million-dollar solution developments and maintenance outsourcing managements. He is also a technology evangelist and speaker in prestigious events such as Microsoft TechEd, on latest and cutting-edge technologies such as HoloLens, Internet of Things, Connected Car, and Cloud technologies such as Azure.

Paul Glavich was an ASP.NET MVP for 13 years and currently works as a principal consultant for Readify. Previously, he was a chief technology officer (CTO) for Saasu (saasu.com), solution architect at Datacom, then senior consultant for readify, and prior to that a technical architect for EDS, Australia. He has over 20 years of industry experience ranging from PICK, C, C++, Delphi, and Visual Basic 3/4/5/6 to his current specialty in .Net with C#, ASP.NET, Azure, Cloud and DevOps.

Paul has been developing in .Net technologies since .Net was first in Beta and was the technical architect for one of the world’s first Internet banking solutions using .Net technology.

Paul can be seen on various .Net-related newsgroups, has presented at the Sydney .Net user group (www.sdnug.org) and TechEd, and is also a member of ASPInsiders (www.aspinsiders.com). He has also written some technical articles, which can be seen on community sites such as ASPAlliance.com (www.aspalliance.com). Paul has authored a total of three books, Beginning AJAX in ASP.NET, Beginning Microsoft ASP.NET AJAX, and the latest book on .Net Performance Testing and Optimization. He is currently focusing on overall architecture, solution design, and Microsoft Cloud solutions.

On a more personal note, Paul is married with three children, three grandkids, holds a fifth degree black belt in Budo-Jitsu, and also practices Wing Chun Kung fu.

There are so many people who have helped me get to where I am today, but it would take another book. So to keep things short, I would like to thank my three children Kristy, Marc, and Elizabeth for being awesome, my parents for buying me that first computer, my nerd friends for nerding out with me, but mostly I would like to thank my wife, Michele, for supporting me in my life and career, and enduring my never-ending stream of technobabble and Dad jokes.

www.PacktPub.com

For support files and downloads related to your book, please visit www.PacktPub.com.

Did you know that Packt offers eBook versions of every book published, with PDF and ePub files available? You can upgrade to the eBook version at www.PacktPub.com and as a print book customer, you are entitled to a discount on the eBook copy. Get in touch with us at service@packtpub.com for more details.

At www.PacktPub.com, you can also read a collection of free technical articles, sign up for a range of free newsletters and receive exclusive discounts and offers on Packt books and eBooks.

https://www.packtpub.com/mapt

Get the most in-demand software skills with Mapt. Mapt gives you full access to all Packt books and video courses, as well as industry-leading tools to help you plan your personal development and advance your career.

Why subscribe?

Fully searchable across every book published by Packt

Copy and paste, print, and bookmark content

On demand and accessible via a web browser

Customer Feedback

Thanks for purchasing this Packt book. At Packt, quality is at the heart of our editorial process. To help us improve, please leave us an honest review on this book's Amazon page at https://www.amazon.com/dp/1787121143.

If you'd like to join our team of regular reviewers, you can e-mail us at customerreviews@packtpub.com. We award our regular reviewers with free eBooks and videos in exchange for their valuable feedback. Help us be relentless in improving our products!

Table of Contents

Preface

What this book covers

What you need for this book

Who this book is for

Conventions

Reader feedback

Customer support

Downloading the example code

Downloading the color images of this book

Errata

Piracy

Questions

Microservices – Getting to Know the Buzzword

What are Microservices?

Microservices hosting platform

The Microservice advantage

Fault tolerance

Technology-agnostic

Development agility

Heterogeneous deployment

Manageability

Reusability

The SOA principle

Issues with SOA

The Microservices solution

Inter-Microservice communication

Communication through user interface

Sharing common code

Composite user interface for the web

Thin backend for rich clients

Synchronous communication

Asynchronous communication

Orchestrated communication

Shared data

Architecture of Microservices-based systems

Conway's law

Summary

Microsoft Azure Platform and Services Primer

PaaS for Microservices

Abstract infrastructure challenges

Simplified application lifecycle management

Simplifying development

Microsoft Azure – the choice of a hosting platform

Summary

Understanding Azure Service Fabric

The Service Fabric advantage

Highly scalable

Support for partitioning

Rolling updates

State redundancy

High-density deployment

Automatic fault tolerance

Heterogeneous hosting platforms

Technology agnostic

Centralized management

Service Fabric as an orchestrator

Orchestration as a Service

Is a cluster resource manager similar to an Azure load balancer?

Architecture of cluster resource manager

Architecture of Service Fabric

Transport Subsystem

Federation Subsystem

Reliability Subsystem

Management Subsystem

Hosting subsystem

Communication subsystem

Testability Subsystem

Deconstructing Service Fabric

Infrastructure model

Cluster

Node

System services

Naming service

Image store service

Upgrade service

Failover manager service

Cluster manager service

Service placement

Application model

Programming model

Guest Executables

Reliable Services

Reliable Actors

Creating a cluster on Azure

Basics

Cluster configuration

Security

Summary

Viewing your cluster status

Service Fabric Explorer

Summary view

Cluster Map

Fault domains

Upgrade domains

Viewing applications and services

Cluster nodes

Actions

System

Preparing your system

Summary

Hands-on with Service Fabric – Guest Executables

Service Fabric discovery and communication

Service protocols

Service discovery

Connections from applications external to Service Fabric

Configuring ports and protocols

Configuring the service manifest

Configuring the custom endpoint

Configuring the Azure load balancer

Configuring the health check

Built-in communication API

Deploying a Guest Executable

Understanding the manifests

Package structure

Packaging Guest Executables using Visual Studio

Manually packaging a Guest Executable

Creating the directory structure

Adding code and configuration

Updating service manifest

Updating the application manifest

Deployment

Deploying a Guest Container

Deploying Windows Container

Container image deployment and activation

Resource governance

Container port to host port mapping

Container-to-container discovery and communication

Configuring and setting environment variables

Deploying a Linux container

Summary

Hands on with Service Fabric – Reliable Services

Exploring the Service Fabric Explorer

Application Type

Application instance

Service type

Partition

Replica

Stateless Reliable Services

Stateless service architecture

Stateless service lifecycle

Scaling stateless services

Stateless frontend and stateless middle-tier

Stateless frontend and stateful middle-tier

Reliable Services communication

Exploring the application model

Stateful service

Stateful service architecture

Reliable Collections

Up and down counter application

Stateful service lifecycle

Service partitioning

Service replicas

Summary

Reliable Actors

Actor model

What is an Actor?

Actors in Service Fabric

Actor lifetime

Saving state

Distribution and failover

Actor communication

The Actor proxy

Concurrency

Reentrancy

Asynchronous drivers

Timers

Actor reminders

Actor events

Your first Reliable Actors application

Summary

Microservices Architecture Patterns Motivation

Creating an architecture

Defining the solution boundaries

Creating the solution structure

Component design

Classification of architectural patterns

Optimization or non-functional patterns

Operational patterns

Implementation or functional patterns

Picking up the right architecture pattern

Context

Forces

Complementing patterns

Applying a pattern

Structural variation

Behavioral variation

Internal variation

Domain dependent variation

Summary

Microservices Architectural Patterns

Architectural patterns

Service proxy

Problem

Solution

Considerations

Related patterns

Use cases

Service Façade \ API Gateway

Problem

Solution

Considerations

Related patterns

Use cases

Reliable Data Store

Problem

Solution

Considerations

Related patterns

Use cases

Cluster Orchestrator

Problem

Solution

Considerations

Related patterns

Use cases

Auto-scaling

Problem

Solution

Considerations

Related patterns

Use cases

Partitioning

Problem

Solution

Considerations

Related patterns

Use cases

Centralized Diagnostics Service

Problem

Solution

Considerations

Related patterns

Use cases

High Density Deployment

Problem

Solution

Considerations

Related patterns

Use cases

API Gateway

Problem

Solution

Considerations

Related patterns

Use cases

Latency Optimized Load Balancing

Problem

Solution

Considerations

Related patterns

Use cases

Queue Driven Decomposing Strategy

Problem

Solution

Considerations

Related patterns

Use cases

Circuit Breaker

Problem

Solution

Considerations

Related patterns

Use cases

Message Broker for Communication

Problem

Solution

Considerations

Related patterns

Use cases

Compensating Transaction

Problem

Solution

Considerations

Related patterns

Use cases

Rate Limiter

Problem

Solution

Considerations

Related patterns

Use cases

Sagas

Problem

Solution

Considerations

Related patterns

Use cases

Master Data Management

Problem

Solution

Considerations

Related patterns

Use cases

CQRS – Command Query Responsibility Segregation

Problem

Solution

Microservices in CQRS

Advantages

Considerations

Related patterns

Use cases

Event Sourcing

Problem

Solution

Considerations

Related patterns

Use cases

Remindable Actors

Problem

Solution

Considerations

Related patterns

Use cases

Managed Actors

Problem

Solution

Considerations

Related patterns

Use cases

Summary

Securing and Managing Your Microservices

Securing the communication channels

Inter-node communication

Client to node security

Certificate security

Azure Active Directory security

Publishing an application to a secured cluster

Managing Service Fabric clusters with Windows PowerShell

Prerequisites

Deploying Service Fabric applications

Upgrading Service Fabric applications

Removing Service Fabric applications

Summary

Diagnostics and Monitoring

Health entities

Health state

Health policies

Cluster health policy

Application health policy

Service type health policy

Health evaluation

Health reporting

Centralized logging

Collecting logs

Diagnostic extension

Deploying the Diagnostics extension

Summary

Continuous Integration and Continuous Deployment

Continuous Integration

Continuous Delivery

Deploying Service Fabric application on a standalone cluster

Deploying the application

Summary

Serverless Microservices

Before committing to Nanoservices

Building Nanoservices with Azure Functions

Function app templates

Timer function apps

Data processing function apps

Webhook and API function apps

Summary

Preface

Let’s take a drive down the memory lane. Imagine it is the 90’s, and you have been asked to build an application for an enterprise. You go to an IT shop and procure a server (or ask someone to do it for you), after several days or weeks the server shows up. Next, you configure the server, the network and the many settings it has. Finally, you deploy your application on the server and turn on the lights, and everyone is happy.  

Business requirements change and now your customer needs some changes to be made to the application. So you go back make the changes, take the now upgraded application package and dump it on the server. You shut down the machine and bring it back up to bring up your refreshed application. You make compromises with downtime but are happy with the convenience of bundling everything into a single package and pushing it to the server.

So, in this early period, the agility of hardware limited the agility of software. It wasn’t easy to scale infrastructure and therefore altering the systems at the snap of fingers wasn’t a possibility. Moreover, during those times internet applications were more of a value addition than a necessity. Thus, traditional hardware agility supported monolithic architecture, which requires building applications as one big bundle of tightly coupled services.

Fast forward a couple of years and the advent of cloud changed the game. With the cloud, infrastructure has become easy to provision on-demand, and it also eliminates any waste of developer time that traditionally went into preparing the servers to host applications. Today software is a differentiator and not a value addition for any enterprise. Owing to the competition, high frequency of change, and continually changing customer requirements require decoupling an application to make it easy to scale, test and deploy.

The advent of cloud and the need for applications to be decoupled into smaller components that fulfill a single responsibility (Single Responsibility Principle) for the ease of development, deployment, and scale lead to the rise of the Microservice architecture.

However, breaking down a monolith into small independent services comes with the overhead of management of these services. This need of managing the Microservices gave birth to Azure Service Fabric which apart from its many services acts as a cluster manager. Azure Service Fabric Cluster Manager governs a set of virtual machines and handles Microservice deployment, placement, and failover. Azure Service Fabric also governs the application model and has capabilities to drive Application Lifecycle Management (ALM).

Microsoft itself has used Service Fabric for many years to host its planet-scale services such as Azure SQL Database, Azure IoT hub, Skype for Business, Cortana, and Intune.

This book introduces its readers to the concept of Microservices and Microsoft Azure Service Fabric as a distributed platform to host enterprise-grade Microservices. Learning the concepts of technology is often not sufficient. In practical scenarios, developers and architects often search for guidance on some of the most common design challenges. This book addresses common architectural challenges associated with the Microservice architecture, using proven architectural patterns.

What this book covers

Chapter 1, Microservices – Getting to Know the Buzzword, lays the foundation of concepts of Microservices and explores the scenarios, where Microservices are best suited for your application.

Chapter 2, Microsoft Azure Platform and Services Primer, provides a very fast-paced overview of Microsoft Azure as a platform for hosting internet-scale applications.

Chapter 3, Understanding Azure Service Fabric, explains the basic concepts and architecture of Azure Service Fabric.

Chapter 4, Hands-on with Service Fabric – Guest Executables, talks about building and deploying applications as Guest Executables on a Service Fabric cluster.

Chapter 5, Hands on with Service Fabric – Reliable Services, explains the concept of Reliable Services programming model for building Microservices hosted on Service Fabric.

Chapter 6, Reliable Actors, introduces Actor programming model on Service Fabric and the ways to build and deploy actors on a Service Fabric cluster.

Chapter 7, Microservices Architecture Patterns Motivation, provides an overview of the motivation behind driving Microservices architectural patterns. The chapter also talks about the classification of the patterns that are discussed in this book.

Chapter 8, Microservices Architectural Patterns, introduces a catalog of design patterns categorized by its application. Each design pattern explains the problem and the proven solution for that problem. The pattern concludes with considerations that should be taken while applying the pattern and the use cases where the pattern can be applied.

Chapter 9, Securing and Managing Your Microservices, will guide you on securing your Microservices deployment on a Service Fabric cluster.

Chapter 10, Diagnostics and Monitoring, covers how to set up diagnostics and monitoring in your Service Fabric application. You will also learn how to use Service Fabric Explorer to monitor the cluster.

Chapter 11, Continuous Integration and Continuous Deployment, takes you through the process of deploying your Microservices application on a Service Fabric cluster using Visual Studio Team Services.

Chapter 12, Serverless Microservices, helps you understand the concept of Serverless Computing and building Microservices using Azure functions.

What you need for this book

The examples found in this book require a Microsoft Azure subscription. You can sign up for a free trial account via the Azure website: https://azure.microsoft.com/.

You will need Windows 7+, latest Service Fabric SDK, latest Azure SDK, latest Azure PowerShell, 4GB RAM, 30 GB available Hard Disk space, Visual Studio 2017, and Visual Studio Team Service for executing the practical examples in this book.

Who this book is for

The book is aimed at IT architects, system administrators, and DevOps engineers who have a basic knowledge of the Microsoft Azure platform and are working on, or are curious about, the concepts of Microservices and the Microservice architecture.

This book assumes that you are proficient in .NET, especially in C# development. A little knowledge of Azure SDK, Azure Management Portal, Azure PowerShell, and Azure Command Line Interface (Azure CLI) will help you navigate through this book easily.

Conventions

In this book, you will find a number of text styles that distinguish between different kinds of information. Here are some examples of these styles and an explanation of their meaning.

Code words in text, database table names, folder names, filenames, file extensions, pathnames, dummy URLs, user input, and Twitter handles are shown as follows: the argument cancellationToken is an object of type CancellationToken

A block of code is set as follows:

    ServiceEndpoint Type=Input Protocol=http