JavaScript at Scale

Table of Contents

JavaScript at Scale

Credits

About the Author

About the Reviewers

www.PacktPub.com

Support files, eBooks, discount offers, and more

Why subscribe?

Free access for Packt account holders

Preface

What this book covers

What you need for this book

Who this book is for

Conventions

Reader feedback

Customer support

Downloading the example code

Errata

Piracy

Questions

1. Scale from a JavaScript Perspective

Scaling influencers

The need for scale

Growing user base

Building new features

Hiring more developers

Architectural perspectives

The browser is a unique environment

Component design

Component communication

Load time

Responsiveness

Addressability

Configurability

Making architectural trade-offs

Defining your constants

Performance for ease of development

Configurability for performance

Performance for substitutability

Ease of development for addressability

Maintainability for performance

Less features for maintainability

Leveraging frameworks

Frameworks versus libraries

Implementing patterns consistently

Performance is built in

Leverage community wisdom

Frameworks don't scale out-of-the-box

Summary

2. Influencers of Scale

Scaling users

License fees

Subscription fees

Consumption fees

Ad-supported

Open source

Communicating users

Support mechanisms

Feedback mechanisms

Notifying users

User metrics

Scaling users example

Scaling features

Application value

Killer features versus features that kill

Data-driven features

Competing with other products

Modifying existing features

Supporting user groups and roles

Introducing new services

Consuming real-time data

Scaling features example

Scaling development

Finding development resources

Development responsibilities

Too many resources

Scaling development example

Influencer checklist

User checklist

What's the business model of our software?

Does our application have different user roles?

Do our users communicate with each other using our software?

How do we support our application?

How do we collect feedback from users?

How do we notify users with relevant information?

What type of user metrics should we collect?

Feature checklist

What's the core value proposition of our software?

How do we determine the feasibility of a feature?

Can we make informed decisions about our features?

Who's our competition?

How do we make what we have better?

How do we integrate user management into our features?

Are our features tightly coupled to backend services?

How does the frontend stay synchronized with backend data?

Developer checklist

How do we find the right development resources?

How do we allocate development responsibilities?

Can we avoid hiring too many resources?

Summary

3. Component Composition

Generic component types

Modules

Routers

Models/Collections

Controllers/Views

Templates

Application-specific components

Extending generic components

Identifying common data and functionality

Extending router components

Extending models/collections

Extending controllers/views

Mapping features to components

Generic features

Specific features

Decomposing components

Maintaining and debugging components

Re-factoring complex components

Pluggable business logic

Extending versus configuring

Stateless business logic

Organizing component code

Summary

4. Component Communication and Responsibilities

Communication models

Message-passing models

Event models

Communication data schema

Naming conventions

Data format

Common data

Traceable component communication

Subscribing to events

Globally-logging events

Event lifecycle

Communication overhead

Event frequency

Callback execution time

Callback complexity

Areas of communication responsibility

Backend API

Web socket updates

DOM updates

Loosely-coupled communication

Substituting components

Handling unexpected events

Component layers

Event flow direction

Mapping to developer responsibilities

Mentally mapping the code

Summary

5. Addressability and Navigation

Approaches to routing

Hash URIs

Traditional URIs

How routers work

Router responsibilities

Router events

URI parts and patterns

Encoding information

Designing URIs

Mapping resources to URIs

Building URIs manually

Automating resource URIs

Triggering routes

User actions

Redirecting users

Router configuration

Static route declarations

Registration events

Deactivating routes

Troubleshooting routers

Conflicting routes

Logging initial configuration

Logging route events

Handling invalid resource states

Summary

6. User Preferences and Defaults

Preference types

Locales

Behavior

Appearance

Supporting locales

Deciding on locales to support

Maintaining locales

Setting the locale

Choosing locales

Storing locale preferences

Locales in URIs

Generic component configuration

Deciding on configuration values

Stored and hard-coded default values

Backend implications

Loading configuration values

Configuring behavior

Enabling and disabling components

Changing quantities

Changing order

Configuring notifications

Inline options

Changing the look and feel

Theme tools

Selecting a theme

Individual style preferences

Performance implications

Configurable locale performance

Configurable behavior performance

Configurable theme performance

Summary

7. Load Time and Responsiveness

Component artifacts

Component dependencies

Building components

Loading components

Loading modules

Lazy module loading

Module load latency

Communication bottlenecks

Reducing indirection

Profiling code

Component optimization

Components that maintain state

Dealing with side-effects

DOM rendering techniques

API data

Load latency

Working with large data sets

Optimizing components at runtime

Summary

8. Portability and Testing

Decoupling the backend

Mocking the backend API

Frontend entry points

Mocking tools

Generating mock data sets

Performing actions

Feature design process

Designing the API

Implementing the mock

Implementing the feature

Reconciling mock data with API data

Unit testing tools

Tools built into frameworks

Standalone unit testing tools

Toolchains and automation

Testing mock scenarios

Mock APIs and test fixtures

Scenario generation tools

End-to-end tests and continuous integration

Summary

9. Scaling Down

Scaling constraints

JavaScript artifact size

Network bandwidth

Memory consumption

CPU consumption

Backend capabilities

Conflicting features

Overlapping functionality

Irrelevant features

Customer demand

Design failures

Unnecessary components

Inefficient data processing

Excessively creative markup

Application composition

Feature enablement

New feature impact

Essential libraries

Summary

10. Coping with Failure

Failing fast

Using quality constraints

Providing meaningful feedback

When we can't fail fast...

Fault tolerance

Classifying critical behavior

Detecting and containing errant behavior

Disabling defective components

Gracefully degrading functionality

Failure recovery

Retrying failed operations

Restarting components

Manual user intervention

When we can't recover from failures...

Performance and complexity

Exception handling

State checking

Notifying other components

Logging and debugging

Meaningful error logs

Warning about potential failures

Informing and instructing users

Improving the architecture

Documenting failure scenarios

Improving component classification

Complexity promotes failure

Summary

Index

JavaScript at Scale

JavaScript at Scale

Copyright © 2015 Packt Publishing

All rights reserved. No part of this book may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, without the prior written permission of the publisher, except in the case of brief quotations embedded in critical articles or reviews.

Every effort has been made in the preparation of this book to ensure the accuracy of the information presented. However, the information contained in this book is sold without warranty, either express or implied. Neither the author, nor Packt Publishing, and its dealers and distributors will be held liable for any damages caused or alleged to be caused directly or indirectly by this book.

Packt Publishing has endeavored to provide trademark information about all of the companies and products mentioned in this book by the appropriate use of capitals. However, Packt Publishing cannot guarantee the accuracy of this information.

First published: July 2015

Production reference: 1270715

Published by Packt Publishing Ltd.

Livery Place

35 Livery Street

Birmingham B3 2PB, UK.

ISBN 978-1-78528-215-7

www.packtpub.com

Credits

Author

Adam Boduch

Reviewers

August N. Marcello III

Yogesh Singh

Nikolay Sokolov

Serkan Yersen

Commissioning Editor

Edward Gordon

Acquisition Editors

Kevin Colaco

Owen Roberts

Content Development Editor

Divij Kotian

Technical Editor

Ryan Kochery

Copy Editor

Angad Singh

Project Coordinator

Nikhil Nair

Proofreader

Safis Editing

Indexer

Rekha Nair

Graphics

Jason Monteiro

Production Coordinator

Melwyn Dsa

Cover Work

Melwyn Dsa

About the Author

Adam Boduch has been involved with large-scale JavaScript development for nearly 10 years. Before moving to the frontend, he worked on several large-scale cloud computing products using Python and Linux. No stranger to complexity, Adam has practical experience with real-world software systems and the scaling challenges they pose. He is the author of several JavaScript books, including Lo-Dash Essentials, and is passionate about innovative user experiences and high performance.

Adam lives in Toronto and is a senior software engineer at Virtustream.

I'd like to thank my mom and dad.

About the Reviewers

August N. Marcello III is a highly passionate software engineer with nearly 2 decades of experience in the design, implementation, and deployment of modern client-side web application architectures in the enterprise. An exclusive focus on delivering compelling SaaS based user experiences throughout the web ecosystem has proven both personally and professionally rewarding for him. His passion for emerging technologies in general, combined with a particular focus on forward-thinking JavaScript platforms, has been a primary driver in his pursuit of technical excellence. When he's not coding, he can be found trail running, mountain biking, and spending time with family and friends. Visit him online at www.augustmarcello.com.

Many thanks to Chuck, Mark, Eric, and Adam, with whom I had the privilege to work and learn. Gratitude to my family, friends, and the experiences I have been blessed to be a part of.

Yogesh Singh graduated in computer science and engineering from JSS Academy of Technical Education, India. He is a full-stack web developer with experience in major server-side web development stack (ASP.NET and Node.js) and advanced knowledge of HTML, CSS and JavaScript.

Yogesh is enthusiastic about JavaScript, and its library and framework (Backbone, AngularJS, jQuery, and Underscore).

He started his career in data mining and data warehousing, with expert level knowledge in database development. He is a Microsoft Certified Solutions Associate (MCSA) in MSSQL.

He is a self-learner and enjoys learning algorithms and data structure. He achieved a statement of accomplishment from Standford University (Coursera) for algorithms.

Currently, he is working at Gainsight as a full-stack developer. Previously, he worked at OLX India and MAQ Software.

In his spare time, he likes to blog at http://mylearning.in. His LinkedIn profile can be found at https://www.linkedin.com/in/yogesh21

I would like to thank my family, friends, and colleagues for their support.

Nikolay Sokolov is a software engineer with vast experience in cloud computing, deployment automation, and enterprise software development. Currently, he is working on core platform development at Tonomi (http://tonomi.com/), delivering the autonomic management of cloud applications based on the flexible component model.

Feel free to contact him at https://twitter.com/chemikadze

Serkan Yersen is a software developer from San Francisco. He is the author of open source libraries such as ifvisible.js, underscore.py, and kwargs.js. Serkan has specialized in building large-scale JavaScript applications and creating UIs that will be used by a large variety of users. From 2006 to 2012, Serkan worked for http://www.jotform.com/ and built a complex form builder, which is being used by millions of users. Right now, he is building web applications for Home Depot and Redbeacon (http://www.redbeacon.com/). You can reach him at http://serkan.io/.

www.PacktPub.com

Support files, eBooks, discount offers, and more

For support files and downloads related to your book, please visit www.PacktPub.com.

Did you know that Packt offers eBook versions of every book published, with PDF and ePub files available? You can upgrade to the eBook version at www.PacktPub.com and as a print book customer, you are entitled to a discount on the eBook copy. Get in touch with us at for more details.

At www.PacktPub.com, you can also read a collection of free technical articles, sign up for a range of free newsletters and receive exclusive discounts and offers on Packt books and eBooks.

https://www2.packtpub.com/books/subscription/packtlib

Do you need instant solutions to your IT questions? PacktLib is Packt's online digital book library. Here, you can search, access, and read Packt's entire library of books.

Why subscribe?

Fully searchable across every book published by Packt

Copy and paste, print, and bookmark content

On demand and accessible via a web browser

Free access for Packt account holders

If you have an account with Packt at www.PacktPub.com, you can use this to access PacktLib today and view 9 entirely free books. Simply use your login credentials for immediate access.

This book is for Melissa, Jason, and Simon. Thanks for all the love and support.

Preface

Some applications just get it right. These are the exceptions rather than the rule. Lots of JavaScript applications get one or two things right, and other things very wrong. The things we get wrong are a side effect of the scaling influencers that we never considered. This is a book about scaling our frontend architectures to meet the quality requirements asked of us. Scaling JavaScript applications is an interesting and fun problem. There're so many moving parts—the users, the developers, the deployment environments, the browser environments, and the task of bringing all of these factors together to form a meaningful user experience. What are we scaling, and why? The aim of this book is to help us answer these questions.

What this book covers

Chapter 1, Scale from a JavaScript Perspective, introduces the idea of scalable JavaScript applications and what makes them different from other applications that scale.

Chapter 2, Influencers of Scale, helps us understand that the need to scale helps us design better architectures.

Chapter 3, Component Composition, explains how the patterns that form the core of our architecture serve as blueprints for assembling components.

Chapter 4, Component Communication and Responsibilities, explains how components that communicate with one another are a scaling constraint. It tells us how features are the result of component communication patterns.

Chapter 5, Addressability and Navigation, elaborates on large-scale web applications with URIs that point to resources, and how designs that scale can handle a growing number of URIs.

Chapter 6, User Preferences and Defaults, tells us why users need control over certain aspects of our software. And it also explains that scalable application components are configurable.

Chapter 7, Load Time and Responsiveness, explains how more moving parts means performance degradation across the application. This includes making trade-offs that keep our UI responsive, while adding new features.

Chapter 8, Portability and Testing, covers writing JavaScript code that's not tightly coupled with a single environment. This includes creating portable mock data and portable tests.

Chapter 9, Scaling Down, explains how removing unused or buggy components from applications is essential, if we want to scale up in other areas.

Chapter 10, Coping with Failure, explains that large-scale JavaScript architectures can't fall over as a result of a bug in one component. This includes how designing with failure in mind is the key to achieving scale in a broad number of scenarios.

What you need for this book

NodeJS

Code Editor/IDE

A modern Web browser

Who this book is for

This book is intended for a senior JavaScript developer who is curious about architectural issues in the frontend. There's no prerequisite framework knowledge required, however, most of the concepts presented throughout the book are adaptations of components found in frameworks such as Backbone, Angular, or Ember. Strong JavaScript language skills are required, and all code examples are presented using ECMAScript 6 syntax.

Conventions

In this book, you will find a number of text styles that distinguish between different kinds of information. Here are some examples of these styles and an explanation of their meaning.

Code words in text, database table names, folder names, filenames, file extensions, pathnames, dummy URLs, user input, and Twitter handles are shown as follows: For example, users/31729. Here, the router will need to find a pattern that matches this string, and the pattern will also specify how to extract the 31729 variable.

A block of code is set as follows:

// Renders the sections of the view. Each section

    // either has a renderer, or it doesn't. Either way,

    // content is returned.

    render() {

Note

Warnings or important notes appear in a box like this.

Tip

Tips and tricks appear like this.

Reader feedback

Feedback from our readers is always welcome. Let us know what you think about this book—what you liked or disliked. Reader feedback is important for us as it helps us develop titles that you will really get the most out of.

To send us general feedback, simply e-mail <feedback@packtpub.com>, and mention the book's title in the subject of your message.

If there is a topic that you have expertise in and you are interested in either writing or contributing to a book, see our author guide at www.packtpub.com/authors.

Customer support

Now that you are the proud owner of a Packt book, we have a number of things to help you to get the most from your purchase.

Downloading the example code

You can download the example code files from your account at http://www.packtpub.com for all the Packt Publishing books you have purchased. If you purchased this book elsewhere, you can visit http://www.packtpub.com/support and register to have the files e-mailed directly to you.

Errata

Although we have taken every care to ensure the accuracy of our content, mistakes do happen. If you find a mistake in one of our books—maybe a mistake in the text or the code—we would be grateful if you could report this to us. By doing so, you can save other readers from frustration and help us improve subsequent versions of this book. If you find any errata, please report them by visiting http://www.packtpub.com/submit-errata, selecting your book, clicking on the Errata Submission Form link, and entering the details of your errata. Once your errata are verified, your submission will be accepted and the errata will be uploaded to our website or added to any list of existing errata under the Errata section of that title.

To view the previously submitted errata, go to https://www.packtpub.com/books/content/support and enter the name of the book in the search field. The required information will appear under the Errata section.

Piracy

Piracy of copyrighted material on the Internet is an ongoing problem across all media. At Packt, we take the protection of our copyright and licenses very seriously. If you come across any illegal copies of our works in any form on the Internet, please provide us with the location address or website name immediately so that we can pursue a remedy.

Please contact us at <copyright@packtpub.com> with a link to the suspected pirated material.

We appreciate your help in protecting our authors and our ability to bring you valuable content.

Questions

If you have a problem with any aspect of this book, you can contact us at <questions@packtpub.com>, and we will do our best to address the problem.

Chapter 1. Scale from a JavaScript Perspective

JavaScript applications are getting bigger. That's because we can do more with the language—more than most thought possible. After all, JavaScript was conceived as a means to activate otherwise static web pages. A means by which to fill gaps in HTML, as it were. Year after year, more and more web sites started developing JavaScript code to improve the functionality of their pages.

Despite the frustrations of certain language idiosyncrasies, JavaScript popularity gained critical mass—today it's the most popular programming language on GitHub (http://githut.info/). From then onward, web sites started looking more like applications that a user would install on their desktop. Libraries and frameworks started popping up left right and center. Why? Because frontend JavaScript applications are large and complex.

In the present day frontend development profession, there's a lot of tools at our disposal. The JavaScript language has evolved into something that's usable on its own; it's becoming less dependent on libraries to perform the most fundamental and basic programming tasks. This is especially true of the next iteration of the ECMAScript specification, where problems that have plagued developers for years are at least partially addressed by constructs added to the language. This, of course, doesn't negate the need for application frameworks. The frontend development environment and its supporting web standards are far from perfect, but they're improving.

Something that's been missing from the frontend development picture for a long time is architecture. Frontend architectures have become prevalent in recent years due to the complexity of what's being implemented. Sophisticated tools, allow frontend developers to design an architecture that's able to scale with the problems we're trying to solve. And that's the crux of this book—JavaScript architectures that scale. But scale to what exactly? It's not your traditional scaling problem in computing, where you need to handle more load in a distributed server environment. Scaling in the frontend presents its own unique challenges and constraints. This chapter will define some of the scaling issues faced by JavaScript architectures.

Scaling influencers

We don't scale our software systems just because we can. While it's common to tout scalability, these claims need to be put into practice. In order to do so, there has to be a reason for scalable software. If there's no need to scale, then it's much easier, not to mention cost-effective, to simply build a system that doesn't scale. Putting something that was built to handle a wide variety of scaling issues into a context where scale isn't warranted just feels clunky. Especially to the end user.

So we, as JavaScript developers and architects, need to acknowledge and understand the influences that necessitate scalability. While it's true that not all JavaScript applications need to scale, it may not always be the case. For example, it's difficult to say that we know this system isn't going to need to scale in any meaningful way, so let's not invest the time and effort to make it scalable. Unless we're developing a throw-away system, there's always going to be expectations of growth and success.

At the opposite end of the spectrum, JavaScript applications aren't born as mature scalable systems. They grow up, accumulating scalable properties along the way. Scaling influencers are an effective tool for those of us working on JavaScript projects. We don't want to over-engineer something straight from inception, and we don't want to build something that's tied-down by early decisions, limiting its ability to scale.

The need for scale

Scaling software is a reactive event. Thinking about scaling influencers helps us proactively prepare for these scaling events. In other systems, such as web application backends, these scaling events may be brief spikes, and are generally handled automatically. For example, there's an increased load due to more users issuing more requests. The load balancer kicks in and distributes the load evenly across backend servers. In the extreme case, the system may automatically provision new backend resources when needed, and destroy them when they're no longer of use.

Scaling events in the frontend aren't like that. Rather, the scaling events that take place generally happen over longer periods of time, and are more complex. The unique aspect of JavaScript applications is that the only hardware resources available to them are those available to the browser in which they run. They get their data from the backend, and this may scale up perfectly fine, but that's not what we're concerned with. As our software grows, a necessary side-effect of doing something successfully, is that we need to pay attention to the influencers of scale.

The preceding figure shows us a top-down flow chart of scaling influencers, starting with users, who require that our software implements features. Depending on various aspects of the features, such as their size and how they relate to other features, this influences the team of developers working on features. As we move down through the scaling influencers, this grows.

Growing user base

We're not building an application for just one user. If we were, there would be no need to scale our efforts. While what we build might be based on the requirements of one user representative, our software serves the needs of many users. We need to anticipate a growing user base as our application evolves. There's no exact target user count, although, depending on the nature of our application, we may set goals for the number of active users, possibly by benchmarking similar applications using a tool such as http://www.alexa.com/. For example, if our application is exposed on the public internet, we want lots of registered users. On the other hand, we might target private installations, and there, the number of users joining the system is a little slower. But even in the latter case, we still want the number of deployments to go up, increasing the total number of people using our software.

The number of users interacting with our frontend is the largest influencer of scale. With each user added, along with the various architectural perspectives, growth happens exponentially. If you look at it from a top-down point of view, users call the shots. At the end of the day, our application exists to serve them. The better we're able to scale our JavaScript code, the more users we'll please.

Building new features

Perhaps the most obvious side-effect of successful software with a strong user base is the features necessary to keep those users happy. The feature set grows along with the users of the system. This is often overlooked by projects, despite the obviousness of new features. We know they're coming, yet, little thought goes into how the endless stream of features going into our code impedes our ability to scale up our efforts.

This is especially tricky when the software is in its infancy. The organization developing the