README for the src/shielding subject folder (#38707)

Peter Bengtsson · gracepark · web-flow · commit 52349625b9f5 · 2023-06-28T16:53:39.000Z
Co-authored-by: Grace Park &lt;gracepark@github.com&gt;
diff --git a/src/search/README.md b/src/search/README.md
@@ -125,5 +125,5 @@ Each record represents a section of a page. Sections are derived by splitting up
 - It's not strictly necessary to set an `objectID` as the search index will create one automatically, but by creating our own we have a guarantee that subsequent invocations of this upload script will overwrite existing records instead of creating numerous duplicate records with differing IDs.
 - Our search querying has typo tolerance. Try spelling something wrong and see what you get!
 - Our search querying has lots of controls for customizing each index, so we can add weights to certain attributes and create rules like "title is more important than body", etc. But it works pretty well as-is without any configuration.
-- Our search querying has support for "advanced query syntax" for exact matching of quoted expressions and exclusion of words preceded by a `-` sign. This is off by default but we have it enabled in our browser client. The settings in the web interface can be overridden by the search endpoint. See [middleware/search.js]([middleware/search.js).
+- Our search querying has support for "advanced query syntax" for exact matching of quoted expressions and exclusion of words preceded by a `-` sign. This is off by default, but it is enabled in our browser client. The settings in the web interface can be overridden by the search endpoint. See [middleware/search.js](middleware/search.js).
 - When needed, the Docs Engineering team can commit updates to the search index, as long as the label `skip-index-check` is applied to the PR.
diff --git a/src/shielding/README.md b/src/shielding/README.md
@@ -0,0 +1,36 @@
+# Shielding
+
+## Overview
+
+Essentially code in our server that controls the prevention of "junk requests" is scripted HTTP requests to endpoints that are *not* made by regular browser users.
+
+For example, there's middleware code that sees if a `GET` request
+comes in with a bunch of random looking query strings keys. This would cause a PASS on the CDN but would not actually matter to the rendering. In this
+case, we spot this early and return a redirect response to the same URL
+without the unrecognized query string keys so that if the request follows
+redirects, the eventual 200 would be normalized by a common URL so the CDN
+can serve a HIT.
+
+Here's an in-time discussion post that summaries the *need* and much of the
+recent things we've done to fortify our backend servers to avoid unnecessary
+work loads:
+
+**[How we have fortified Docs for better resiliency and availability (June 2023)](https://github.com/github/docs-engineering/discussions/3262)**
+
+## How it works
+
+At its root, the `src/shielding/middleware/index.js` is injected into our
+Express server. From there, it loads all its individual middleware handlers.
+
+Each middleware is one file that focuses on a single use-case. The
+use-cases are borne from studying log files (CDN and Azure App Service) to
+spot patterns of request abuse.
+
+## Notes
+
+- The best place to do shielding is as close to the client(s) as possible,
+i.e. in the CDN or in Azure Frontdoor. Having the code in our own backend
+has the advantage that it's easier to write custom business logic
+along with end-to-end tests.
+- Some shielding "tricks" appear in other places throughout the code
+base such as controlling the 404 response for `/assets/*` URLs.
