Skip to content

Commit 44aef63

Browse files
committed
update docs and move parts from README to index.md
1 parent c1d7d06 commit 44aef63

File tree

4 files changed

+139
-87
lines changed

4 files changed

+139
-87
lines changed

README.md

Lines changed: 19 additions & 44 deletions
Original file line numberDiff line numberDiff line change
@@ -8,40 +8,25 @@
88

99
<img src="docs/diagrams/logo.png" width="200">
1010

11-
## Introduction to the Postgres Operator
11+
The Postgres operator manages PostgreSQL clusters on Kubernetes. Configuration
12+
happens only via manifests. Therefore, it integrates easily in automated CI/CD
13+
pipelines with no access to Kubernetes directly.
1214

13-
The Postgres [operator](https://coreos.com/blog/introducing-operators.html)
14-
manages PostgreSQL clusters on Kubernetes:
15-
16-
1. The operator watches additions, updates, and deletions of PostgreSQL cluster
17-
manifests and changes the running clusters accordingly. For example, when a
18-
user submits a new manifest, the operator spawns a new Postgres cluster with
19-
necessary entities such as StatefulSets, Services, and also Postgres roles. See this
20-
[Postgres cluster manifest](manifests/complete-postgres-manifest.yaml)
21-
for settings that a manifest may contain.
22-
23-
2. The operator also watches updates to [its own configuration](manifests/configmap.yaml)
24-
and alters running Postgres clusters if necessary. For instance, if a pod
25-
Docker image is changed, the operator carries out the rolling update. That
26-
is, the operator re-spawns one-by-one pods of each StatefulSet it manages
27-
with the new Docker image.
28-
29-
3. Finally, the operator periodically synchronizes the actual state of each
30-
Postgres cluster with the desired state defined in the cluster's manifest.
31-
32-
4. The operator aims to be hands free and configuration happens only via manifests and its own config.
33-
This enables easy integration in automated deploy pipelines with no access to Kubernetes directly.
15+
By default, the operator is building up on two other Open Source projects
16+
developed at Zalando. [Spilo](https://github.com/zalando/spilo) provides the
17+
Docker image that contains PostgreSQL incl. some pre-compiled extensions. Spilo
18+
also includes [Patroni]((https://github.com/zalando/spilo), to manage highly
19+
available Postgres cluster powered by streaming replication.
3420

3521
# Getting started
3622

3723
For a quick first impression follow the instructions of [this](docs/quickstart.md)
3824
tutorial.
3925

40-
# Google Summer of Code
41-
42-
The Postgres Operator made it to the [Google Summer of Code 2019](https://summerofcode.withgoogle.com/)! As a brand new mentoring organization, we are now looking for our first mentees. Check [our ideas](https://github.com/zalando/postgres-operator/blob/master/docs/gsoc-2019/ideas.md#google-summer-of-code-2019) and start discussion in [the issue tracker](https://github.com/zalando/postgres-operator/issues). And don't forget to spread a word about our GSoC participation to attract even more students.
26+
# Documentation
4327

44-
## Table of contents
28+
There is a browser-friendly version of this documentation at
29+
[postgres-operator.readthedocs.io](https://postgres-operator.readthedocs.io)
4530

4631
* [concepts](docs/index.md)
4732
* [user documentation](docs/user.md)
@@ -51,26 +36,16 @@ The Postgres Operator made it to the [Google Summer of Code 2019](https://summer
5136
* [cluster manifest reference](docs/reference/cluster_manifest.md)
5237
* [command-line options and environment variables](docs/reference/command_line_and_environment.md)
5338

54-
The rest of this document is a tutorial to get you up and running locally with the operator on Minikube.
55-
56-
## Overview of involved entities
57-
58-
Here is a diagram, that summarizes what would be created by the operator, when a
59-
new Postgres cluster CRD is submitted:
60-
61-
![postgresql-operator](docs/diagrams/operator.png "K8S resources, created by operator")
62-
63-
This picture is not complete without an overview of what is inside a single cluster pod, so
64-
let's zoom in:
65-
66-
![pod](docs/diagrams/pod.png "Database pod components")
67-
68-
These two diagrams should help you to understand the basics of what kind of
69-
functionality the operator provides.
39+
# Google Summer of Code
7040

71-
There is a browser-friendly version of this documentation at [postgres-operator.readthedocs.io](https://postgres-operator.readthedocs.io)
41+
The Postgres Operator made it to the [Google Summer of Code 2019](https://summerofcode.withgoogle.com/)!
42+
As a brand new mentoring organization, we are now looking for our first mentees.
43+
Check [our ideas](https://github.com/zalando/postgres-operator/blob/master/docs/gsoc-2019/ideas.md#google-summer-of-code-2019)
44+
and start discussion in [the issue tracker](https://github.com/zalando/postgres-operator/issues).
45+
And don't forget to spread a word about our GSoC participation to attract even
46+
more students.
7247

73-
## Community
48+
# Community
7449

7550
There are two places to get in touch with the community:
7651
1. The [GitHub issue tracker](https://github.com/zalando/postgres-operator/issues)

docs/administrator.md

Lines changed: 63 additions & 21 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,7 @@
1-
# How to configure PostgreSQL operator
1+
# Administrator Guide
2+
3+
Learn how to configure and manage the Postgres Operator in your Kubernetes
4+
environment.
25

36
## Select the namespace to deploy to
47

@@ -120,13 +123,17 @@ data:
120123
```
121124

122125
Note that the Kubernetes version 1.13 brings [taint-based eviction](https://kubernetes.io/docs/concepts/configuration/taint-and-toleration/#taint-based-evictions) to the beta stage and enables it by default.
123-
Postgres pods by default receive tolerations for `unreachable` and `noExecute` taints with the timeout of `5m`.
124-
Depending on your setup, you may want to adjust these parameters to prevent master pods from being evicted by the Kubernetes runtime.
125-
To prevent eviction completely, specify the toleration by leaving out the `tolerationSeconds` value (similar to how Kubernetes' own DaemonSets are configured)
126+
Postgres pods by default receive tolerations for `unreachable` and `noExecute`
127+
taints with the timeout of `5m`. Depending on your setup, you may want to adjust
128+
these parameters to prevent master pods from being evicted by the Kubernetes
129+
runtime. To prevent eviction completely, specify the toleration by leaving out
130+
the `tolerationSeconds` value (similar to how Kubernetes' own DaemonSets are
131+
configured)
126132

127133
### Enable pod anti affinity
128134

129-
To ensure Postgres pods are running on different topologies, you can use [pod anti affinity](https://kubernetes.io/docs/concepts/configuration/assign-pod-node/)
135+
To ensure Postgres pods are running on different topologies, you can use
136+
[pod anti affinity](https://kubernetes.io/docs/concepts/configuration/assign-pod-node/)
130137
and configure the required topology in the operator ConfigMap.
131138

132139
Enable pod anti affinity by adding following line to the operator ConfigMap:
@@ -140,9 +147,10 @@ data:
140147
enable_pod_antiaffinity: "true"
141148
```
142149

143-
By default the topology key for the pod anti affinity is set to `kubernetes.io/hostname`,
144-
you can set another topology key e.g. `failure-domain.beta.kubernetes.io/zone` by adding following line
145-
to the operator ConfigMap, see [built-in node labels](https://kubernetes.io/docs/concepts/configuration/assign-pod-node/#interlude-built-in-node-labels) for available topology keys:
150+
By default the topology key for the pod anti affinity is set to
151+
`kubernetes.io/hostname`, you can set another topology key e.g.
152+
`failure-domain.beta.kubernetes.io/zone` by adding following line to the
153+
operator ConfigMap, see [built-in node labels](https://kubernetes.io/docs/concepts/configuration/assign-pod-node/#interlude-built-in-node-labels) for available topology keys:
146154

147155
```yaml
148156
apiVersion: v1
@@ -157,9 +165,9 @@ data:
157165
### Add cluster-specific labels
158166

159167
In some cases, you might want to add `labels` that are specific to a given
160-
postgres cluster, in order to identify its child objects.
161-
The typical use case is to add labels that identifies the `Pods` created by the
162-
operator, in order to implement fine-controlled `NetworkPolicies`.
168+
postgres cluster, in order to identify its child objects. The typical use case
169+
is to add labels that identifies the `Pods` created by the operator, in order
170+
to implement fine-controlled `NetworkPolicies`.
163171

164172
**OperatorConfiguration**
165173

@@ -292,28 +300,62 @@ report success as a result of the last operation applied to them.
292300
The operator is capable of maintaining roles of multiple kinds within a
293301
Postgres database cluster:
294302

295-
* **System roles** are roles necessary for the proper work of Postgres itself such as a replication role or the initial superuser role. The operator delegates creating such roles to Patroni and only establishes relevant secrets.
303+
* **System roles** are roles necessary for the proper work of Postgres itself
304+
such as a replication role or the initial superuser role. The operator delegates
305+
creating such roles to Patroni and only establishes relevant secrets.
296306

297-
* **Infrastructure roles** are roles for processes originating from external systems, e.g. monitoring robots. The operator creates such roles in all Postgres clusters it manages assuming that Kubernetes secrets with the relevant credentials exist beforehand.
307+
* **Infrastructure roles** are roles for processes originating from external
308+
systems, e.g. monitoring robots. The operator creates such roles in all Postgres
309+
clusters it manages assuming that Kubernetes secrets with the relevant
310+
credentials exist beforehand.
298311

299-
* **Per-cluster robot users** are also roles for processes originating from external systems but defined for an individual Postgres cluster in its manifest. A typical example is a role for connections from an application that uses the database.
312+
* **Per-cluster robot users** are also roles for processes originating from
313+
external systems but defined for an individual Postgres cluster in its manifest.
314+
A typical example is a role for connections from an application that uses the
315+
database.
300316

301-
* **Human users** originate from the Teams API that returns a list of the team members given a team id. The operator differentiates between (a) product teams that own a particular Postgres cluster and are granted admin rights to maintain it, and (b) Postgres superuser teams that get the superuser access to all Postgres databases running in a Kubernetes cluster for the purposes of maintaining and troubleshooting.
317+
* **Human users** originate from the Teams API that returns a list of the team
318+
members given a team id. The operator differentiates between (a) product teams
319+
that own a particular Postgres cluster and are granted admin rights to maintain
320+
it, and (b) Postgres superuser teams that get the superuser access to all
321+
Postgres databases running in a Kubernetes cluster for the purposes of
322+
maintaining and troubleshooting.
302323

303324
## Understanding rolling update of Spilo pods
304325

305-
The operator logs reasons for a rolling update with the `info` level and a diff between the old and new StatefulSet specs with the `debug` level. To benefit from numerous escape characters in the latter log entry, view it in CLI with `echo -e`. Note that the resultant message will contain some noise because the `PodTemplate` used by the operator is yet to be updated with the default values used internally in Kubernetes.
326+
The operator logs reasons for a rolling update with the `info` level and a diff
327+
between the old and new StatefulSet specs with the `debug` level. To benefit
328+
from numerous escape characters in the latter log entry, view it in CLI with
329+
`echo -e`. Note that the resultant message will contain some noise because the
330+
`PodTemplate` used by the operator is yet to be updated with the default values
331+
used internally in Kubernetes.
306332

307333
## Logical backups
308334

309-
The operator can manage k8s cron jobs to run logical backups of Postgres clusters. The cron job periodically spawns a batch job that runs a single pod. The backup script within this pod's container can connect to a DB for a logical backup. The operator updates cron jobs during Sync if the job schedule changes; the job name acts as the job identifier. These jobs are to be enabled for each indvidual Postgres cluster by setting `enableLogicalBackup: true` in its manifest. Notes:
335+
The operator can manage k8s cron jobs to run logical backups of Postgres
336+
clusters. The cron job periodically spawns a batch job that runs a single pod.
337+
The backup script within this pod's container can connect to a DB for a logical
338+
backup. The operator updates cron jobs during Sync if the job schedule changes;
339+
the job name acts as the job identifier. These jobs are to be enabled for each
340+
individual Postgres cluster by setting `enableLogicalBackup: true` in its
341+
manifest. Notes:
310342

311-
1. The [example image](../docker/logical-backup/Dockerfile) implements the backup via `pg_dumpall` and upload of compressed and encrypted results to an S3 bucket; the default image ``registry.opensource.zalan.do/acid/logical-backup`` is the same image built with the Zalando-internal CI pipeline. `pg_dumpall` requires a `superuser` access to a DB and runs on the replica when possible.
343+
1. The [example image](../docker/logical-backup/Dockerfile) implements the
344+
backup via `pg_dumpall` and upload of compressed and encrypted results to an S3
345+
bucket; the default image ``registry.opensource.zalan.do/acid/logical-backup``
346+
is the same image built with the Zalando-internal CI pipeline. `pg_dumpall`
347+
requires a `superuser` access to a DB and runs on the replica when possible.
312348

313-
2. Due to the [limitation of Kubernetes cron jobs](https://kubernetes.io/docs/concepts/workloads/controllers/cron-jobs/#cron-job-limitations) it is highly advisable to set up additional monitoring for this feature; such monitoring is outside of the scope of operator responsibilities.
349+
2. Due to the [limitation of Kubernetes cron jobs](https://kubernetes.io/docs/concepts/workloads/controllers/cron-jobs/#cron-job-limitations)
350+
it is highly advisable to set up additional monitoring for this feature; such
351+
monitoring is outside of the scope of operator responsibilities.
314352

315353
3. The operator does not remove old backups.
316354

317-
4. You may use your own image by overwriting the relevant field in the operator configuration. Any such image must ensure the logical backup is able to finish [in presence of pod restarts](https://kubernetes.io/docs/concepts/workloads/controllers/jobs-run-to-completion/#handling-pod-and-container-failures) and [simultaneous invocations](https://kubernetes.io/docs/concepts/workloads/controllers/cron-jobs/#cron-job-limitations) of the backup cron job.
355+
4. You may use your own image by overwriting the relevant field in the operator
356+
configuration. Any such image must ensure the logical backup is able to finish
357+
[in presence of pod restarts](https://kubernetes.io/docs/concepts/workloads/controllers/jobs-run-to-completion/#handling-pod-and-container-failures) and [simultaneous invocations](https://kubernetes.io/docs/concepts/workloads/controllers/cron-jobs/#cron-job-limitations) of the backup cron job.
318358

319-
5. For that feature to work, your RBAC policy must enable operations on the `cronjobs` resource from the `batch` API group for the operator service account. See [example RBAC](../manifests/operator-service-account-rbac.yaml)
359+
5. For that feature to work, your RBAC policy must enable operations on the
360+
`cronjobs` resource from the `batch` API group for the operator service account.
361+
See [example RBAC](../manifests/operator-service-account-rbac.yaml)

docs/index.md

Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -20,6 +20,10 @@ manages PostgreSQL clusters on Kubernetes:
2020
3. Finally, the operator periodically synchronizes the actual state of each
2121
Postgres cluster with the desired state defined in the cluster's manifest.
2222

23+
4. The operator aims to be hands free and configuration happens only via
24+
manifests and its own config. This enables easy integration in automated
25+
deploy pipelines with no access to Kubernetes directly.
26+
2327
## Concepts
2428

2529
### Scope
@@ -37,6 +41,22 @@ experience.
3741
Monitoring of clusters is not in scope, for this good tools already exist from
3842
ZMON to Prometheus and more Postgres specific options.
3943

44+
## Overview of involved entities
45+
46+
Here is a diagram, that summarizes what would be created by the operator, when a
47+
new Postgres cluster CRD is submitted:
48+
49+
![postgresql-operator](diagrams/operator.png "K8S resources, created by operator")
50+
51+
This picture is not complete without an overview of what is inside a single
52+
cluster pod, so let's zoom in:
53+
54+
![pod](diagrams/pod.png "Database pod components")
55+
56+
These two diagrams should help you to understand the basics of what kind of
57+
functionality the operator provides.
58+
59+
4060
### Status
4161

4262
This project is currently in active development. It is however already

0 commit comments

Comments
 (0)