Skip to content

Commit 59f0c55

Browse files
alexeyklyukinalexeyklyukin
authored
Allow configuring pod priority globally and per cluster. (zalando#353)
* Allow configuring pod priority globally and per cluster. Allow to specify pod priority class for all pods managed by the operator, as well as for those belonging to individual clusters. Controlled by the pod_priority_class_name operator configuration parameter and the podPriorityClassName manifest option. See https://kubernetes.io/docs/concepts/configuration/pod-priority-preemption/#priorityclass for the explanation on how to define priority classes since Kubernetes 1.8. Some import order changes are due to go fmt. Removal of OrphanDependents deprecated field. Code review by @zerg-junior
1 parent ac7b132 commit 59f0c55

21 files changed

+96
-68
lines changed

‎docs/reference/cluster_manifest.md

Lines changed: 8 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -89,7 +89,14 @@ Those are parameters grouped directly under the `spec` key in the manifest.
8989
examples](https://kubernetes.io/docs/concepts/configuration/taint-and-toleration/)
9090
for details on tolerations and possible values of those keys. When set, this
9191
value overrides the `pod_toleration` setting from the operator. Optional.
92-
92+
93+
* **podPriorityClassName**
94+
a name of the [priority
95+
class](https://kubernetes.io/docs/concepts/configuration/pod-priority-preemption/#priorityclass)
96+
that should be assigned to the cluster pods. When not specified, the value
97+
is taken from the `pod_priority_class_name` operator parameter, if not set
98+
then the default priority class is taken. The priority class itself must be defined in advance.
99+
93100
## Postgres parameters
94101

95102
Those parameters are grouped under the `postgresql` top-level key.

‎docs/reference/operator_parameters.md

Lines changed: 25 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -116,10 +116,15 @@ configuration they are grouped under the `kubernetes` key.
116116
option. If not defined, a simple definition that contains only the name will be used. The default is empty.
117117

118118
* **pod_service_account_role_binding_definition**
119-
This definition must bind pod service account to a role with permission sufficient for the pods to start and for Patroni to access k8s endpoints; service account on its own lacks any such rights starting with k8s v1.8. If not excplicitly defined by the user, a simple definition that binds the account to the operator's own 'zalando-postgres-operator' cluster role will be used. The default is empty.
119+
This definition must bind pod service account to a role with permission
120+
sufficient for the pods to start and for Patroni to access k8s endpoints;
121+
service account on its own lacks any such rights starting with k8s v1.8. If
122+
not excplicitly defined by the user, a simple definition that binds the
123+
account to the operator's own 'zalando-postgres-operator' cluster role will
124+
be used. The default is empty.
120125

121126
* **pod_terminate_grace_period**
122-
Patroni pods are [terminated
127+
Postgres pods are [terminated
123128
forcefully](https://kubernetes.io/docs/concepts/workloads/pods/pod/#termination-of-pods)
124129
after this timeout. The default is `5m`.
125130

@@ -151,7 +156,7 @@ configuration they are grouped under the `kubernetes` key.
151156
name of the secret containing infrastructure roles names and passwords.
152157

153158
* **pod_role_label**
154-
name of the label assigned to the postgres pods (and services/endpoints) by
159+
name of the label assigned to the Postgres pods (and services/endpoints) by
155160
the operator. The default is `spilo-role`.
156161

157162
* **cluster_labels**
@@ -168,7 +173,7 @@ configuration they are grouped under the `kubernetes` key.
168173
considered `ready`. The operator uses values of those labels to detect the
169174
start of the Kubernetes cluster upgrade procedure and move master pods off
170175
the nodes to be decommissioned. When the set is not empty, the operator also
171-
assigns the `Affinity` clause to the postgres pods to be scheduled only on
176+
assigns the `Affinity` clause to the Postgres pods to be scheduled only on
172177
`ready` nodes. The default is empty.
173178

174179
* **toleration**
@@ -184,6 +189,13 @@ configuration they are grouped under the `kubernetes` key.
184189
All variables from that ConfigMap are injected to the pod's environment, on
185190
conflicts they are overridden by the environment variables generated by the
186191
operator. The default is empty.
192+
193+
* **pod_priority_class_name**
194+
a name of the [priority
195+
class](https://kubernetes.io/docs/concepts/configuration/pod-priority-preemption/#priorityclass)
196+
that should be assigned to the Postgres pods. The priority class itself must be defined in advance.
197+
Default is empty (use the default priority class).
198+
187199

188200
## Kubernetes resource requests
189201

@@ -231,8 +243,8 @@ CRD-based configuration.
231243
possible issues faster. The default is `10m`.
232244

233245
* **pod_deletion_wait_timeout**
234-
timeout when waiting for the pods to be deleted when removing the cluster or
235-
recreating pods. The default is `10m`.
246+
timeout when waiting for the Postgres pods to be deleted when removing the
247+
cluster or recreating pods. The default is `10m`.
236248

237249
* **ready_wait_interval**
238250
the interval between consecutive attempts waiting for the postgres CRD to be
@@ -285,18 +297,19 @@ either. In the CRD-based configuration those options are grouped under the
285297

286298
* **wal_s3_bucket**
287299
S3 bucket to use for shipping WAL segments with WAL-E. A bucket has to be
288-
present and accessible by Patroni managed pods. At the moment, supported
289-
services by Spilo are S3 and GCS. The default is empty.
300+
present and accessible by Postgres pods. At the moment, supported services by
301+
Spilo are S3 and GCS. The default is empty.
290302

291303
* **log_s3_bucket**
292304
S3 bucket to use for shipping postgres daily logs. Works only with S3 on AWS.
293-
The bucket has to be present and accessible by Patroni managed pods. At the
294-
moment Spilo does not yet support this. The default is empty.
305+
The bucket has to be present and accessible by Postgres pods. At the moment
306+
Spilo does not yet support this. The default is empty.
295307

296308
* **kube_iam_role**
297-
AWS IAM role to supply in the `iam.amazonaws.com/role` annotation of Patroni
309+
AWS IAM role to supply in the `iam.amazonaws.com/role` annotation of Postgres
298310
pods. Only used when combined with
299-
[kube2iam](https://github.com/jtblin/kube2iam) project on AWS. The default is empty.
311+
[kube2iam](https://github.com/jtblin/kube2iam) project on AWS. The default is
312+
empty.
300313

301314
* **aws_region**
302315
AWS region used to store ESB volumes. The default is `eu-central-1`.

‎pkg/cluster/cluster.go

Lines changed: 10 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -12,11 +12,11 @@ import (
1212
"time"
1313

1414
"github.com/Sirupsen/logrus"
15-
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
16-
"k8s.io/apimachinery/pkg/types"
17-
"k8s.io/api/core/v1"
1815
"k8s.io/api/apps/v1beta1"
16+
"k8s.io/api/core/v1"
1917
policybeta1 "k8s.io/api/policy/v1beta1"
18+
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
19+
"k8s.io/apimachinery/pkg/types"
2020
"k8s.io/client-go/rest"
2121
"k8s.io/client-go/tools/cache"
2222

@@ -91,7 +91,7 @@ type compareStatefulsetResult struct {
9191

9292
// New creates a new cluster. This function should be called from a controller.
9393
func New(cfg Config, kubeClient k8sutil.KubernetesClient, pgSpec spec.Postgresql, logger *logrus.Entry) *Cluster {
94-
orphanDependents := true
94+
deletePropagationPolicy := metav1.DeletePropagationOrphan
9595

9696
podEventsQueue := cache.NewFIFO(func(obj interface{}) (string, error) {
9797
e, ok := obj.(spec.PodEvent)
@@ -113,7 +113,7 @@ func New(cfg Config, kubeClient k8sutil.KubernetesClient, pgSpec spec.Postgresql
113113
Services: make(map[PostgresRole]*v1.Service),
114114
Endpoints: make(map[PostgresRole]*v1.Endpoints)},
115115
userSyncStrategy: users.DefaultUserSyncStrategy{},
116-
deleteOptions: &metav1.DeleteOptions{OrphanDependents: &orphanDependents},
116+
deleteOptions: &metav1.DeleteOptions{PropagationPolicy: &deletePropagationPolicy},
117117
podEventsQueue: podEventsQueue,
118118
KubeClient: kubeClient,
119119
}
@@ -601,7 +601,7 @@ func (c *Cluster) Delete() {
601601
}
602602

603603
for _, obj := range c.Secrets {
604-
if delete, user := c.shouldDeleteSecret(obj); !delete {
604+
if doDelete, user := c.shouldDeleteSecret(obj); !doDelete {
605605
c.logger.Warningf("not removing secret %q for the system user %q", obj.GetName(), user)
606606
continue
607607
}
@@ -951,11 +951,11 @@ func (c *Cluster) deletePatroniClusterEndpoints() error {
951951
return util.NameFromMeta(ep.ObjectMeta), err
952952
}
953953

954-
delete := func(name string) error {
954+
deleteEndpointFn := func(name string) error {
955955
return c.KubeClient.Endpoints(c.Namespace).Delete(name, c.deleteOptions)
956956
}
957957

958-
return c.deleteClusterObject(get, delete, "endpoint")
958+
return c.deleteClusterObject(get, deleteEndpointFn, "endpoint")
959959
}
960960

961961
func (c *Cluster) deletePatroniClusterConfigMaps() error {
@@ -964,9 +964,9 @@ func (c *Cluster) deletePatroniClusterConfigMaps() error {
964964
return util.NameFromMeta(cm.ObjectMeta), err
965965
}
966966

967-
delete := func(name string) error {
967+
deleteConfigMapFn := func(name string) error {
968968
return c.KubeClient.ConfigMaps(c.Namespace).Delete(name, c.deleteOptions)
969969
}
970970

971-
return c.deleteClusterObject(get, delete, "configmap")
971+
return c.deleteClusterObject(get, deleteConfigMapFn, "configmap")
972972
}

‎pkg/cluster/exec.go

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -5,9 +5,9 @@ import (
55
"fmt"
66
"strings"
77

8+
"k8s.io/api/core/v1"
89
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
910
"k8s.io/client-go/kubernetes/scheme"
10-
"k8s.io/api/core/v1"
1111
"k8s.io/client-go/tools/remotecommand"
1212

1313
"github.com/zalando-incubator/postgres-operator/pkg/spec"
@@ -59,9 +59,9 @@ func (c *Cluster) ExecCommand(podName *spec.NamespacedName, command ...string) (
5959
}
6060

6161
err = exec.Stream(remotecommand.StreamOptions{
62-
Stdout: &execOut,
63-
Stderr: &execErr,
64-
Tty: false,
62+
Stdout: &execOut,
63+
Stderr: &execErr,
64+
Tty: false,
6565
})
6666

6767
if err != nil {

‎pkg/cluster/k8sres.go

Lines changed: 16 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -6,6 +6,7 @@ import (
66
"sort"
77

88
"github.com/Sirupsen/logrus"
9+
910
"k8s.io/apimachinery/pkg/api/resource"
1011
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
1112
"k8s.io/apimachinery/pkg/types"
@@ -15,6 +16,7 @@ import (
1516
policybeta1 "k8s.io/api/policy/v1beta1"
1617

1718
"github.com/zalando-incubator/postgres-operator/pkg/spec"
19+
"github.com/zalando-incubator/postgres-operator/pkg/util"
1820
"github.com/zalando-incubator/postgres-operator/pkg/util/constants"
1921
"k8s.io/apimachinery/pkg/labels"
2022
)
@@ -399,6 +401,7 @@ func generatePodTemplate(
399401
terminateGracePeriod int64,
400402
podServiceAccountName string,
401403
kubeIAMRole string,
404+
priorityClassName string,
402405
) (*v1.PodTemplateSpec, error) {
403406

404407
terminateGracePeriodSeconds := terminateGracePeriod
@@ -416,6 +419,10 @@ func generatePodTemplate(
416419
podSpec.Affinity = nodeAffinity
417420
}
418421

422+
if priorityClassName != "" {
423+
podSpec.PriorityClassName = priorityClassName
424+
}
425+
419426
template := v1.PodTemplateSpec{
420427
ObjectMeta: metav1.ObjectMeta{
421428
Labels: labels,
@@ -662,7 +669,7 @@ func (c *Cluster) generateStatefulSet(spec *spec.PostgresSpec) (*v1beta1.Statefu
662669
c.containerName(), c.logger)
663670

664671
// pickup the docker image for the spilo container
665-
effectiveDockerImage := getEffectiveDockerImage(c.OpConfig.DockerImage, spec.DockerImage)
672+
effectiveDockerImage := util.Coalesce(spec.DockerImage, c.OpConfig.DockerImage)
666673

667674
volumeMounts := generateVolumeMounts()
668675

@@ -696,6 +703,7 @@ func (c *Cluster) generateStatefulSet(spec *spec.PostgresSpec) (*v1beta1.Statefu
696703
}
697704

698705
tolerationSpec := tolerations(&spec.Tolerations, c.OpConfig.PodToleration)
706+
effectivePodPriorityClassName := util.Coalesce(spec.PodPriorityClassName, c.OpConfig.PodPriorityClassName)
699707

700708
// generate pod template for the statefulset, based on the spilo container and sidecards
701709
if podTemplate, err = generatePodTemplate(
@@ -707,8 +715,13 @@ func (c *Cluster) generateStatefulSet(spec *spec.PostgresSpec) (*v1beta1.Statefu
707715
nodeAffinity(c.OpConfig.NodeReadinessLabel),
708716
int64(c.OpConfig.PodTerminateGracePeriod.Seconds()),
709717
c.OpConfig.PodServiceAccountName,
710-
c.OpConfig.KubeIAMRole); err != nil {
711-
return nil, fmt.Errorf("could not generate pod template: %v", err)
718+
c.OpConfig.KubeIAMRole,
719+
effectivePodPriorityClassName); err != nil{
720+
return nil, fmt.Errorf("could not generate pod template: %v", err)
721+
}
722+
723+
if err != nil {
724+
return nil, fmt.Errorf("could not generate pod template: %v", err)
712725
}
713726

714727
if volumeClaimTemplate, err = generatePersistentVolumeClaimTemplate(spec.Volume.Size,
@@ -737,13 +750,6 @@ func (c *Cluster) generateStatefulSet(spec *spec.PostgresSpec) (*v1beta1.Statefu
737750
return statefulSet, nil
738751
}
739752

740-
func getEffectiveDockerImage(globalDockerImage, clusterDockerImage string) string {
741-
if clusterDockerImage == "" {
742-
return globalDockerImage
743-
}
744-
return clusterDockerImage
745-
}
746-
747753
func generateScalyrSidecarSpec(clusterName, APIKey, serverURL, dockerImage string,
748754
containerResources *spec.Resources, logger *logrus.Entry) *spec.Sidecar {
749755
if APIKey == "" || dockerImage == "" {

‎pkg/cluster/pod.go

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -4,8 +4,8 @@ import (
44
"fmt"
55
"math/rand"
66

7-
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
87
"k8s.io/api/core/v1"
8+
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
99

1010
"github.com/zalando-incubator/postgres-operator/pkg/spec"
1111
"github.com/zalando-incubator/postgres-operator/pkg/util"

‎pkg/cluster/resources.go

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -5,11 +5,11 @@ import (
55
"strconv"
66
"strings"
77

8-
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
9-
"k8s.io/apimachinery/pkg/types"
10-
"k8s.io/api/core/v1"
118
"k8s.io/api/apps/v1beta1"
9+
"k8s.io/api/core/v1"
1210
policybeta1 "k8s.io/api/policy/v1beta1"
11+
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
12+
"k8s.io/apimachinery/pkg/types"
1313

1414
"github.com/zalando-incubator/postgres-operator/pkg/util"
1515
"github.com/zalando-incubator/postgres-operator/pkg/util/constants"
@@ -272,10 +272,10 @@ func (c *Cluster) replaceStatefulSet(newStatefulSet *v1beta1.StatefulSet) error
272272
c.logger.Debugf("replacing statefulset")
273273

274274
// Delete the current statefulset without deleting the pods
275-
orphanDepencies := true
275+
deletePropagationPolicy := metav1.DeletePropagationOrphan
276276
oldStatefulset := c.Statefulset
277277

278-
options := metav1.DeleteOptions{OrphanDependents: &orphanDepencies}
278+
options := metav1.DeleteOptions{PropagationPolicy: &deletePropagationPolicy}
279279
if err := c.KubeClient.StatefulSets(oldStatefulset.Namespace).Delete(oldStatefulset.Name, &options); err != nil {
280280
return fmt.Errorf("could not delete statefulset %q: %v", statefulSetName, err)
281281
}

‎pkg/cluster/sync.go

Lines changed: 1 addition & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -6,7 +6,6 @@ import (
66

77
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
88
policybeta1 "k8s.io/api/policy/v1beta1"
9-
"k8s.io/api/policy/v1beta1"
109
"k8s.io/api/core/v1"
1110

1211
"github.com/zalando-incubator/postgres-operator/pkg/spec"
@@ -188,7 +187,7 @@ func (c *Cluster) syncEndpoint(role PostgresRole) error {
188187

189188
func (c *Cluster) syncPodDisruptionBudget(isUpdate bool) error {
190189
var (
191-
pdb *v1beta1.PodDisruptionBudget
190+
pdb *policybeta1.PodDisruptionBudget
192191
err error
193192
)
194193
if pdb, err = c.KubeClient.PodDisruptionBudgets(c.Namespace).Get(c.podDisruptionBudgetName(), metav1.GetOptions{}); err == nil {

‎pkg/cluster/util.go

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -11,11 +11,11 @@ import (
1111
"strings"
1212
"time"
1313

14-
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
15-
"k8s.io/apimachinery/pkg/labels"
16-
"k8s.io/api/core/v1"
1714
"k8s.io/api/apps/v1beta1"
15+
"k8s.io/api/core/v1"
1816
policybeta1 "k8s.io/api/policy/v1beta1"
17+
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
18+
"k8s.io/apimachinery/pkg/labels"
1919

2020
"github.com/zalando-incubator/postgres-operator/pkg/spec"
2121
"github.com/zalando-incubator/postgres-operator/pkg/util"

‎pkg/cluster/volumes.go

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -5,9 +5,9 @@ import (
55
"strconv"
66
"strings"
77

8+
"k8s.io/api/core/v1"
89
"k8s.io/apimachinery/pkg/api/resource"
910
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
10-
"k8s.io/api/core/v1"
1111

1212
"github.com/zalando-incubator/postgres-operator/pkg/spec"
1313
"github.com/zalando-incubator/postgres-operator/pkg/util"

‎pkg/controller/controller.go

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -6,11 +6,11 @@ import (
66
"sync"
77

88
"github.com/Sirupsen/logrus"
9+
"k8s.io/api/core/v1"
10+
rbacv1beta1 "k8s.io/api/rbac/v1beta1"
911
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
1012
"k8s.io/apimachinery/pkg/types"
1113
"k8s.io/client-go/kubernetes/scheme"
12-
"k8s.io/api/core/v1"
13-
rbacv1beta1 "k8s.io/api/rbac/v1beta1"
1414
"k8s.io/client-go/tools/cache"
1515

1616
"github.com/zalando-incubator/postgres-operator/pkg/apiserver"

‎pkg/controller/node.go

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,11 +1,11 @@
11
package controller
22

33
import (
4+
"k8s.io/api/core/v1"
45
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
56
"k8s.io/apimachinery/pkg/labels"
67
"k8s.io/apimachinery/pkg/runtime"
78
"k8s.io/apimachinery/pkg/watch"
8-
"k8s.io/api/core/v1"
99

1010
"github.com/zalando-incubator/postgres-operator/pkg/cluster"
1111
"github.com/zalando-incubator/postgres-operator/pkg/util"

‎pkg/controller/node_test.go

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -4,8 +4,8 @@ import (
44
"testing"
55

66
"github.com/zalando-incubator/postgres-operator/pkg/spec"
7-
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
87
"k8s.io/api/core/v1"
8+
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
99
)
1010

1111
const (

0 commit comments

Comments
 (0)