Skip to content

Extract controller configuration handling and provide Quarkus extension #238

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 83 commits into from
Dec 18, 2020
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
83 commits
Select commit Hold shift + click to select a range
31fe80c
refactor: extract default finalizer name logic into own method
metacosm Nov 26, 2020
07e4741
feat: Controllers can now be named for configuration purposes
metacosm Nov 26, 2020
89add61
feat: encapsulate controllers' configuration behind an interface
metacosm Nov 26, 2020
08f87a6
refactor: use controller's configuration instead of ControllerUtils
metacosm Nov 26, 2020
240d1df
fix: Controller doesn't have resource class field anymore
metacosm Nov 26, 2020
e5725ec
refactor: move configuration retrieval to ConfigurationService
metacosm Nov 27, 2020
3042075
refactor: rename DefaultConfiguration to AnnotationConfiguration
metacosm Nov 27, 2020
8629f8d
feat: add RetryConfiguration
metacosm Nov 27, 2020
a42d002
fix: change default watchAllNamespaces implementation, add constant
metacosm Nov 27, 2020
0ffeee9
feat: add cluster scope and namespaces support to Controller
metacosm Nov 27, 2020
aaf8a53
refactor: extract default controller name generation to ControllerUtils
metacosm Nov 27, 2020
e310600
feat: initial rewrite of Spring Boot auto-configuration for new format
metacosm Nov 27, 2020
48870b7
fix: provide reasonable defaults
metacosm Nov 27, 2020
819e054
fix: update to new format
metacosm Nov 27, 2020
859e983
fix: update to new format
metacosm Nov 28, 2020
685ba78
feat: initial commit for Quarkus extension
metacosm Nov 28, 2020
a0890ee
feat: add getDoneableClass on ControllerConfiguration
metacosm Dec 1, 2020
1375740
feat: add initial quarkus (not working) sample
metacosm Dec 1, 2020
b6dcbcb
feat: make jar indexable by Quarkus
metacosm Dec 2, 2020
fe42b61
fix: remove unneeded Spring dependency from parent pom
metacosm Dec 2, 2020
c07e262
chore: better names for quarkus extension modules
metacosm Dec 2, 2020
6a43c27
chore: upgrade to Quarkus 1.10.2
metacosm Dec 2, 2020
0b5af44
feat: add getDefaultResourceControllerName method
metacosm Dec 3, 2020
f309d80
feat: import common sample classes in quarkus sample
metacosm Dec 3, 2020
d28e3c0
feat: initial support for build-time configuration, still needs work
metacosm Dec 3, 2020
e968f10
feat: use Doneable from configuration
metacosm Dec 3, 2020
912433f
feat: add fromConfiguration factory method
metacosm Dec 3, 2020
86449d0
feat: add register method that simply uses the controller's config
metacosm Dec 3, 2020
7c7b719
feat: register the controllers using their defined configuration
metacosm Dec 3, 2020
38e7088
refactor: move config package to api
metacosm Dec 3, 2020
c81e43c
refactor: extract api module
metacosm Dec 3, 2020
d7b0224
refactor: move runtime configuration handling into separate module
metacosm Dec 4, 2020
e08d060
fix: code organization and missing changes from rebase
metacosm Dec 4, 2020
7d21132
fix: more re-organization and fixes after rebase
metacosm Dec 4, 2020
8a26cc8
fix: update auto-service to avoid guava conflict with compile-testing
metacosm Dec 4, 2020
34c8b62
refactor: use function reference
metacosm Dec 4, 2020
02fd97b
fix: extract TestCustomResource to work around issue #249
metacosm Dec 4, 2020
f3a7924
fix: add runtime-configuration dependency to process controller
metacosm Dec 4, 2020
32a4dd7
feat: add output of controller class when doneable cannot be found
metacosm Dec 4, 2020
61a271b
feat: switch to SyntheticBean (sync commit)
metacosm Dec 8, 2020
777b53a
fix: use default configuration instead of creating one
metacosm Dec 8, 2020
691d3a1
chore: remove unneeded class
metacosm Dec 8, 2020
a87b3e3
fix: use classes to make sure we have the proper names
metacosm Dec 8, 2020
da34720
fix: make QuarkusControllerConfiguration serializable
metacosm Dec 8, 2020
38e60b3
fix: mark configuration service as runtime init
metacosm Dec 8, 2020
5754bbb
fix: load class using TCCL
metacosm Dec 8, 2020
f8c5929
fix: doneable is generated at the end of build, load it when needed
metacosm Dec 8, 2020
d14761f
feat: remove ClientConfiguration, replace it by fabric8 client config
metacosm Dec 8, 2020
52d99db
fix: remove unneeded ResourceControllerBuildItem
metacosm Dec 8, 2020
4db2a49
chore: update to Quarkus 1.10.3
metacosm Dec 8, 2020
d62dca1
feat: split extension into several steps
metacosm Dec 8, 2020
bc020cd
fix: register controller as bean, not CR class… :facepalm:
metacosm Dec 9, 2020
96e6f58
fix: wait for configuration service before creating the operator
metacosm Dec 9, 2020
2132c4a
fix: avoid unnecessarily loading classes at build time
metacosm Dec 9, 2020
4601f94
fix: Quarkus doesn't like anonymous interface implementations
metacosm Dec 9, 2020
736da3c
fix: make QuarkusControllerConfiguration RecordableConstructor-enabled
metacosm Dec 9, 2020
407a241
fix: make QuarkusConfigurationService also provide ConfigurationService
metacosm Dec 9, 2020
75150db
refactor: simplify and streamline things
metacosm Dec 9, 2020
53fccb9
feat: check that injection properly works
metacosm Dec 9, 2020
8bebad0
chore: re-format
metacosm Dec 9, 2020
5ae707f
chore: use proper version
metacosm Dec 9, 2020
1e905af
chore: remove wrong imports
metacosm Dec 9, 2020
9c95948
chore: restore back to compiling state after rebase
metacosm Dec 9, 2020
cfebbc7
fix: wrong parent version
metacosm Dec 9, 2020
8efc8e2
feat: automatically register detected controllers
metacosm Dec 10, 2020
eedf0d5
chore: remove unneeded dependencies
metacosm Dec 10, 2020
94f3bdd
fix: wrong/duplicated imports
metacosm Dec 11, 2020
dbc5394
fix: update versions in POM files
metacosm Dec 16, 2020
c98f825
fix: restore to compiling state after rebase
metacosm Dec 16, 2020
8f1c8de
fix: move compile-fixtures to proper spot
metacosm Dec 16, 2020
b3066f2
chore: move integration tests to runtime-configuration module
metacosm Dec 17, 2020
465b53f
refactor: fold api module back into operator-framework
metacosm Dec 17, 2020
85c466f
refactor: move classes back to their previous spot
metacosm Dec 17, 2020
495ff39
refactor: rename operator-framework module to operator-framework-core
metacosm Dec 17, 2020
9c1c559
refactor: rename runtime-configuration to operator-framework
metacosm Dec 17, 2020
bebc154
refactor: rename SB starter to follow SB conventions
metacosm Dec 17, 2020
09e3446
refactor: rename quarkus extension for consistency
metacosm Dec 17, 2020
257f4aa
refactor: move test sample / duplicate some to fix integration tests
metacosm Dec 17, 2020
0b1b421
chore: clean-up
metacosm Dec 17, 2020
47cbc10
fix: move CRD descriptors where they're needed
metacosm Dec 17, 2020
d0e3774
fix: move tests where appropriate
metacosm Dec 17, 2020
08b00a3
chore: use static imports
metacosm Dec 17, 2020
06033fc
doc: update README
metacosm Dec 17, 2020
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
57 changes: 51 additions & 6 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -63,14 +63,15 @@ Add [dependency](https://search.maven.org/search?q=a:operator-framework) to your
</dependency>
```

Main method initializing the Operator and registering a controller..
Main method initializing the Operator and registering a controller.

```java
public class Runner {

public static void main(String[] args) {
Operator operator = new Operator(new DefaultKubernetesClient());
operator.registerController(new WebServerController());
Operator operator = new Operator(new DefaultKubernetesClient(),
DefaultConfigurationService.instance());
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's not really easy to understand what DefaultConfigurationService is :(.
I think there is a way to make better before the final release.

Copy link
Collaborator Author

@metacosm metacosm Dec 17, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The problem is to be able to provide the configuration from the annotation processor in the absence of injection. What would you propose to make this simpler/clearer?
The same thing could apply to DefaultKubernetesClient for that matter… it shouldn't be exposed either, IMO.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ConfigurationProvider might be a better name?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

From the user's point of view, there is only one ConfigurationService in the classpath belonging to the imported dependency, so would be best to hide it completely from the user by default.
I'm fine with the current approach and naming though, can become simpler in next iterations.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@psycho-ir the problem is that we cannot put it there by default because then it would be pulled in by the Quarkus implementation as well, which is exactly what I'm trying to avoid 😄

operator.register(new WebServerController());
}
}
```
Expand Down Expand Up @@ -135,18 +136,62 @@ public class WebServerSpec {
}
}
```

#### Quarkus

A [Quarkus](https://quarkus.io) extension is also provided to ease the development of Quarkus-based operators.

Add [this dependency] (https://search.maven.org/search?q=a:operator-framework-quarkus-extension)
to your project:

```xml
<dependency>
<groupId>io.javaoperatorsdk</groupId>
<artifactId>operator-framework-quarkus-extension</artifactId>
<version>{see https://search.maven.org/search?q=a:operator-framework-quarkus-extension for latest version}</version>
</dependency>
```

Create an Application, Quarkus will automatically create and inject a `KubernetesClient`, `Operator`
and `ConfigurationService` instances that your application can use, as shown below:

```java
@QuarkusMain
public class QuarkusOperator implements QuarkusApplication {

@Inject KubernetesClient client;

@Inject Operator operator;

@Inject ConfigurationService configuration;

public static void main(String... args) {
Quarkus.run(QuarkusOperator.class, args);
}

@Override
public int run(String... args) throws Exception {
final var config = configuration.getConfigurationFor(new CustomServiceController(client));
System.out.println("CR class: " + config.getCustomResourceClass());
System.out.println("Doneable class = " + config.getDoneableClass());

Quarkus.waitForExit();
return 0;
}
}
```

#### Spring Boot

You can also let Spring Boot wire your application together and automatically register the controllers.

Add [this dependency](https://search.maven.org/search?q=a:spring-boot-operator-framework-starter) to your project:
Add [this dependency](https://search.maven.org/search?q=a:operator-framework-spring-boot-starter) to your project:

```xml
<dependency>
<groupId>io.javaoperatorsdk</groupId>
<artifactId>spring-boot-operator-framework-starter</artifactId>
<version>{see https://search.maven.org/search?q=a:spring-boot-operator-framework-starter for latest version}</version>
<artifactId>operator-framework-spring-boot-starter</artifactId>
<version>{see https://search.maven.org/search?q=a:operator-framework-spring-boot-starter for latest version}</version>
</dependency>
```

Expand Down
73 changes: 73 additions & 0 deletions operator-framework-core/pom.xml
Original file line number Diff line number Diff line change
@@ -0,0 +1,73 @@
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<parent>
<groupId>io.javaoperatorsdk</groupId>
<artifactId>java-operator-sdk</artifactId>
<version>1.5.1-SNAPSHOT</version>
<relativePath>../pom.xml</relativePath>
</parent>

<artifactId>operator-framework-core</artifactId>
<name>Operator SDK - Framework - Core</name>
<description>Core framework for implementing Kubernetes operators</description>
<packaging>jar</packaging>

<properties>
<java.version>11</java.version>
<maven.compiler.source>11</maven.compiler.source>
<maven.compiler.target>11</maven.compiler.target>
</properties>

<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>${surefire.version}</version>
</plugin>
</plugins>
</build>


<dependencies>
<dependency>
<groupId>io.fabric8</groupId>
<artifactId>openshift-client</artifactId>
</dependency>
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-api</artifactId>
</dependency>

<dependency>
<groupId>org.junit.jupiter</groupId>
<artifactId>junit-jupiter-api</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.junit.jupiter</groupId>
<artifactId>junit-jupiter-engine</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.mockito</groupId>
<artifactId>mockito-core</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.apache.logging.log4j</groupId>
<artifactId>log4j-slf4j-impl</artifactId>
<version>2.13.3</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.assertj</groupId>
<artifactId>assertj-core</artifactId>
<version>3.18.0</version>
<scope>test</scope>
</dependency>
</dependencies>
</project>
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
package io.javaoperatorsdk.operator;

import io.fabric8.kubernetes.client.CustomResource;
import io.javaoperatorsdk.operator.api.ResourceController;
import java.util.Locale;

public class ControllerUtils {

private static final String FINALIZER_NAME_SUFFIX = "/finalizer";

public static String getDefaultFinalizerName(String crdName) {
return crdName + FINALIZER_NAME_SUFFIX;
}

public static boolean hasGivenFinalizer(CustomResource resource, String finalizer) {
return resource.getMetadata().getFinalizers() != null
&& resource.getMetadata().getFinalizers().contains(finalizer);
}

public static String getDefaultNameFor(ResourceController controller) {
return getDefaultNameFor(controller.getClass());
}

public static String getDefaultNameFor(Class<? extends ResourceController> controllerClass) {
return getDefaultResourceControllerName(controllerClass.getSimpleName());
}

public static String getDefaultResourceControllerName(String rcControllerClassName) {
final var lastDot = rcControllerClassName.lastIndexOf('.');
if (lastDot > 0) {
rcControllerClassName = rcControllerClassName.substring(lastDot);
}
return rcControllerClassName.toLowerCase(Locale.ROOT);
}
}
Original file line number Diff line number Diff line change
@@ -1,8 +1,5 @@
package io.javaoperatorsdk.operator;

import static io.javaoperatorsdk.operator.ControllerUtils.getCrdName;
import static io.javaoperatorsdk.operator.ControllerUtils.getCustomResourceClass;

import io.fabric8.kubernetes.api.model.apiextensions.v1beta1.CustomResourceDefinition;
import io.fabric8.kubernetes.client.CustomResource;
import io.fabric8.kubernetes.client.CustomResourceDoneable;
Expand All @@ -13,11 +10,13 @@
import io.fabric8.kubernetes.client.dsl.internal.CustomResourceOperationsImpl;
import io.fabric8.kubernetes.internal.KubernetesDeserializer;
import io.javaoperatorsdk.operator.api.ResourceController;
import io.javaoperatorsdk.operator.api.config.ConfigurationService;
import io.javaoperatorsdk.operator.processing.CustomResourceCache;
import io.javaoperatorsdk.operator.processing.DefaultEventHandler;
import io.javaoperatorsdk.operator.processing.EventDispatcher;
import io.javaoperatorsdk.operator.processing.event.DefaultEventSourceManager;
import io.javaoperatorsdk.operator.processing.event.internal.CustomResourceEventSource;
import io.javaoperatorsdk.operator.processing.retry.GenericRetry;
import io.javaoperatorsdk.operator.processing.retry.Retry;
import java.util.Arrays;
import java.util.HashMap;
Expand All @@ -30,11 +29,21 @@ public class Operator {

private static final Logger log = LoggerFactory.getLogger(Operator.class);
private final KubernetesClient k8sClient;
private final ConfigurationService configurationService;
private Map<Class<? extends CustomResource>, CustomResourceOperationsImpl> customResourceClients =
new HashMap<>();

public Operator(KubernetesClient k8sClient) {
public Operator(KubernetesClient k8sClient, ConfigurationService configurationService) {
this.k8sClient = k8sClient;
this.configurationService = configurationService;
}

public <R extends CustomResource> void register(ResourceController<R> controller)
throws OperatorException {
final var configuration = configurationService.getConfigurationFor(controller);
final var retry = GenericRetry.fromConfiguration(configuration.getRetryConfiguration());
final var targetNamespaces = configuration.getNamespaces().toArray(new String[] {});
registerController(controller, configuration.watchAllNamespaces(), retry, targetNamespaces);
}

public <R extends CustomResource> void registerControllerForAllNamespaces(
Expand Down Expand Up @@ -65,16 +74,14 @@ private <R extends CustomResource> void registerController(
Retry retry,
String... targetNamespaces)
throws OperatorException {
Class<R> resClass = getCustomResourceClass(controller);
final var configuration = configurationService.getConfigurationFor(controller);
Class<R> resClass = configuration.getCustomResourceClass();
CustomResourceDefinitionContext crd = getCustomResourceDefinitionForController(controller);
KubernetesDeserializer.registerCustomKind(crd.getVersion(), crd.getKind(), resClass);
String finalizer = ControllerUtils.getFinalizer(controller);
String finalizer = configuration.getFinalizer();
MixedOperation client =
k8sClient.customResources(
crd,
resClass,
CustomResourceList.class,
ControllerUtils.getCustomResourceDoneableClass(controller));
crd, resClass, CustomResourceList.class, configuration.getDoneableClass());
EventDispatcher eventDispatcher =
new EventDispatcher(
controller, finalizer, new EventDispatcher.CustomResourceFacade(client));
Expand All @@ -98,7 +105,7 @@ private <R extends CustomResource> void registerController(
watchAllNamespaces,
targetNamespaces,
defaultEventHandler,
ControllerUtils.getGenerationEventProcessing(controller),
configuration.isGenerationAware(),
finalizer);
eventSourceManager.registerCustomResourceEventSource(customResourceEventSource);

Expand Down Expand Up @@ -133,7 +140,7 @@ private CustomResourceEventSource createCustomResourceEventSource(

private CustomResourceDefinitionContext getCustomResourceDefinitionForController(
ResourceController controller) {
String crdName = getCrdName(controller);
final var crdName = configurationService.getConfigurationFor(controller).getCRDName();
CustomResourceDefinition customResourceDefinition =
k8sClient.customResourceDefinitions().withName(crdName).get();
if (customResourceDefinition == null) {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -8,10 +8,13 @@
@Retention(RetentionPolicy.RUNTIME)
@Target({ElementType.TYPE})
public @interface Controller {

String NULL = "";

String crdName();

String name() default NULL;

/**
* Optional finalizer name, if it is not, the crdName will be used as the name of the finalizer
* too.
Expand All @@ -24,4 +27,8 @@
* href="https://kubernetes.io/docs/tasks/access-kubernetes-api/custom-resources/custom-resource-definitions/#status-subresource">here</a>
*/
boolean generationAwareEventProcessing() default true;

boolean isClusterScoped() default false;

String[] namespaces() default {};
}
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@

import io.fabric8.kubernetes.client.CustomResource;
import io.javaoperatorsdk.operator.processing.event.EventSourceManager;
import java.util.Locale;

public interface ResourceController<R extends CustomResource> {

Expand Down Expand Up @@ -37,4 +38,20 @@ public interface ResourceController<R extends CustomResource> {
* @param eventSourceManager
*/
default void init(EventSourceManager eventSourceManager) {}

default String getName() {
final var clazz = getClass();

// if the controller annotation has a name attribute, use it
final var annotation = clazz.getAnnotation(Controller.class);
if (annotation != null) {
final var name = annotation.name();
if (!Controller.NULL.equals(name)) {
return name;
}
}

// otherwise, use the lower-cased class name
return clazz.getSimpleName().toLowerCase(Locale.ROOT);
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
package io.javaoperatorsdk.operator.api.config;

import io.fabric8.kubernetes.client.Config;
import io.fabric8.kubernetes.client.CustomResource;
import io.javaoperatorsdk.operator.api.ResourceController;

public interface ConfigurationService {

<R extends CustomResource> ControllerConfiguration<R> getConfigurationFor(
ResourceController<R> controller);

default Config getClientConfiguration() {
return Config.autoConfigure(null);
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
package io.javaoperatorsdk.operator.api.config;

import io.fabric8.kubernetes.client.CustomResource;
import io.fabric8.kubernetes.client.CustomResourceDoneable;
import java.util.Collections;
import java.util.Set;

public interface ControllerConfiguration<R extends CustomResource> {

String WATCH_ALL_NAMESPACES_MARKER = "ALL_NAMESPACES";

String getName();

String getCRDName();

String getFinalizer();

boolean isGenerationAware();

Class<R> getCustomResourceClass();

Class<? extends CustomResourceDoneable<R>> getDoneableClass();

default boolean isClusterScoped() {
return false;
}

default Set<String> getNamespaces() {
return Collections.emptySet();
}

default boolean watchAllNamespaces() {
return getNamespaces().contains(WATCH_ALL_NAMESPACES_MARKER);
}

default RetryConfiguration getRetryConfiguration() {
return RetryConfiguration.DEFAULT;
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
package io.javaoperatorsdk.operator.api.config;

public class DefaultRetryConfiguration implements RetryConfiguration {}
Loading