Skip to content

chore: Rewrite rbac rego -> SQL clause #5138

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 23 commits into from
Nov 28, 2022
Merged

Conversation

Emyrk
Copy link
Member

@Emyrk Emyrk commented Nov 20, 2022

What this does

The previous code became hard to follow as we added more edge cases. All these edge cases happened because of our jsonb columns in SQL. This new code removes all those edge cases from the general code, as has the exceptions passed into the package. This code handles our custom jsonb columns:

https://github.com/coder/coder/blob/stevenmasley/rego_to_sql_redo/coderd/rbac/regosql/aclGroupVar.go

The previous code had a lot of stuff like this that was hard to follow. Changing it would cause random side effects, something we don't want in our auth code.

Implementation Notes

GetAuthorizedTemplates

I added GetAuthorizedTemplates and added a great unit test which tests all this SQL.

Testing

Testing is critical as this package generates SQL clauses, but does not actually execute the SQL clauses. It is possible the SQL that is generated is either syntactically incorrect for SQL, or behaviorally incorrect. It is up to us to come up with the SQL we desire, and make unit tests with these asserted cases.

Implementing Types

Types implement their supported operations on the types themselves. This is opposed to implementing a nested conditional structure such as:

// Trying to do a = b
switch a.(type) {
  case string:
    switch b.(type): 
      case string:
        return fmt.Sprintf("%s = %s, a, b)
}

This nesting structure blows up quick and becomes very difficult to follow. Implementing it on the type is much more straight forward and easier to understand. It also allows for easy recursion for things like (a = b) = (c = d). This is 2 equalities being compared, and in the structure above things like this become quite a hassle.

The other benefit to this, is that we can override behavior outside the package. The biggest problem in the first implementation is that there is no easy way to handle arbitrary jsonb types. I made an attempt at this by creating a Tree type for variables, but it's unnecessarily complex for the 1 column type we care about that is the exception. In this new design the exception is a custom type passed into the generator. So the generation code does not need to handle these like it did before.

@Emyrk Emyrk marked this pull request as ready for review November 21, 2022 18:38
@Emyrk Emyrk requested a review from kylecarbs November 21, 2022 18:39
Copy link
Member

@kylecarbs kylecarbs left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approving this so the improvement can get in. Did you run the benchmarks against this change?

@Emyrk
Copy link
Member Author

Emyrk commented Nov 23, 2022

Did you run the benchmarks against this change?

We do not have a benchmark function for the resulting SQL. The convert part is pretty quick, and this change won't make that any faster. All the cost is in the SQL execution.

Edit: I just ran a test against main. It is taking around 10ms for both this branch and main to run this conversion. Which is mostly spent in the rego prep (which has not changed). This cost was already known. I am curious to see how efficient the SQL query is on large datasets.

Right now I do not make any attempt to optimize the SQL, but we absolutely can if needed in the future.

Copy link
Member

@mafredri mafredri left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great, there's quite a few bits I don't fully understand due to being unfamiliar with rego, but it looks solid. It'd be nice to find an alternative solution for re-writing the generated DB query functions, but not sure what that would look like.

Maybe sqlc plugins could be utilized there? 🤔 https://docs.sqlc.dev/en/latest/guides/plugins.html

@Emyrk
Copy link
Member Author

Emyrk commented Nov 28, 2022

Looks great, there's quite a few bits I don't fully understand due to being unfamiliar with rego, but it looks solid. It'd be nice to find an alternative solution for re-writing the generated DB query functions, but not sure what that would look like.

Maybe sqlc plugins could be utilized there? thinking https://docs.sqlc.dev/en/latest/guides/plugins.html

I have not looked into their plugins. That would be awesome if we could extend sqlc and not write custom db code.

@Emyrk
Copy link
Member Author

Emyrk commented Nov 28, 2022

@kylecarbs just pinging about what @mafredri linked: https://docs.sqlc.dev/en/latest/guides/plugins.html. Would be cool to use that instead of modelqueries.go

Copy link
Contributor

@f0ssel f0ssel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I did not review the code in the rego directory since I was very unfamiliar with it, but everything else regarding coderd seems reasonable. Agree about extending sqlc, we know that we can with plugins but because you need to integrate with a standalone binary it's more work that just some code. I'd be very happy if we did the upfront work to make a plugin if it resulted in our modelqueries.go workaround going away.

@Emyrk Emyrk merged commit ab9298f into main Nov 28, 2022
@Emyrk Emyrk deleted the stevenmasley/rego_to_sql_redo branch November 28, 2022 18:12
@github-actions github-actions bot locked and limited conversation to collaborators Nov 28, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants