Skip to content

Commit dfb9ad1

Browse files
johnstcnjohnstcn
committed
fixup! chore: add documentation for authz and authztest
1 parent 44c02a1 commit dfb9ad1

File tree

2 files changed

+57
-85
lines changed

2 files changed

+57
-85
lines changed

‎coderd/authz/README.md

+1-1
Original file line numberDiff line numberDiff line change
@@ -58,7 +58,7 @@ This can be represented by the following truth table, where Y represents *positi
5858

5959
A *role* is a set of permissions. When evaluating a role's permission to form an action, all the relevant permissions for the role are combined at each level. Permissions at a higher level override permissions at a lower level.
6060

61-
The following table shows the per-level role evaluation.
61+
The following table shows the per-level role evaluation logic.
6262
Y indicates that the role provides positive permissions, N indicates the role provides negative permissions, and _ indicates the role does not provide positive or negative permissions. YN_ indicates that the value in the cell does not matter for the access result.
6363

6464
| Role (example) | Site | Org | User | Result |

‎coderd/authz/authztest/README.md

+56-84
Original file line numberDiff line numberDiff line change
@@ -1,18 +1,12 @@
11
# Authztest
2-
32
Package `authztest` implements _exhaustive_ unit testing for the `authz` package.
4-
53
## Why this exists
6-
74
The `authz.Authorize` function has three* inputs:
85
- Subject (for example, a user or API key)
96
- Resource (for example, a workspace or a DevURL)
107
- Action (for example, read or write).
11-
128
**Not including the ruleset, which we're keeping static for the moment.*
13-
149
Normally to test a pure function like this, you'd write a table test with all of the permutations by hand, for example:
15-
1610
```go
1711
func Test_Authorize(t *testing.T) {
1812
....
@@ -39,55 +33,36 @@ func Test_Authorize(t *testing.T) {
3933
}
4034
}
4135
```
42-
4336
This approach is problematic because of the cardinality of the RBAC model.
44-
4537
Recall that the legacy `pkg/access/authorize`:
4638
- Exposes 8 possible actions, 5 possible site-level roles, 4 possible org-level roles, and 24 possible resource types
4739
- Enforces site-wide versus organization-wide permissions separately
48-
4940
The new authentication model must maintain backward compatibility with this model, whilst allowing additional features such as:
5041
- User-level ownership (which means user-level permission enforcement)
5142
- Resources shared between users (which means permissions granular down to resource IDs)
5243
- Custom roles
53-
5444
The resulting permissions model ([documented in Notion](https://www.notion.so/coderhq/Workspaces-V2-Authz-RBAC-24fd193386eb4cf79a282a2a69e8f917)) results in a large **finite** solution space in the order of **hundreds of millions**.
55-
5645
We want to have a high level of confidence that changes to the implementation **do not have unintended side-effects**.
5746
This means that simply manually writing a set of test cases possibly risks errors slipping through the cracks.
58-
5947
Instead, we generate (almost) all possible sets of inputs to the library, and ensure that `authz.Authorize` performs as expected.
60-
6148
The actual investigation of the solution space is [documented in Notion](https://www.notion.so/coderhq/Authz-Exhaustive-Testing-7683ea694c6e4c12ab0124439916b13a), but the crucial take-away of that document is:
6249
- There is a **large** but **finite** number of possible inputs to `authz.Authorize`,
6350
- The solution space can be broken down into 9 groups, and
6451
- Most importantly, *each group has the same expected result.*
65-
6652
## Testing Methodology
67-
68-
6953
We group the search space into a number of groups. Each group corresponds to a set of test cases with the same expected result. Each group consists of a set of **impactful** permissions and a set of **noise** permissions.
70-
7154
**Impactful** permissions are the top-level permissions that are expected to override anything else, and should be the only inputs that determine the expected result.
7255
**Noise** is simply a set of additional permissions at a lower level that *should not* be impactful.
73-
74-
For each group, we take the **impactful set** of permissions, and add **noise**, and combine this into a role.
75-
56+
For each group, we take the **impactful set** of permissions, and add **noise**, and combine this into a role.
7657
We then take the *set cross-product* of the **impactful set** and the **noise**, and assert that the expected access level of that role to perform a given action.
77-
7858
As some of these sets are quite large, we sample some of the noise to reduce the search space.
79-
80-
TODO: example.
81-
82-
## Permission Permutations
83-
59+
**Example:**
60+
`+site:resource:abc123:create` will always override `-user:resource:*:*`, `-user:*:abc123:*`, `-org:resource:*:create`, and so on. All permutations of those sorts of noise permissions should never change the expected result.
61+
## Role Permutations
8462
Recall that we define a permission as a 4-tuple of `(level, resource_type, resource_id, action)` (for example, `(site, workspace, 123, read)`).
85-
8663
A `Set` is a slice of permissions. The search space of all possible permissions is too large, so instead this package allows generating more meaningful sets for testing. This is equivalent to pruning in AI problems: a technique to reduce the size of the search space by removing parts that do not have significance.
87-
88-
This is the final pruned search space used in authz. Each set is represented by a Y, N, or _. The leftmost set in a row that is not '_' is the impactful set. The impactful set determines the access result. All other sets are non-impactful, and should include the `<nil>` permission. The resulting search space for a row is the cross product between all sets in said row.
89-
90-
64+
This is the final pruned search space used in authz. Each set is represented by a Y, N, or _. The leftmost set in a row that is not '_' is the impactful set. The impactful set determines the access result. All other sets are non-impactful, and should include the `<nil>` permission.
65+
The resulting search space for a row is the cross product between all sets in said row. `+` indicates the union of two sets. For example, Y+_ indicates the union of all positive permissions and abstain permissions.
9166
| Row | * | Site | Org | Org:mem | User | Access |
9267
|-----|------|------|------|---------|------|--------|
9368
| W+ | Y+_ | YN_ | YN_ | YN_ | YN_ | Y |
@@ -102,56 +77,53 @@ This is the final pruned search space used in authz. Each set is represented by
10277
| U- | _ | _ | _ | _ | N+Y_ | N |
10378
| A+ | _ | _ | _ | _ | Y+_ | Y |
10479
| A- | _ | _ | _ | _ | _ | N |
105-
106-
Each row in the above table corresponds to a set of test cases. These are described in the next section.
107-
108-
## Test Cases
109-
110-
There are 12 possible permutations.
111-
112-
### Case 1: W+
113-
114-
TODO
115-
116-
### Case 2: W-
117-
118-
TODO
119-
120-
### Case 3: S+
121-
122-
TODO
123-
124-
### Case 4: S-
125-
126-
TODO
127-
128-
### Case 5: O+
129-
130-
TODO
131-
132-
### Case 6: O-
133-
134-
TODO
135-
136-
### Case 7: M+
137-
138-
TODO
139-
140-
### Case 8: M-
141-
142-
TODO
143-
144-
### Case 9: U+
145-
146-
TODO
147-
148-
### Case 10: U-
149-
150-
TODO
151-
152-
### Case 11: A+
153-
154-
TODO
155-
### Case 12: A-
156-
157-
TODO
80+
Each row in the above table corresponds to a set of role permutations.
81+
There are 12 possible groups of role permutations:
82+
- Case 1 (W+):
83+
- Impactful set: positive wildcard permissions.
84+
- Noise: positive, negative, abstain across site, org, org-member, and user levels.
85+
- Expected result: allow.
86+
- Case 2 (W-):
87+
- Impactful set: negative wildcard permissions.
88+
- Noise: positive, negative, abstain across site, org, org-member, and user levels.
89+
- Expected result: deny.
90+
- Case 3 (S+):
91+
- Impactful set: positive site-level permissions.
92+
- Noise: positive, negative, abstain across org, org-member, and user levels.
93+
- Expected result: allow.
94+
- Case 4 (S-):
95+
- Impactful set: negative site-level permissions.
96+
- Noise: positive, negative, abstain across org, org-member, and user levels.
97+
- Expected result: deny.
98+
- Case 5 (O+):
99+
- Impactful set: positive org-level permissions.
100+
- Noise: positive, negative, abstain across org-member and user levels.
101+
- Expected result: allow.
102+
- Case 6 (O-):
103+
- Impactful set: negative org-level permissions.
104+
- Noise: positive, negative, abstain across org-member and user levels.
105+
- Expected result: deny.
106+
- Case 7 (M+):
107+
- Impactful set: positive org-member permissions.
108+
- Noise: positive, negative, abstain on user level.
109+
- Expected result: allow.
110+
- Case 8 (M-):
111+
- Impactful set: negative org-member permissions.
112+
- Noise: positive, negative, abstain on user level.
113+
- Expected result: deny.
114+
- Case 9 (U+):
115+
- Impactful set: positive user-level permissions.
116+
- Noise: empty set.
117+
- Expected result: allow.
118+
- Case 10 (U-):
119+
- Impactful set: negative user-level permissions.
120+
- Noise: empty set.
121+
- Expected result: deny.
122+
- Case 11 (A+):
123+
- Impactful set: nil permission.
124+
- Noise: positive on user-level.
125+
- Expected result: allow.
126+
- Case 12 (A-):
127+
- Impactful set: nil permission.
128+
- Noise: abstain on user level.
129+
- Expected result: deny.

0 commit comments

Comments
 (0)