Skip to content

Commit c4f27a0

Browse files
HarshCasperHarshCasper
authored
parity coverage automation (#106)
* start work * try the script out * revamp script * finish up
1 parent 0636ace commit c4f27a0

File tree

4 files changed

+558
-0
lines changed

4 files changed

+558
-0
lines changed

‎.github/bot_templates/PARITY_COVERAGE_DOCS_PR.md

Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,18 @@
1+
# 📖 Parity Metrics Docs Update Report 📖
2+
This PR has been automatically generated to update the AWS parity coverage docs.
3+
It aggregates the latest parity coverage test results from our [test pipeline on GitHub](https://github.com/localstack/localstack/actions/workflows/aws-main.yml?query=branch%3Amaster) as well as from our Pro integration tests.
4+
5+
## 👷🏽 Handle this PR
6+
The following options describe how to interact with this PR / the auto-update:
7+
8+
✔️ **Accept Changes**
9+
If the changes are satisfying, just squash-merge the PR and delete the source branch.
10+
11+
🚫 **Ignore Changes**
12+
If you want to ignore the changes in this PR, just close the PR and *do not delete* the source branch. The PR will not be opened and a new PR will not be created for as long as the generated code does not change (or the branch is deleted). As soon as there are new changes, a new PR will be created.
13+
14+
✏️ **Adapt Changes**
15+
*Don't do this.* The APIs are auto-generated. If you decide that the APIs should look different, you have to change the code-generation.
16+
17+
⏸️ **Pause Updates**
18+
Remove the cron-schedule trigger of the GitHub Action workflow which creates these PRs. The action can then still be triggered manually, but it will not be executed automatically.

‎.github/workflows/docs-parity-updates.yml

Lines changed: 141 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,141 @@
1+
name: Update Parity Docs
2+
3+
on:
4+
schedule:
5+
- cron: 0 5 * * MON
6+
workflow_dispatch:
7+
inputs:
8+
targetBranch:
9+
required: false
10+
type: string
11+
default: 'master'
12+
13+
jobs:
14+
update-parity-coverage-docs:
15+
name: Update Parity Docs
16+
runs-on: ubuntu-latest
17+
steps:
18+
- name: Checkout docs
19+
uses: actions/checkout@v4
20+
with:
21+
fetch-depth: 0
22+
path: docs
23+
ref: ${{ github.event.inputs.targetBranch || 'master' }}
24+
25+
- name: Set up system wide dependencies
26+
run: |
27+
sudo apt-get install jq wget
28+
29+
- name: Set up Python 3.11
30+
id: setup-python
31+
uses: actions/setup-python@v5
32+
with:
33+
python-version: "3.11"
34+
35+
- name: Setup Node
36+
uses: actions/setup-node@v4
37+
with:
38+
node-version: '20'
39+
40+
- name: Download scripts from meta repository
41+
run: |
42+
curl -o /tmp/get_latest_github_metrics.sh -L https://raw.githubusercontent.com/localstack/meta/main/scripts/get_latest_github_metrics.sh -H 'Accept: application/vnd.github.v3.raw'
43+
chmod +x /tmp/get_latest_github_metrics.sh
44+
45+
- name: Download metrics data from Moto Integration test pipeline (GitHub)
46+
working-directory: docs
47+
run: /tmp/get_latest_github_metrics.sh ./target main
48+
env:
49+
GITHUB_TOKEN: ${{ secrets.PRO_ACCESS_TOKEN }}
50+
REPOSITORY_NAME: localstack-moto-test-coverage
51+
ARTIFACT_ID: test-metrics
52+
WORKFLOW: moto-integration-tests
53+
PREFIX_ARTIFACT: moto-integration-test
54+
FILTER_SUCCESS: 0
55+
56+
- name: Download metrics data from Terraform Integration test pipeline (GitHub)
57+
working-directory: docs
58+
run: /tmp/get_latest_github_metrics.sh ./target main
59+
env:
60+
GITHUB_TOKEN: ${{ secrets.PRO_ACCESS_TOKEN }}
61+
REPOSITORY_NAME: localstack-terraform-test
62+
ARTIFACT_ID: test-metrics
63+
WORKFLOW: "Terraform Tests"
64+
FILTER_SUCCESS: 0
65+
66+
- name: Download metrics data from Pro pipeline (GitHub)
67+
working-directory: docs
68+
run: /tmp/get_latest_github_metrics.sh ./target master
69+
env:
70+
GITHUB_TOKEN: ${{ secrets.PRO_ACCESS_TOKEN }}
71+
REPOSITORY_NAME: localstack-ext
72+
ARTIFACT_ID: parity-metric-ext-raw-*
73+
WORKFLOW: "AWS / Build, Test, Push"
74+
PREFIX_ARTIFACT: pro-integration-test
75+
76+
- name: Download coverage (capture-notimplemented) data from Pro pipeline (GitHub)
77+
working-directory: docs
78+
run: /tmp/get_latest_github_metrics.sh ./target master
79+
env:
80+
GITHUB_TOKEN: ${{ secrets.PRO_ACCESS_TOKEN }}
81+
REPOSITORY_NAME: localstack-ext
82+
ARTIFACT_ID: capture-notimplemented-pro
83+
WORKFLOW: "AWS / Build, Test, Push"
84+
RESOURCE_FOLDER: "metrics-implementation-details"
85+
86+
- name: Download metrics data from Community pipeline (GitHub)
87+
working-directory: docs
88+
run: /tmp/get_latest_github_metrics.sh ./target master
89+
env:
90+
GITHUB_TOKEN: ${{ secrets.PRO_ACCESS_TOKEN }}
91+
REPOSITORY_NAME: localstack
92+
ARTIFACT_ID: parity-metric-raw-amd*
93+
WORKFLOW: "AWS / Build, Test, Push"
94+
PREFIX_ARTIFACT: community-integration-test
95+
96+
- name: Download coverage (capture-notimplemented) data from Community pipeline (GitHub)
97+
working-directory: docs
98+
run: /tmp/get_latest_github_metrics.sh ./target master
99+
env:
100+
GITHUB_TOKEN: ${{ secrets.PRO_ACCESS_TOKEN }}
101+
REPOSITORY_NAME: localstack
102+
ARTIFACT_ID: capture-notimplemented
103+
WORKFLOW: "AWS / Build, Test, Push"
104+
RESOURCE_FOLDER: "metrics-implementation-details/community"
105+
106+
- name: Create Parity Coverage Docs
107+
working-directory: docs
108+
run: |
109+
python3 -m scripts.create_data_coverage -i target/metrics-implementation-details -r target/metrics-raw -o target/updated_coverage -s src/data/coverage/service_display_name.json
110+
mv -f target/updated_coverage/data/*.json src/data/coverage
111+
112+
- name: Check for changes
113+
id: check-for-changes
114+
working-directory: docs
115+
run: |
116+
# Check if there are changed files and store the result in resources/diff-check.log
117+
# Check against the PR branch if it exists, otherwise against the master
118+
# Store the result in resources/diff-check.log and store the diff count in the GitHub Action output "diff-count"
119+
mkdir -p resources
120+
(git diff --name-only origin/parity-coverage-auto-updates data/coverage/ 2>/dev/null || git diff --name-only origin/${{ github.event.inputs.targetBranch || 'master' }} src/data/coverage/ 2>/dev/null) | tee -a resources/diff-check.log
121+
echo "diff-count=$(cat resources/diff-check.log | wc -l)" >> $GITHUB_OUTPUT
122+
123+
- name: Read PR markdown template
124+
if: ${{ success() && steps.check-for-changes.outputs.diff-count != '0' && steps.check-for-changes.outputs.diff-count != '' }}
125+
id: template
126+
uses: juliangruber/read-file-action@v1
127+
with:
128+
path: docs/.github/bot_templates/PARITY_COVERAGE_DOCS_PR.md
129+
130+
- name: Create PR
131+
uses: peter-evans/create-pull-request@v7
132+
if: ${{ success() && steps.check-for-changes.outputs.diff-count != '0' && steps.check-for-changes.outputs.diff-count != '' }}
133+
with:
134+
path: docs
135+
title: "Update Parity Coverage Docs"
136+
body: "${{ steps.template.outputs.content }}"
137+
branch: "parity-coverage-auto-updates"
138+
author: "LocalStack Bot <localstack-bot@users.noreply.github.com>"
139+
committer: "LocalStack Bot <localstack-bot@users.noreply.github.com>"
140+
commit-message: "update generated parity coverage docs"
141+
token: ${{ secrets.PRO_ACCESS_TOKEN }}

‎.gitignore

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -33,3 +33,6 @@ __pycache__/
3333

3434
# diff check
3535
resources/diff-check.log
36+
37+
# Parity coverage
38+
target/

0 commit comments

Comments
 (0)