Skip to content

Include CFn tests in test selection #12518

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Draft
wants to merge 14 commits into
base: main
Choose a base branch
from
Draft

Include CFn tests in test selection #12518

wants to merge 14 commits into from

Conversation

simonrw
Copy link
Contributor

@simonrw simonrw commented Apr 11, 2025

  • Ensure CloudFormation tests are executed on service changes

Motivation

TBD

Changes

  • Move test files
  • Validate api gateway tests
  • Revalidate tests

@simonrw simonrw added the semver: patch Non-breaking changes which can be included in patch releases label Apr 11, 2025
@simonrw simonrw self-assigned this Apr 11, 2025
@simonrw simonrw force-pushed the test-selection/include-cfn-tests-in-service-tests branch from 2164675 to a7b9f5a Compare April 11, 2025 15:59
@github-actions GitHub Actions
Copy link

github-actions bot commented Apr 11, 2025 •
edited
Loading

LocalStack Community integration with Pro

    2 files  ±0      2 suites  ±0   1h 53m 40s ⏱️ - 2m 43s
4 354 tests  - 3  4 008 ✅  - 3  346 💤 ±0  0 ❌ ±0 
4 360 runs  +1  4 012 ✅ +1  348 💤 ±0  0 ❌ ±0 

Results for commit 0a67334. ± Comparison against base commit 5ad7f8e.

This pull request removes 136 and adds 133 tests. Note that renamed tests count towards both. 
tests.aws.services.cloudformation.resources.test_acm ‑ test_cfn_acm_certificate
tests.aws.services.cloudformation.resources.test_apigateway ‑ test_account
tests.aws.services.cloudformation.resources.test_apigateway ‑ test_api_gateway_with_policy_as_dict
tests.aws.services.cloudformation.resources.test_apigateway ‑ test_cfn_apigateway_aws_integration
tests.aws.services.cloudformation.resources.test_apigateway ‑ test_cfn_apigateway_rest_api
tests.aws.services.cloudformation.resources.test_apigateway ‑ test_cfn_apigateway_swagger_import
tests.aws.services.cloudformation.resources.test_apigateway ‑ test_cfn_deploy_apigateway_from_s3_swagger
tests.aws.services.cloudformation.resources.test_apigateway ‑ test_cfn_deploy_apigateway_integration
tests.aws.services.cloudformation.resources.test_apigateway ‑ test_cfn_deploy_apigateway_models
tests.aws.services.cloudformation.resources.test_apigateway ‑ test_cfn_with_apigateway_resources
…

tests.aws.services.acm.resources.test_acm ‑ test_cfn_acm_certificate
tests.aws.services.apigateway.resources.test_apigateway ‑ test_account
tests.aws.services.apigateway.resources.test_apigateway ‑ test_api_gateway_with_policy_as_dict
tests.aws.services.apigateway.resources.test_apigateway ‑ test_cfn_apigateway_aws_integration
tests.aws.services.apigateway.resources.test_apigateway ‑ test_cfn_apigateway_rest_api
tests.aws.services.apigateway.resources.test_apigateway ‑ test_cfn_apigateway_swagger_import
tests.aws.services.apigateway.resources.test_apigateway ‑ test_cfn_deploy_apigateway_from_s3_swagger
tests.aws.services.apigateway.resources.test_apigateway ‑ test_cfn_deploy_apigateway_integration
tests.aws.services.apigateway.resources.test_apigateway ‑ test_cfn_deploy_apigateway_models
tests.aws.services.apigateway.resources.test_apigateway ‑ test_cfn_with_apigateway_resources
…
This pull request removes 5 skipped tests and adds 5 skipped tests. Note that renamed tests count towards both. 
tests.aws.services.cloudformation.resources.test_ec2 ‑ test_cfn_update_ec2_instance_type
tests.aws.services.cloudformation.resources.test_lambda ‑ test_lambda_vpc
tests.aws.services.cloudformation.resources.test_opensearch ‑ test_domain
tests.aws.services.cloudformation.resources.test_redshift ‑ test_redshift_cluster
tests.aws.services.cloudformation.resources.test_stepfunctions ‑ test_retry_and_catch

tests.aws.services.ec2.resources.test_ec2 ‑ test_cfn_update_ec2_instance_type
tests.aws.services.lambda_.resources.test_lambda ‑ test_lambda_vpc
tests.aws.services.opensearch.resources.test_opensearch ‑ test_domain
tests.aws.services.redshift.resources.test_redshift ‑ test_redshift_cluster
tests.aws.services.stepfunctions.resources.test_stepfunctions ‑ test_retry_and_catch

♻️ This comment has been updated with latest results.

@simonrw simonrw force-pushed the test-selection/include-cfn-tests-in-service-tests branch from 3ee0658 to 10956fa Compare April 11, 2025 21:08
@github-actions GitHub Actions
Copy link

github-actions bot commented Apr 11, 2025 •
edited
Loading

S3 Image Test Results (AMD64 / ARM64)

    2 files  ±0    2 suites  ±0   8m 0s ⏱️ -4s
  517 tests ±0  467 ✅ ±0   50 💤 ±0  0 ❌ ±0 
1 034 runs  ±0  934 ✅ ±0  100 💤 ±0  0 ❌ ±0 

Results for commit de821d7. ± Comparison against base commit 5bb9747.

♻️ This comment has been updated with latest results.

@simonrw simonrw force-pushed the test-selection/include-cfn-tests-in-service-tests branch 3 times, most recently from 0a67334 to 5432c3f Compare April 16, 2025 15:30
@simonrw simonrw added this to the Playground milestone Apr 25, 2025
@simonrw simonrw force-pushed the test-selection/include-cfn-tests-in-service-tests branch from 5432c3f to 01a3f3f Compare July 30, 2025 21:19
@github-actions GitHub Actions
Copy link

github-actions bot commented Jul 30, 2025 •
edited
Loading

Test Results - Preflight, Unit

22 159 tests  +1   20 422 ✅ +1   6m 23s ⏱️ -2s
     1 suites ±0    1 737 💤 ±0 
     1 files   ±0        0 ❌ ±0 

Results for commit de821d7. ± Comparison against base commit 5bb9747.

♻️ This comment has been updated with latest results.

@github-actions GitHub Actions
Copy link

github-actions bot commented Jul 30, 2025 •
edited
Loading

Test Results (amd64) - Acceptance

7 tests  ±0   5 ✅ ±0   3m 12s ⏱️ +3s
1 suites ±0   2 💤 ±0 
1 files   ±0   0 ❌ ±0 

Results for commit de821d7. ± Comparison against base commit 5bb9747.

♻️ This comment has been updated with latest results.

@github-actions GitHub Actions
Copy link

github-actions bot commented Jul 30, 2025 •
edited
Loading

Test Results (amd64) - Integration, Bootstrap

    5 files      5 suites   2h 23m 20s ⏱️
5 003 tests 4 402 ✅ 601 💤 0 ❌
5 009 runs  4 402 ✅ 607 💤 0 ❌

Results for commit de821d7.

♻️ This comment has been updated with latest results.

@simonrw simonrw force-pushed the test-selection/include-cfn-tests-in-service-tests branch from 4aeada4 to 569f66d Compare August 28, 2025 16:12
@simonrw simonrw added the skip-docs Pull request does not require documentation changes label Aug 28, 2025
@github-actions GitHub Actions
Copy link

github-actions bot commented Aug 28, 2025 •
edited
Loading

Test Results - Alternative Providers

922 tests   674 ✅  31m 56s ⏱️
  4 suites  248 💤
  4 files      0 ❌

Results for commit de821d7.

♻️ This comment has been updated with latest results.

@simonrw simonrw force-pushed the test-selection/include-cfn-tests-in-service-tests branch from fddfda9 to de821d7 Compare August 29, 2025 20:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@dominikschubert dominikschubert Awaiting requested review from dominikschubert dominikschubert will be requested when the pull request is marked ready for review dominikschubert is a code owner

@joe4dev joe4dev Awaiting requested review from joe4dev joe4dev will be requested when the pull request is marked ready for review joe4dev is a code owner

@gregfurman gregfurman Awaiting requested review from gregfurman gregfurman will be requested when the pull request is marked ready for review gregfurman is a code owner

@thrau thrau Awaiting requested review from thrau thrau will be requested when the pull request is marked ready for review thrau is a code owner

@baermat baermat Awaiting requested review from baermat baermat will be requested when the pull request is marked ready for review baermat is a code owner

@bentsku bentsku Awaiting requested review from bentsku bentsku will be requested when the pull request is marked ready for review bentsku is a code owner

@macnev2013 macnev2013 Awaiting requested review from macnev2013 macnev2013 will be requested when the pull request is marked ready for review macnev2013 is a code owner

@k-a-il k-a-il Awaiting requested review from k-a-il k-a-il will be requested when the pull request is marked ready for review k-a-il is a code owner

@giograno giograno Awaiting requested review from giograno giograno will be requested when the pull request is marked ready for review giograno is a code owner

@alexrashed alexrashed Awaiting requested review from alexrashed alexrashed will be requested when the pull request is marked ready for review alexrashed is a code owner

@silv-io silv-io Awaiting requested review from silv-io silv-io will be requested when the pull request is marked ready for review silv-io is a code owner

@pinzon pinzon Awaiting requested review from pinzon pinzon will be requested when the pull request is marked ready for review pinzon is a code owner

@steffyP steffyP Awaiting requested review from steffyP steffyP will be requested when the pull request is marked ready for review steffyP is a code owner

@dfangl dfangl Awaiting requested review from dfangl dfangl will be requested when the pull request is marked ready for review dfangl is a code owner

@sannya-singal sannya-singal Awaiting requested review from sannya-singal sannya-singal will be requested when the pull request is marked ready for review sannya-singal is a code owner

@maxhoheiser maxhoheiser Awaiting requested review from maxhoheiser maxhoheiser will be requested when the pull request is marked ready for review maxhoheiser is a code owner

@viren-nadkarni viren-nadkarni Awaiting requested review from viren-nadkarni viren-nadkarni will be requested when the pull request is marked ready for review viren-nadkarni is a code owner

@cloutierMat cloutierMat Awaiting requested review from cloutierMat cloutierMat will be requested when the pull request is marked ready for review cloutierMat is a code owner

At least 1 approving review is required to merge this pull request.

Assignees

@simonrw simonrw

Labels
semver: patch Non-breaking changes which can be included in patch releases skip-docs Pull request does not require documentation changes
Projects
None yet
Milestone
Playground
Development

Successfully merging this pull request may close these issues.
1 participant
@simonrw