Skip to content

[AB Testing PoC] Build ab-testing config into key-values #13897

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Draft
wants to merge 7 commits into
base: commercial/ab-testing-poc
Choose a base branch
from
Draft

[AB Testing PoC] Build ab-testing config into key-values #13897

wants to merge 7 commits into from

Conversation

Jakeii
Copy link
Member

@Jakeii Jakeii commented May 1, 2025 •
edited
Loading

What does this change?

Tweaks to config discussed with the team, groups are now a list of strings and size is defined at the test level and dived by the number of variants.

Tests will be assigned mvt ranges based on their index in the array, the size of the test and the number of groups.

I've removed the allowOverlap: true | false in favour of a testSpace: 0 | 1, which I think is more explicit about what is going on (and has simpler logic).

For 100% tests to run concurrently with other tests, they should be assigned to a different test space. This means that some users will be in multiple tests at the same time, this is what happens at the moment for both client and server side tests, but I think this is more obvious that it is happening.

If someone were to attempt to add a test that exceeds the available mvt space they would get an error, and prompted to use another test space.

@Jakeii Jakeii requested a review from a team May 1, 2025 15:55
@github-actions GitHub Actions
Copy link

github-actions bot commented May 1, 2025 •
edited
Loading

Size Change: 0 B

Total Size: 973 kB

ℹ️ View Unchanged
Filename Size
dotcom-rendering/dist/1008.client.web.********************.js 2.88 kB
dotcom-rendering/dist/1032.client.web.********************.js 3.31 kB
dotcom-rendering/dist/117.client.web.********************.js 2.87 kB
dotcom-rendering/dist/1656.client.web.********************.js 49.6 kB
dotcom-rendering/dist/1823.client.web.********************.js 3.9 kB
dotcom-rendering/dist/2045.client.web.********************.js 2.63 kB
dotcom-rendering/dist/2161.client.web.********************.js 3.88 kB
dotcom-rendering/dist/223.client.web.********************.js 527 B
dotcom-rendering/dist/2270.client.web.********************.js 3.65 kB
dotcom-rendering/dist/2309.client.web.********************.js 3.27 kB
dotcom-rendering/dist/2345.client.web.********************.js 3.38 kB
dotcom-rendering/dist/2360.client.web.********************.js 20 kB
dotcom-rendering/dist/2502.client.web.********************.js 157 B
dotcom-rendering/dist/302.client.web.********************.js 8.84 kB
dotcom-rendering/dist/3226.client.web.********************.js 2.85 kB
dotcom-rendering/dist/3292.client.web.********************.js 2.11 kB
dotcom-rendering/dist/3308.client.web.********************.js 3.12 kB
dotcom-rendering/dist/3485.client.web.********************.js 4.67 kB
dotcom-rendering/dist/3598.client.web.********************.js 3.65 kB
dotcom-rendering/dist/3605.client.web.********************.js 15.1 kB
dotcom-rendering/dist/3619.client.web.********************.js 4.46 kB
dotcom-rendering/dist/3950.client.web.********************.js 3.36 kB
dotcom-rendering/dist/4133.client.web.********************.js 3.1 kB
dotcom-rendering/dist/4258.client.web.********************.js 4.08 kB
dotcom-rendering/dist/4306.client.web.********************.js 22.7 kB
dotcom-rendering/dist/4308.client.web.********************.js 3.57 kB
dotcom-rendering/dist/4324.client.web.********************.js 619 B
dotcom-rendering/dist/4457.client.web.********************.js 6.26 kB
dotcom-rendering/dist/4501.client.web.********************.js 4.29 kB
dotcom-rendering/dist/476.client.web.********************.js 5.42 kB
dotcom-rendering/dist/5144.client.web.********************.js 2.55 kB
dotcom-rendering/dist/5448.client.web.********************.js 4.37 kB
dotcom-rendering/dist/545.client.web.********************.js 531 B
dotcom-rendering/dist/5455.client.web.********************.js 7.15 kB
dotcom-rendering/dist/562.client.web.********************.js 11.5 kB
dotcom-rendering/dist/6038.client.web.********************.js 2.76 kB
dotcom-rendering/dist/6217.client.web.********************.js 3.75 kB
dotcom-rendering/dist/6480.client.web.********************.js 3.68 kB
dotcom-rendering/dist/6540.client.web.********************.js 2.62 kB
dotcom-rendering/dist/6541.client.web.********************.js 2.91 kB
dotcom-rendering/dist/6562.client.web.********************.js 439 B
dotcom-rendering/dist/6584.client.web.********************.js 2.68 kB
dotcom-rendering/dist/6722.client.web.********************.js 3.03 kB
dotcom-rendering/dist/6802.client.web.********************.js 2.51 kB
dotcom-rendering/dist/6873.client.web.********************.js 4.13 kB
dotcom-rendering/dist/695.client.web.********************.js 4.51 kB
dotcom-rendering/dist/7116.client.web.********************.js 23.2 kB
dotcom-rendering/dist/732.client.web.********************.js 3.07 kB
dotcom-rendering/dist/7398.client.web.********************.js 3.57 kB
dotcom-rendering/dist/7487.client.web.********************.js 4.25 kB
dotcom-rendering/dist/7927.client.web.********************.js 2.77 kB
dotcom-rendering/dist/8107.client.web.********************.js 20.3 kB
dotcom-rendering/dist/8151.client.web.********************.js 4.17 kB
dotcom-rendering/dist/8212.client.web.********************.js 5.04 kB
dotcom-rendering/dist/8385.client.web.********************.js 3.88 kB
dotcom-rendering/dist/8387.client.web.********************.js 2.78 kB
dotcom-rendering/dist/8436.client.web.********************.js 3.63 kB
dotcom-rendering/dist/8602.client.web.********************.js 3.39 kB
dotcom-rendering/dist/8619.client.web.********************.js 4.44 kB
dotcom-rendering/dist/864.client.web.********************.js 3.17 kB
dotcom-rendering/dist/8942.client.web.********************.js 4.43 kB
dotcom-rendering/dist/8973.client.web.********************.js 16.5 kB
dotcom-rendering/dist/8985.client.web.********************.js 11.1 kB
dotcom-rendering/dist/9018.client.web.********************.js 3.73 kB
dotcom-rendering/dist/9135.client.web.********************.js 3.54 kB
dotcom-rendering/dist/9139.client.web.********************.js 3.13 kB
dotcom-rendering/dist/9376.client.web.********************.js 2.52 kB
dotcom-rendering/dist/960.client.web.********************.js 7.99 kB
dotcom-rendering/dist/967.client.web.********************.js 2.87 kB
dotcom-rendering/dist/9709.client.web.********************.js 2.73 kB
dotcom-rendering/dist/9822.client.web.********************.js 7.63 kB
dotcom-rendering/dist/9837.client.web.********************.js 3.29 kB
dotcom-rendering/dist/9870.client.web.********************.js 4.77 kB
dotcom-rendering/dist/Accessibility-importable.client.web.********************.js 8.43 kB
dotcom-rendering/dist/AdBlockAsk-importable.client.web.********************.js 2.99 kB
dotcom-rendering/dist/AdPortals-importable.client.web.********************.js 4.76 kB
dotcom-rendering/dist/AlreadyVisited-importable.client.web.********************.js 425 B
dotcom-rendering/dist/AppsEpic-importable.client.web.********************.js 3.61 kB
dotcom-rendering/dist/AppsFooter-importable.client.web.********************.js 2.69 kB
dotcom-rendering/dist/AppsLightboxImage-importable.client.web.********************.js 2.66 kB
dotcom-rendering/dist/AppsLightboxImageStore-importable.client.web.********************.js 2.61 kB
dotcom-rendering/dist/AudioAtomWrapper-importable.client.web.********************.js 2.76 kB
dotcom-rendering/dist/AudioPlayerWrapper-importable.client.web.********************.js 6.61 kB
dotcom-rendering/dist/AustralianTerritorySwitcher-importable.client.web.********************.js 4.61 kB
dotcom-rendering/dist/Branding-importable.client.web.********************.js 2.89 kB
dotcom-rendering/dist/braze-web-sdk-core.client.web.********************.js 67.7 kB
dotcom-rendering/dist/BrazeMessaging-importable.client.web.********************.js 1.68 kB
dotcom-rendering/dist/CalloutBlockComponent-importable.client.web.********************.js 6.73 kB
dotcom-rendering/dist/CalloutEmbedBlockComponent-importable.client.web.********************.js 5.77 kB
dotcom-rendering/dist/CardCommentCount-importable.client.web.********************.js 2.67 kB
dotcom-rendering/dist/Carousel-importable.client.web.********************.js 6.67 kB
dotcom-rendering/dist/CarouselForNewsletters-importable.client.web.********************.js 4.55 kB
dotcom-rendering/dist/ChartAtom-importable.client.web.********************.js 539 B
dotcom-rendering/dist/CommentCount-importable.client.web.********************.js 2.3 kB
dotcom-rendering/dist/CrosswordComponent-importable.client.web.********************.js 2.88 kB
dotcom-rendering/dist/DiscussionApps-importable.client.web.********************.js 1.07 kB
dotcom-rendering/dist/DiscussionMeta-importable.client.web.********************.js 2.41 kB
dotcom-rendering/dist/DiscussionWeb-importable.client.web.********************.js 3.45 kB
dotcom-rendering/dist/DocumentBlockComponent-importable.client.web.********************.js 2.85 kB
dotcom-rendering/dist/Dropdown-importable.client.web.********************.js 1.72 kB
dotcom-rendering/dist/EditionSwitcherBanner-importable.client.web.********************.js 4.43 kB
dotcom-rendering/dist/EmbedBlockComponent-importable.client.web.********************.js 3.98 kB
dotcom-rendering/dist/EnhancePinnedPost-importable.client.web.********************.js 2.02 kB
dotcom-rendering/dist/FetchOnwardsData-importable.client.web.********************.js 1.94 kB
dotcom-rendering/dist/FilterKeyEventsToggle-importable.client.web.********************.js 3.72 kB
dotcom-rendering/dist/FocusStyles-importable.client.web.********************.js 618 B
dotcom-rendering/dist/FollowWrapper-importable.client.web.********************.js 2.53 kB
dotcom-rendering/dist/FootballMatchesPageWrapper-importable.client.web.********************.js 7.2 kB
dotcom-rendering/dist/FootballTablesCompetitionSelect-importable.client.web.********************.js 3.26 kB
dotcom-rendering/dist/FooterLabel-importable.client.web.********************.js 364 B
dotcom-rendering/dist/FooterReaderRevenueLinks-importable.client.web.********************.js 3.43 kB
dotcom-rendering/dist/frameworks.client.web.********************.js 20.9 kB
dotcom-rendering/dist/FrontSubNav-importable.client.web.********************.js 7.5 kB
dotcom-rendering/dist/GetCricketScoreboard-importable.client.web.********************.js 6.29 kB
dotcom-rendering/dist/GetMatchNav-importable.client.web.********************.js 11.5 kB
dotcom-rendering/dist/GetMatchStats-importable.client.web.********************.js 8.1 kB
dotcom-rendering/dist/GetMatchTabs-importable.client.web.********************.js 2.58 kB
dotcom-rendering/dist/guardian-braze-components-banner.client.web.********************.js 16.1 kB
dotcom-rendering/dist/guardian-braze-components-end-of-article.client.web.********************.js 10.2 kB
dotcom-rendering/dist/GuideAtomWrapper-importable.client.web.********************.js 782 B
dotcom-rendering/dist/index.client.web.********************.js 46.4 kB
dotcom-rendering/dist/InstagramBlockComponent-importable.client.web.********************.js 2.9 kB
dotcom-rendering/dist/InteractiveAtomMessenger-importable.client.web.********************.js 854 B
dotcom-rendering/dist/InteractiveBlockComponent-importable.client.web.********************.js 8.83 kB
dotcom-rendering/dist/InteractiveContentsBlockComponent-importable.client.web.********************.js 3.75 kB
dotcom-rendering/dist/KeyEventsCarousel-importable.client.web.********************.js 5.69 kB
dotcom-rendering/dist/KnowledgeQuizAtom-importable.client.web.********************.js 3.19 kB
dotcom-rendering/dist/LatestLinks-importable.client.web.********************.js 7.96 kB
dotcom-rendering/dist/LightboxHash-importable.client.web.********************.js 436 B
dotcom-rendering/dist/LightboxLayout-importable.client.web.********************.js 6.56 kB
dotcom-rendering/dist/LiveBlogEpic-importable.client.web.********************.js 3.56 kB
dotcom-rendering/dist/LiveblogGutterAskWrapper-importable.client.web.********************.js 2.49 kB
dotcom-rendering/dist/LiveblogNotifications-importable.client.web.********************.js 4.84 kB
dotcom-rendering/dist/Liveness-importable.client.web.********************.js 4.71 kB
dotcom-rendering/dist/LoopVideo-importable.client.web.********************.js 3.54 kB
dotcom-rendering/dist/ManyNewsletterSignUp-importable.client.web.********************.js 7.65 kB
dotcom-rendering/dist/MapEmbedBlockComponent-importable.client.web.********************.js 6.04 kB
dotcom-rendering/dist/Metrics-importable.client.web.********************.js 2.7 kB
dotcom-rendering/dist/MostViewedFooter-importable.client.web.********************.js 3.85 kB
dotcom-rendering/dist/MostViewedFooterData-importable.client.web.********************.js 5.97 kB
dotcom-rendering/dist/MostViewedRightWithAd-importable.client.web.********************.js 5.24 kB
dotcom-rendering/dist/OnwardsUpper-importable.client.web.********************.js 5.32 kB
dotcom-rendering/dist/PersonalityQuizAtom-importable.client.web.********************.js 3.35 kB
dotcom-rendering/dist/ProfileAtom-importable.client.web.********************.js 545 B
dotcom-rendering/dist/ProfileAtomWrapper-importable.client.web.********************.js 805 B
dotcom-rendering/dist/PulsingDot-importable.client.web.********************.js 749 B
dotcom-rendering/dist/QandaAtom-importable.client.web.********************.js 542 B
dotcom-rendering/dist/ReaderRevenueDev-importable.client.web.********************.js 470 B
dotcom-rendering/dist/readerRevenueDevUtils.client.web.********************.js 1.7 kB
dotcom-rendering/dist/RelativeTime-importable.client.web.********************.js 2.56 kB
dotcom-rendering/dist/RichLinkComponent-importable.client.web.********************.js 6.14 kB
dotcom-rendering/dist/ScrollableFeature-importable.client.web.********************.js 6.63 kB
dotcom-rendering/dist/ScrollableHighlights-importable.client.web.********************.js 7.26 kB
dotcom-rendering/dist/ScrollableMedium-importable.client.web.********************.js 2.11 kB
dotcom-rendering/dist/ScrollableSmall-importable.client.web.********************.js 2.17 kB
dotcom-rendering/dist/SecureSignup-importable.client.web.********************.js 4.2 kB
dotcom-rendering/dist/SendTargetingParams-importable.client.web.********************.js 2.21 kB
dotcom-rendering/dist/sentry.client.web.********************.js 802 B
dotcom-rendering/dist/SetABTests-importable.client.web.********************.js 3.91 kB
dotcom-rendering/dist/SetAdTargeting-importable.client.web.********************.js 487 B
dotcom-rendering/dist/ShareButton-importable.client.web.********************.js 2.18 kB
dotcom-rendering/dist/shimport.client.web.********************.js 2.8 kB
dotcom-rendering/dist/ShowHideContainers-importable.client.web.********************.js 893 B
dotcom-rendering/dist/ShowMore-importable.client.web.********************.js 919 B
dotcom-rendering/dist/SignInGateMain.client.web.********************.js 4.35 kB
dotcom-rendering/dist/SignInGateMainCheckoutComplete.client.web.********************.js 5.46 kB
dotcom-rendering/dist/SignInGateSelector-importable.client.web.********************.js 6.67 kB
dotcom-rendering/dist/SlideshowCarousel-importable.client.web.********************.js 4.56 kB
dotcom-rendering/dist/SlotBodyEnd-importable.client.web.********************.js 4.91 kB
dotcom-rendering/dist/SpotifyBlockComponent-importable.client.web.********************.js 5.8 kB
dotcom-rendering/dist/StickyBottomBanner-importable.client.web.********************.js 4.04 kB
dotcom-rendering/dist/SubNav-importable.client.web.********************.js 2.44 kB
dotcom-rendering/dist/TableOfContents-importable.client.web.********************.js 3.52 kB
dotcom-rendering/dist/TimelineAtom-importable.client.web.********************.js 1.23 kB
dotcom-rendering/dist/Titlepiece-importable.client.web.********************.js 13.6 kB
dotcom-rendering/dist/TopBar-importable.client.web.********************.js 8.85 kB
dotcom-rendering/dist/TopBarSupport-importable.client.web.********************.js 2.5 kB
dotcom-rendering/dist/TweetBlockComponent-importable.client.web.********************.js 1.13 kB
dotcom-rendering/dist/UnsafeEmbedBlockComponent-importable.client.web.********************.js 2.92 kB
dotcom-rendering/dist/VideoFacebookBlockComponent-importable.client.web.********************.js 6.06 kB
dotcom-rendering/dist/VineBlockComponent-importable.client.web.********************.js 3.32 kB
dotcom-rendering/dist/YoutubeBlockComponent-importable.client.web.********************.js 843 B

compressed-size-action

@github-actions GitHub Actions
Copy link

github-actions bot commented May 1, 2025 •
edited
Loading

Deploy build 18324 of dotcom:rendering-all to CODE

All deployment options

From guardian/actions-riff-raff.

@github-actions GitHub Actions
Copy link

github-actions bot commented May 6, 2025 •
edited
Loading

Deploy build 42528 of Mobile::mobile-apps-rendering to CODE

All deployment options

From guardian/actions-riff-raff.

@Jakeii Jakeii force-pushed the jlk/ab-testing/build branch from ae501d9 to 9d7ef45 Compare May 7, 2025 15:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
Commercial 💰
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
1 participant
@Jakeii