Skip to content

Fix ContinuationSequenceNumber in SubscribeToShard #12981

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 2 commits into
base: main
Choose a base branch
from
Open

Fix ContinuationSequenceNumber in SubscribeToShard #12981

wants to merge 2 commits into from
+76 −45

Conversation

NigelRook
Copy link

Motivation

Currently, when using SubscribeToShard, SubscribeToShardEvents will always have a ContinuationSequenceNumber of 0.

#10000

Changes

When using Kinesis SubscribeToShard, ContinuationSequenceNumber will now match the sequence number of the last record sent, rather than 0

Testing

Tested locally with a WIP consumer

@localstack-bot
Copy link
Contributor

localstack-bot commented Aug 8, 2025 •
edited
Loading

All contributors have signed the CLA ✍️ ✅
Posted by the CLA Assistant Lite bot.

localstack-bot
localstack-bot reviewed Aug 8, 2025
View reviewed changes
Copy link
Contributor

@localstack-bot localstack-bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Welcome to LocalStack! Thanks for raising your first Pull Request and landing in your contributions. Our team will reach out with any reviews or feedbacks that we have shortly. We recommend joining our Slack Community and share your PR on the #community channel to share your contributions with us. Please make sure you are following our contributing guidelines and our Code of Conduct.

@NigelRook
Copy link
Author

I have read the CLA Document and I hereby sign the CLA

@NigelRook
Copy link
Author

recheck

@NigelRook NigelRook force-pushed the fix-continuation-sequence-numbner branch from 950217c to 82651e8 Compare August 8, 2025 16:40
localstack-bot added a commit that referenced this pull request Aug 8, 2025
@localstack-bot
@NigelRook has signed the CLA in #12981
92011fa
@gregfurman gregfurman self-requested a review August 18, 2025 09:07
@gregfurman gregfurman added aws:kinesis Amazon Kinesis semver: minor Non-breaking changes which can be included in minor releases, but not in patch releases labels Aug 18, 2025
@gregfurman gregfurman added this to the 4.8 milestone Aug 18, 2025
@gregfurman gregfurman force-pushed the fix-continuation-sequence-numbner branch from 82651e8 to c7ea85c Compare August 18, 2025 09:09
@alexrashed alexrashed linked an issue Aug 18, 2025 that may be closed by this pull request
bug: Kinesis Subscriber returns SubscribeToShardEvent with invalid sequence number of 0 #10000
Open
1 task
@gregfurman gregfurman force-pushed the fix-continuation-sequence-numbner branch from c7ea85c to a8a64a4 Compare August 18, 2025 09:40
@gregfurman
Copy link
Contributor

Hey @NigelRook! Thanks for the PR 🚀 This approach will 100% close that bug ticket and unblock users, so am fine with it going in. Would it be alright if I pushed some parity tests to this feature branch? You'll retain full authorship of the contribution since we squash commits in anycase 🙂

Just to note, there is a minor distinction we should draw between the value of ContinuationSequenceNumber and an actual record's SequenceNumber. From the AWS docs (bolding is my own):

Use ContinuationSequenceNumber for checkpointing because it captures your shard progress even when no data is written to the shard.

So a snapshot test against AWS would actually fail -- since ContinuationSequenceNumber is a logical checkpoint marker that may not correspond to an actual data record. Implementing this behvaiour is definitely out-of-scope so we can explore it later if need be.

@NigelRook
Copy link
Author

Totally fine with you pushing improvements and/or tests in here

@gregfurman gregfurman force-pushed the fix-continuation-sequence-numbner branch from a8a64a4 to 1770d8a Compare August 19, 2025 13:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@localstack-bot localstack-bot localstack-bot left review comments

@gregfurman gregfurman Awaiting requested review from gregfurman

@dominikschubert dominikschubert Awaiting requested review from dominikschubert dominikschubert is a code owner

@steffyP steffyP Awaiting requested review from steffyP steffyP is a code owner

At least 1 approving review is required to merge this pull request.

Assignees

@NigelRook NigelRook

Labels
aws:kinesis Amazon Kinesis semver: minor Non-breaking changes which can be included in minor releases, but not in patch releases
Projects
None yet
Milestone
4.8
Development

Successfully merging this pull request may close these issues.

bug: Kinesis Subscriber returns SubscribeToShardEvent with invalid sequence number of 0
3 participants
@NigelRook @localstack-bot @gregfurman