refactor!: Adapt to the Crawlee v1.0 #470
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
1 task
janbuchar
reviewed
May 16, 2025
d27c080 to
82efd3e
Compare
067b793 to
104a168
Compare
dc7f0a7 to
a3d68a2
Compare
janbuchar
reviewed
Jun 26, 2025
| @@ -11,14 +11,14 @@ async def main() -> None: | |||
| await dataset.export_to( | |||
| content_type='csv', | |||
| key='data.csv', | |||
| to_key_value_store_name='my-cool-key-value-store', | |||
| to_kvs_name='my-cool-key-value-store', | |||
There was a problem hiding this comment.
Is this BC break worth it?
There was a problem hiding this comment.
let's evaluate all the potential BCs at the end
There was a problem hiding this comment.
Sure. I thought we are nearing that now 😁
There was a problem hiding this comment.
Since we're just re-exporting the storages from crawlee here, there will be many more cases than this one. I'm not saying we have to rename this particular argument (and I will undo it if you insist—just I don't like those long identifiers, especially when we can use the KVS abbreviation).
vdusek
added a commit
that referenced
this pull request
Jul 28, 2025
- Add more integration tests for RQ (mostly to better test the new Apify RQ in the #470). - Utilization of `Actor.log` - as they are seen in the test output.
d59b100 to
f5189c5
Compare
dc6885b to
ae3044e
Compare
Drop forefront info from local chache as it is unreliable and complicates the flow. Only the platform knows the real forefront, do not try to keep imperfect copy of it locally and rather design the system to work without being aware of the cached forefront.
Pijukatel
approved these changes
Aug 12, 2025
Mantisus
reviewed
Aug 12, 2025
| @@ -0,0 +1,18 @@ | |||
| --- | |||
| id: upgrading-to-v2 | |||
| title: Upgrading to v2 | |||
|
All the issues in integration tests are resolved. Let's resolve the remaining things (deduplication, upgrading guide, ...) in separate PRs. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
Issues
test_actor_on_platform_max_request_retriesflakiness #540Testing