Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Add support for activity/revisions/logs retention (directus#24058)
* Add retention support for `directus_activity` and `directus_revisions` (directus#23410) * add retention support * add changeset * add docs * rework imports * skip processing for no records * improve retention tests * improve tests * add error handling * remove redundant while condition in retention * improve error message * increase retention to 90 days * add lock * Use the action constant * Skip the full table count * Add false to disable in docs * Reduce batch to 500 for db compat * Add timeout to lock * skip revisions if activities is more frequent * Update docs/self-hosted/config-options.md Co-authored-by: Kevin Lewis <kvn@lws.io> * Update docs/self-hosted/config-options.md Co-authored-by: Kevin Lewis <kvn@lws.io> * RETENTION_TASKS are no longer constant * fix formatting for retentions enabled config option * reword RETENTION_SCHEDULE config option * add explicit return types and update to async for consistency * fix schedules tests to be async * update jsdoc to correctly indicate schedule job * add correct default value for retention schedules --------- Co-authored-by: ian <licitdev@gmail.com> Co-authored-by: Kevin Lewis <kvn@lws.io> * Removed outdated logic for versioning (directus#23749) * Extract comments to a separate table (directus#22295) * Add directus_comments migration * Add comments controller and service * Remove from activity * Update system-data and types * Refactor app with new endpoints * Expose service * Update app minimal permissions * Add collection translation * Define relations * Allow comment creation only if there's item read access * Patch for MSSQL double constraints issue * Fix users service test * Add sdk support * Update specs * Fix formatting * Fix specs error * Patch whoopsie * Remove obsolete GraphQL mutations * Update required fields * Remove unused vars * Allow edit and delete of legacy activity comments * Remove legacy comments from SDK * Add changeset * Batch upwards migration * Update SDK to use keysOrQuery Co-authored-by: Brainslug <br41nslug@users.noreply.github.com> * Update implementation for keysOrQuery * Remove singleton check * Update SDK to use keysOrQuery 2 Co-authored-by: Brainslug <br41nslug@users.noreply.github.com> * Update keysOrQuery typedoc * Fix import * Update migration timestamp * fixed import * Update api/src/utils/get-service.ts * utilize chunk processing in migration * formatting * only services extended from itemservice should be added * remove redundant checks from comment header * update comment service to v11 permission format * specify missing required fields * Mock comments in users test * Simplify migration and update date * WIP legacy access * Optimise imports * WIP app cleanup * Update loadUserPreviews typing Co-authored-by: Daniel Biegler <DanielBiegler@users.noreply.github.com> * Read legacy comments * Parse using comments service * Perform migration directly * Fix legacy app sort query which uses id * Migrate legacy comments in mutations * Reduce api semver * Update app recommended permissions * Recommend updating of comment only * replace hardcoded type with existing one * Allow users to update or delete their own comments Co-authored-by: daedalus <44623501+ComfortablyCoding@users.noreply.github.com> * Skip further access validation for non-existent collections * Check if collection exists before the admin check Co-authored-by: daedalus <44623501+ComfortablyCoding@users.noreply.github.com> * Fix incorrect legacy check Co-authored-by: Daniel Biegler <DanielBiegler@users.noreply.github.com> * Fix merging of count when db returns count as string type * Remove unused import --------- Co-authored-by: Brainslug <br41nslug@users.noreply.github.com> Co-authored-by: Pascal Jufer <pascal-jufer@bluewin.ch> Co-authored-by: Brainslug <tim@brainslug.nl> Co-authored-by: daedalus <44623501+ComfortablyCoding@users.noreply.github.com> Co-authored-by: Daniel Biegler <DanielBiegler@users.noreply.github.com> * Consolidate content versioning (directus#22413) * Add migration * Use the new delta field * Add cast-json flag * Fix typing * Fetch existing deltas if version created during migration * Add changeset * Add version delta field into sdk schema * Update migration timestamp * Update versions.save() to return finalVersionDelta Co-authored-by: Pascal Jufer <pascal-jufer@bluewin.ch> * Sort on DB level * Update migration date * Disallow passing delta via create/update * Update docs & specs * Fix save response * Remove unnecessary access check Already checked by the subsequent itemsService.readOne call * Update changeset * Don't require update perms on versions for save * Optimize validateCreateData * update to new validateAccess * Update docs/reference/system/versions.md * Remove migration of delta * Rename to legacy * Add missed changes for Remove migration of delta in 2e2f50f * Update docs/reference/system/versions.md --------- Co-authored-by: Pascal Jufer <pascal-jufer@bluewin.ch> Co-authored-by: daedalus <44623501+ComfortablyCoding@users.noreply.github.com> * Update migration dates * api change should be major for versioning * Remove comment paths from activity reference * Added comments reference * Added directus_comments to table of system collections * The linter demands newline * Revert function renaming for patch semver * Use transaction in down migration for comments (directus#23715) * Remove outdated logic for versioning * Fix migration * Update api/src/database/migrations/20240924B-populate-versioning-deltas.ts * add changeset * reword changeset --------- Co-authored-by: Brainslug <br41nslug@users.noreply.github.com> Co-authored-by: Pascal Jufer <pascal-jufer@bluewin.ch> Co-authored-by: Brainslug <tim@brainslug.nl> Co-authored-by: daedalus <44623501+ComfortablyCoding@users.noreply.github.com> Co-authored-by: Daniel Biegler <DanielBiegler@users.noreply.github.com> Co-authored-by: Kevin Lewis <kvn@lws.io> Co-authored-by: Rijk van Zanten <rijkvanzanten@me.com> * Removed outdated logic for comments (directus#23748) * Extract comments to a separate table (directus#22295) * Add directus_comments migration * Add comments controller and service * Remove from activity * Update system-data and types * Refactor app with new endpoints * Expose service * Update app minimal permissions * Add collection translation * Define relations * Allow comment creation only if there's item read access * Patch for MSSQL double constraints issue * Fix users service test * Add sdk support * Update specs * Fix formatting * Fix specs error * Patch whoopsie * Remove obsolete GraphQL mutations * Update required fields * Remove unused vars * Allow edit and delete of legacy activity comments * Remove legacy comments from SDK * Add changeset * Batch upwards migration * Update SDK to use keysOrQuery Co-authored-by: Brainslug <br41nslug@users.noreply.github.com> * Update implementation for keysOrQuery * Remove singleton check * Update SDK to use keysOrQuery 2 Co-authored-by: Brainslug <br41nslug@users.noreply.github.com> * Update keysOrQuery typedoc * Fix import * Update migration timestamp * fixed import * Update api/src/utils/get-service.ts * utilize chunk processing in migration * formatting * only services extended from itemservice should be added * remove redundant checks from comment header * update comment service to v11 permission format * specify missing required fields * Mock comments in users test * Simplify migration and update date * WIP legacy access * Optimise imports * WIP app cleanup * Update loadUserPreviews typing Co-authored-by: Daniel Biegler <DanielBiegler@users.noreply.github.com> * Read legacy comments * Parse using comments service * Perform migration directly * Fix legacy app sort query which uses id * Migrate legacy comments in mutations * Reduce api semver * Update app recommended permissions * Recommend updating of comment only * replace hardcoded type with existing one * Allow users to update or delete their own comments Co-authored-by: daedalus <44623501+ComfortablyCoding@users.noreply.github.com> * Skip further access validation for non-existent collections * Check if collection exists before the admin check Co-authored-by: daedalus <44623501+ComfortablyCoding@users.noreply.github.com> * Fix incorrect legacy check Co-authored-by: Daniel Biegler <DanielBiegler@users.noreply.github.com> * Fix merging of count when db returns count as string type * Remove unused import --------- Co-authored-by: Brainslug <br41nslug@users.noreply.github.com> Co-authored-by: Pascal Jufer <pascal-jufer@bluewin.ch> Co-authored-by: Brainslug <tim@brainslug.nl> Co-authored-by: daedalus <44623501+ComfortablyCoding@users.noreply.github.com> Co-authored-by: Daniel Biegler <DanielBiegler@users.noreply.github.com> * Consolidate content versioning (directus#22413) * Add migration * Use the new delta field * Add cast-json flag * Fix typing * Fetch existing deltas if version created during migration * Add changeset * Add version delta field into sdk schema * Update migration timestamp * Update versions.save() to return finalVersionDelta Co-authored-by: Pascal Jufer <pascal-jufer@bluewin.ch> * Sort on DB level * Update migration date * Disallow passing delta via create/update * Update docs & specs * Fix save response * Remove unnecessary access check Already checked by the subsequent itemsService.readOne call * Update changeset * Don't require update perms on versions for save * Optimize validateCreateData * update to new validateAccess * Update docs/reference/system/versions.md * Remove migration of delta * Rename to legacy * Add missed changes for Remove migration of delta in 2e2f50f * Update docs/reference/system/versions.md --------- Co-authored-by: Pascal Jufer <pascal-jufer@bluewin.ch> Co-authored-by: daedalus <44623501+ComfortablyCoding@users.noreply.github.com> * Update migration dates * api change should be major for versioning * Remove comment paths from activity reference * Added comments reference * Added directus_comments to table of system collections * The linter demands newline * Revert function renaming for patch semver * Use transaction in down migration for comments (directus#23715) * Remove outdated logic for comments * remove duplicate app access permission * remove unused import/params in gql * remove remaining comment code in activity * remove remaining activity logic in comment * add changeset * reword changeset --------- Co-authored-by: Brainslug <br41nslug@users.noreply.github.com> Co-authored-by: Pascal Jufer <pascal-jufer@bluewin.ch> Co-authored-by: Brainslug <tim@brainslug.nl> Co-authored-by: daedalus <44623501+ComfortablyCoding@users.noreply.github.com> Co-authored-by: Daniel Biegler <DanielBiegler@users.noreply.github.com> Co-authored-by: Kevin Lewis <kvn@lws.io> Co-authored-by: Rijk van Zanten <rijkvanzanten@me.com> * Update config-options.md * add missing docs changeset for retentions * ensure selected id is not ambiguous for joins * Restrict comment create, update and delete to authenticated users (directus#23996) * Extract comments to a separate table (directus#22295) * Add directus_comments migration * Add comments controller and service * Remove from activity * Update system-data and types * Refactor app with new endpoints * Expose service * Update app minimal permissions * Add collection translation * Define relations * Allow comment creation only if there's item read access * Patch for MSSQL double constraints issue * Fix users service test * Add sdk support * Update specs * Fix formatting * Fix specs error * Patch whoopsie * Remove obsolete GraphQL mutations * Update required fields * Remove unused vars * Allow edit and delete of legacy activity comments * Remove legacy comments from SDK * Add changeset * Batch upwards migration * Update SDK to use keysOrQuery Co-authored-by: Brainslug <br41nslug@users.noreply.github.com> * Update implementation for keysOrQuery * Remove singleton check * Update SDK to use keysOrQuery 2 Co-authored-by: Brainslug <br41nslug@users.noreply.github.com> * Update keysOrQuery typedoc * Fix import * Update migration timestamp * fixed import * Update api/src/utils/get-service.ts * utilize chunk processing in migration * formatting * only services extended from itemservice should be added * remove redundant checks from comment header * update comment service to v11 permission format * specify missing required fields * Mock comments in users test * Simplify migration and update date * WIP legacy access * Optimise imports * WIP app cleanup * Update loadUserPreviews typing Co-authored-by: Daniel Biegler <DanielBiegler@users.noreply.github.com> * Read legacy comments * Parse using comments service * Perform migration directly * Fix legacy app sort query which uses id * Migrate legacy comments in mutations * Reduce api semver * Update app recommended permissions * Recommend updating of comment only * replace hardcoded type with existing one * Allow users to update or delete their own comments Co-authored-by: daedalus <44623501+ComfortablyCoding@users.noreply.github.com> * Skip further access validation for non-existent collections * Check if collection exists before the admin check Co-authored-by: daedalus <44623501+ComfortablyCoding@users.noreply.github.com> * Fix incorrect legacy check Co-authored-by: Daniel Biegler <DanielBiegler@users.noreply.github.com> * Fix merging of count when db returns count as string type * Remove unused import --------- Co-authored-by: Brainslug <br41nslug@users.noreply.github.com> Co-authored-by: Pascal Jufer <pascal-jufer@bluewin.ch> Co-authored-by: Brainslug <tim@brainslug.nl> Co-authored-by: daedalus <44623501+ComfortablyCoding@users.noreply.github.com> Co-authored-by: Daniel Biegler <DanielBiegler@users.noreply.github.com> * Consolidate content versioning (directus#22413) * Add migration * Use the new delta field * Add cast-json flag * Fix typing * Fetch existing deltas if version created during migration * Add changeset * Add version delta field into sdk schema * Update migration timestamp * Update versions.save() to return finalVersionDelta Co-authored-by: Pascal Jufer <pascal-jufer@bluewin.ch> * Sort on DB level * Update migration date * Disallow passing delta via create/update * Update docs & specs * Fix save response * Remove unnecessary access check Already checked by the subsequent itemsService.readOne call * Update changeset * Don't require update perms on versions for save * Optimize validateCreateData * update to new validateAccess * Update docs/reference/system/versions.md * Remove migration of delta * Rename to legacy * Add missed changes for Remove migration of delta in 2e2f50f * Update docs/reference/system/versions.md --------- Co-authored-by: Pascal Jufer <pascal-jufer@bluewin.ch> Co-authored-by: daedalus <44623501+ComfortablyCoding@users.noreply.github.com> * Update migration dates * api change should be major for versioning * Remove comment paths from activity reference * Added comments reference * Added directus_comments to table of system collections * The linter demands newline * Revert function renaming for patch semver * Use transaction in down migration for comments (directus#23715) * Remove outdated logic for comments * remove duplicate app access permission * remove unused import/params in gql * remove remaining comment code in activity * remove remaining activity logic in comment * add changeset * reword changeset * restrict comment CUD to authenticated users * update docs * add changeset * skip mention processing if none are present * update restriction wording in the docs --------- Co-authored-by: ian <licitdev@gmail.com> Co-authored-by: Brainslug <br41nslug@users.noreply.github.com> Co-authored-by: Pascal Jufer <pascal-jufer@bluewin.ch> Co-authored-by: Brainslug <tim@brainslug.nl> Co-authored-by: Daniel Biegler <DanielBiegler@users.noreply.github.com> Co-authored-by: Kevin Lewis <kvn@lws.io> Co-authored-by: Rijk van Zanten <rijkvanzanten@me.com> * remove foreign key constraint on comment `collection` column * update migration comment wording * Update api/src/database/migrations/20240924A-migrate-legacy-comments.ts Co-authored-by: ian <licitdev@gmail.com> --------- Co-authored-by: ian <licitdev@gmail.com> Co-authored-by: Kevin Lewis <kvn@lws.io> Co-authored-by: Brainslug <br41nslug@users.noreply.github.com> Co-authored-by: Pascal Jufer <pascal-jufer@bluewin.ch> Co-authored-by: Brainslug <tim@brainslug.nl> Co-authored-by: Daniel Biegler <DanielBiegler@users.noreply.github.com> Co-authored-by: Rijk van Zanten <rijkvanzanten@me.com>
- Loading branch information