Skip to content

Adding cache and restructuring annotations #492

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 24 commits into from
Oct 26, 2017

Conversation

jgebal
Copy link
Member

@jgebal jgebal commented Oct 14, 2017

  • Removed duplicate old_tests for ut_annotations - the functionality is covered with new tests.
  • Removed parsing of annotation text (params) from ut_annotations as annotation text can be different depending on use case and it's not parsers responsibility to support all use-cases for annotation text.
  • Moved annotations code to separate directory
  • Added filtering of dba_source by test containing annotations.
  • Added cardinality hint with scaling.
  • Changed naming of procedures and variables.
  • Refactored annotations API to use collections of database objects as output results.
  • Reworked ut_suite_manager to operate on sorted collection for building ut_suite and it's tests.
  • Added cache mechanism to ut_annotation_parser.
  • Changed the way schema is scanned, so that all annotations/sources are scanned with single execution of a query.
  • TODO - change the way annotations are parsed, so that we can scan for "floating" annotations.
  • Annotations that are on top of suite, but associated with procedure, are not valid suite-level annotations (This is why the package old_tests/helpers/test_reporters was updated).
  • Refactored annotations tests

Resolves #458

jgebal added 12 commits October 6, 2017 12:46
…notation text can be different depending on use case and it's not parsers responsibility to support all use-cases for annotation text.

Added standalone/separated function `parse_annotation_params` that takes care of annotation text parsing.
…n as output results.

Reworked `ut_suite_manager` to operate on sorted collection for building ut_suite and it's tests.
Adjusted `test_annotation_parser` to cover the refactoring.
Changed the way schema is scanned, so that all annotations/sources are scanned with single execution of a query.
TODO - change the way annotations are parsed, so that we can scan for "floating" annotations.

Annotations that are on top of suite, but associated with procedure, are not valid suite-level annotations.
This is why the package  `old_tests/helpers/test_reporters` was updated.
Added-back public procedure `ut_suite_manager.config_package` for testing purposes (example with huge ut package)
Added filtering of dba_source by test containing annotations.
Added cardinality hint with scaling.
Removed public procedure `config_package` - used only for one unit test.
Fixed naming of procedures.
Performance improvements on collections.
To test wrapped package we need to have an annotation-like text in it.
This way the exception handler `when ex_package_is_wrapped` gets tested after adding source filters
Removed unused code from ut_meta_data.
Improved code comments.
@jgebal jgebal added this to the v3.1.0 milestone Oct 14, 2017
@jgebal jgebal self-assigned this Oct 14, 2017
@jgebal jgebal requested review from viniciusam, pesse and Pazus October 14, 2017 23:10
@ghost ghost added the in progress label Oct 14, 2017
@jgebal jgebal changed the title Addinc cache and restructuring annotations Adding cache and restructuring annotations Oct 14, 2017
@jgebal jgebal force-pushed the feature/annotations_restructuring branch from 8d87e96 to 8e26ed0 Compare October 15, 2017 23:49
@ghost ghost assigned pesse Oct 16, 2017
Copy link
Member

@pesse pesse left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great performance improvement!

@jgebal
Copy link
Member Author

jgebal commented Oct 18, 2017

I see now that it's missing unit tests.
ut_annotation_manager and ut_annotation_cache_manager need to be unit-tested.
I could use some help with that as I will not be able to spend much more time on the project this week.

@Pazus
Copy link
Member

Pazus commented Oct 18, 2017

Could you add a populate_cache procedure to persist the sources without running tests and a method to clear the cache.

@jgebal
Copy link
Member Author

jgebal commented Oct 20, 2017

@Pazus I'm working on it now. Also refactoring the code a bit - better modularity and segregation of responsibility.

* Added ability to build cache in `ut_annotation_manager.rebuild_annotation_cache`
* Added ability to purge cache in `ut_annotation_cache_manager.purge_cache`
Added wrapper procedure `purge_cache` in `ut_annotation_manager`
Changed tests to use wrapper procedure.
…ons_restructuring

# Conflicts:
#	examples/demo_of_expectations/demo_equal_matcher.sql
Added wrapper procedures `purge_cache`, `rebuild_annotation_cache` in `ut_runner`
Added tests for wrappers.
Updated documentation for annotation cache calls.
@jgebal
Copy link
Member Author

jgebal commented Oct 24, 2017

@Pazus
The ability to build / flush cache is now public in ut_runner.

# Conflicts:
#	old_tests/ut_annotations/ut_annotations.parse_package_annotations.ParseAnnotationParamsWithBrackets.sql
@jgebal jgebal merged commit 7037319 into develop Oct 26, 2017
@jgebal jgebal deleted the feature/annotations_restructuring branch October 26, 2017 20:11
@jgebal jgebal modified the milestones: v3.1.0, v3.0.4 Nov 3, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants