Skip to content

Refactor _get_metadata_file() from updater.py #1205

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed

Conversation

MVrachev
Copy link
Collaborator

@MVrachev MVrachev commented Nov 10, 2020

Description of the changes being introduced by the pull request:

Currently, the _get_metadata_file() from tuf/client/updater.py is really large
and does too many things and some of them partially.
It validates the specification version and the metadata version in it,
but doesn't validate if the metadata has expired or if the signature could be trusted.
So, it's partially validating the metadata file and at the same time calling
_verify_metadata_file() from updater.py for the rest of the work which doesn't make
sense.

It's logically better if we move the validation of the specification
and the metadata version in separate functions which would
be then called in _verify_metadata_file() and that way move all of the validation
will be done in _verify_metadata_file().

Also, this pr is needed if we want to validate locally available metadata files.
Currently, the problem is that part of the validation cannot be done for local files, because
the spec and metadata versions validation is done in get_metadata_file() which is
used for accessing a metada file somewhere on the internet.

Please verify and check that the pull request fulfills the following
requirements
:

  • The code follows the Code Style Guidelines
  • Tests have been added for the bug fix or new feature
  • Docs have been added for the bug fix or new feature

@MVrachev MVrachev changed the title Refactor get metadata file Refactor _get_metadata_file() from updater.py Nov 10, 2020
@MVrachev MVrachev force-pushed the refactor-get_metadata_file branch 3 times, most recently from e21d343 to 3f154b3 Compare November 11, 2020 14:45
Copy link
Member

@jku jku left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's probably a debate here: do we want to do fixes like this now, does this make e.g. Teodoras work more difficult?

Personally I think there's value in this and the idea is correct: it makes the core update loop easier to read and makes us find issues in it. That said, the update loop feels brittle: let's make sure we do not break it... I left comments in code.

I also think we should try to fix the comments and logging when we touch them (make sure they're correct, useful and not too verbose) -- did not leave comments on those yet

@MVrachev
Copy link
Collaborator Author

MVrachev commented Nov 23, 2020

There's probably a debate here: do we want to do fixes like this now, does this make e.g. Teodoras work more difficult?

It will be best if @sechkova could look into those changes and give an opinion as well.
I need those changes for #1214.
I need to be able to abstract the spec and version verifications so that they could be reused when verifying older versions
of root.json.

I also think we should try to fix the comments and logging when we touch them (make sure they're correct, useful and not too verbose) -- did not leave comments on those yet

Yes, a good reminder.
Will have a look at the comments again.
Please leave more comments if you want me to change a specific comment or line.

@sechkova
Copy link
Contributor

I am by no means against improving the code given that this is the currently used version and the rework is only just a dream ... yet.

@jku
Copy link
Member

jku commented Nov 23, 2020

Please leave more comments if you want me to change a specific comment or line.

From memory: expected_version argument would have the typing hint "Optional[int]" but the possibility of None is not documented in several function docstrings. Both _validate_*() functions are quite long considering how little they do: comments and logging are so verbose.

@MVrachev MVrachev force-pushed the refactor-get_metadata_file branch 2 times, most recently from dc1e0c2 to ab51493 Compare December 7, 2020 14:26
@MVrachev
Copy link
Collaborator Author

MVrachev commented Dec 7, 2020

Updated this pr according to @joshuagl and @jku comments.
Had to rebase because there were changes in tuf/client/updater.py.

@MVrachev MVrachev force-pushed the refactor-get_metadata_file branch from ab51493 to d6f061f Compare December 7, 2020 15:50
Currently, the `_get_metadata_file()` from `tuf/client/updater.py`
is really large and does too many things and some of them partially.

Right now, it validates the specification version
and the metadata version in it, but doesn't validate
if the metadata has expired or if the signature could be trusted.
So, it's partially validating the metadata file
and at the same time calling  `_verify_metadata_file()` from updater.py
for the rest of the work which doesn't make sense.

It's logically better if we move the validation of the specification
and the metadata version in separate functions which would
be then called in ` _verify_metadata_file()` and that way
all of the validation will be done in ` _verify_metadata_file()`.

Signed-off-by: Martin Vrachev <mvrachev@vmware.com>
Signed-off-by: Martin Vrachev <mvrachev@vmware.com>
@MVrachev MVrachev force-pushed the refactor-get_metadata_file branch from d6f061f to c8f0c1c Compare December 7, 2020 15:50
Signed-off-by: Martin Vrachev <mvrachev@vmware.com>
@MVrachev
Copy link
Collaborator Author

After a discussion with @joshuagl, I added the last commit to simplify some of the comments and the way we check for spec conformance.

@MVrachev MVrachev closed this Jan 28, 2021
@MVrachev MVrachev deleted the refactor-get_metadata_file branch January 28, 2021 11:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants