Skip to content

[Serializer] symfony#36594 attributes cache breaks normalization #43469

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Conversation

ivannemets-sravniru
Copy link

@ivannemets-sravniru ivannemets-sravniru commented Oct 13, 2021

Q A
Branch? 5.4
Bug fix? yes
New feature? no
Deprecations? no
Tickets Fix #36594
License MIT
Doc PR symfony/symfony-docs#15823

The bug itself is explained in the following (simplified) example:

class Dummy
{
    public array $requiredData; // supposed to be set for any object
    public array $optionalData; // can be not initialized (and if so - ignored by serializer)
}

$object1 = new Dummy();
$object1->requiredData = ['username' => 'foo'];
$json1 = $serializer->serialize($object1, 'json'); // {"requiredData": {"username": "foo"}}
// at this point object normalizer has already cached attributes for Dummy::class and context,
// now it contains array ['requiredData'] - optionalData has been ignored as it's unitialized

// then, while the script is still running, we have another object of the same class with optionalData set
$object2 = new Dummy();
$object2->requiredData = ['username' => 'bar'];
$object2->optionalData = ['email' => 'bar@test.com'];
$json2 = $serializer->serialize($object2, 'json');
// expected: {"requiredData": {"username": "bar"}, "optionalData": {"email": "bar@test.com"}}
// actual: {"requiredData": {"username": "bar"}}
// here normalizer has no clue about optionalData attribute since it reuses attributes cached
// in \Symfony\Component\Serializer\Normalizer\AbstractObjectNormalizer::$attributesCache
// while normalizing the first object

Though this PR created for 5.4 branch, it actually fixes a bug reproducible in 5.3. The reason why I use 5.4 for this fix is that 5.4 introduces a new feature related to the same problem. If this PR gets approved and merged, I can potentially implement the same fix for 5.3, but SKIP_UNINITIALIZED_VALUES will cause some merge conflicts in the future..

As of v 5.3 symfony ignores uninitialized properties by default in ObjectNormalizer::extractAttributes and PropertyNormalizer::extractAttributes (implemented in #38900 and #34791) but this approach is wrong - extractAttributes method MUST return the same attributes list for any instance of the same class, otherwise cached attributes do not match actual attributes list for different instances having different set of initialized/uninitialized properties

So, this PR does a few things:

  1. removes ignoring attributes from all built-in implementations of \Symfony\Component\Serializer\Normalizer\AbstractObjectNormalizer::extractAttributes so that even uninitialized attributes will be cached by normalizer. Instead, we ignore uninitialized attributes when trying to access them while calling \Symfony\Component\Serializer\Normalizer\AbstractObjectNormalizer::getAttributeValue
  2. sets true as the default value for SKIP_UNINITIALIZED_VALUES context setting as I believe this is the expected default behavior for any built-in object normalizer
  3. makes SKIP_UNINITIALIZED_VALUES compatible with not just ObjectNormalizer, but with two other built-in normalizers PropertyNormalizer and GetSetMethodNormalizer as well

@carsonbot
Copy link

Hey!

I see that this is your first PR. That is great! Welcome!

Symfony has a contribution guide which I suggest you to read.

In short:

  • Always add tests
  • Keep backward compatibility (see https://symfony.com/bc).
  • Bug fixes must be submitted against the lowest maintained branch where they apply (see https://symfony.com/releases)
  • Features and deprecations must be submitted against the 5.4 branch.

Review the GitHub status checks of your pull request and try to solve the reported issues. If some tests are failing, try to see if they are failing because of this change.

When two Symfony core team members approve this change, it will be merged and you will become an official Symfony contributor!
If this PR is merged in a lower version branch, it will be merged up to all maintained branches within a few days.

I am going to sit back now and wait for the reviews.

Cheers!

Carsonbot

Copy link
Member

@dunglas dunglas left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks cleaner to me.

@chalasr
Copy link
Member

chalasr commented Oct 13, 2021

Some tests are broken. Can you have a look?

@ivannemets-sravniru
Copy link
Author

@chalasr It turned out that @requires 7.4 does not work with @dataProvider since data providers are always called without taking into account @requires annotation (I didn't know that) - and this caused ParseError in PHP 7.2 tests..

So I fixed it but the check continuous-integration/appveyor/pr is still failing with error:

Build execution time has reached the maximum allowed time for your plan (20 minutes).

and I'm quite sure it has nothing to do with my PR specifically..

So, I don't know what is the next step here

Copy link
Member

@chalasr chalasr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks.
Don't worry about the failing appveyor build, it's unrelated.

@ivannemets-sravniru
Copy link
Author

@dunglas
I re-requested your review since I made some changes to fix tests in PHP 7.2, could you please take another look?
Thanks!

@fabpot fabpot force-pushed the issue/36594-attributes-cache-breaks-normalization branch from 4adec04 to 55818c3 Compare October 16, 2021 15:18
@fabpot
Copy link
Member

fabpot commented Oct 16, 2021

Thank you @ivannemets-sravniru.

@fabpot fabpot merged commit 732acf5 into symfony:5.4 Oct 16, 2021
nicolas-grekas added a commit that referenced this pull request Nov 28, 2021
…s-grekas)

This PR was merged into the 4.4 branch.

Discussion
----------

[Serializer] fix support for lazy/unset properties

| Q             | A
| ------------- | ---
| Branch?       | 4.4
| Bug fix?      | yes
| New feature?  | no
| Deprecations? | no
| Tickets       | Fix #44273 #44283
| License       | MIT
| Doc PR        | -

This basically backports #43469 into 4.4, which is the way to go to fix #44273.
The code that exists to handle uninitialized properties is broken anyway (it was before the recent changes.)

Commits
-------

db043aa [Serializer] fix support for lazy/unset properties
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Serializer] Normalizer incorrectly caching allowedAttributes when typed properties are used.
6 participants