Skip to content

[Serializer] Add CompiledClassMetadataFactory #29117

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Aug 11, 2020
Merged

[Serializer] Add CompiledClassMetadataFactory #29117

merged 1 commit into from
Aug 11, 2020

Conversation

fbourigault
Copy link
Contributor

@fbourigault fbourigault commented Nov 7, 2018

Q A
Branch? master
Bug fix? no
New feature? yes
BC breaks? no
Deprecations? no
Tests pass? yes
Fixed tickets ?
License MIT
Doc PR todo (issue: symfony/symfony-docs#10706)

This introduce a dumped ClassMetadataFactoryInterface implementation to speed up the serializer by leveraging PHP7 immutable array.

Like for #28865, if the user have the opcache extension enabled, the compilation time will be skipped. The user will also have a performance boost when not using opcache as we are no longer fetching ClassMetadata from the PSR-6 cache.

This allow to speed up the normalization (without opcache) by 9-12% depending on how many objects are involved in the graph:

On the FrameworkBundle side, I suggest to add a CacheWarmer to dump the metadata array from configured class list. The list could have a good default which will load the classes found in src/Entity.

Dumping the ClassMetadata

$classMetadatas = [];

foreach([Category::class, Comment::class, Forum::class, Thread::class, User::class] as $class) {
    $classMetadatas[] = $this->classMetadataFactory->getMetadataFor($class);
}

file_put_contents('dumped.php', $this->classMetadataFactoryCompiler->compile($classMetadatas));

Using the dumped ClassMetadata

$classMetadataFactory = new CompiledClassMetadataFactory(
    'dumped.php',
    new CacheClassMetadataFactory(
        new ClassMetadataFactory(new AnnotationLoader(new AnnotationReader())),
        new ApcuAdapter('SymfonyMetadata')
    )
);

To do

  • Tests.
  • Cache warmer.
  • Documentation.
  • Changelog entry.

@javiereguiluz
Copy link
Member

@fbourigault impressive work you did here! The Blackfire comparisons are incredible (the second one removes hundreds of thousands of function calls!!).

In order to help the Symfony Docs team, we'd need a few more things here:

  • The examples you showed, it's all you need to use this in the stand-alone component?
  • When using this inside the Symfony Framework, do you need to enable/change something? Some config option? Some code?

Thanks!

Copy link
Member

@stof stof left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Regarding the performance, what is the difference between that and using a cache backed by the adapter relying on OPCache too ?

@fbourigault
Copy link
Contributor Author

@stof here are the benchmark

I used the following code to use the PhpArrayAdapter:

$adapter = new ApcuAdapter('SymfonyMetadata');

$adapter = new PhpArrayAdapter(__DIR__.'/../metadata.cache', $adapter);

$classMetadataFactory = new CacheClassMetadataFactory(
    new ClassMetadataFactory(new AnnotationLoader(new AnnotationReader())),
    $adapter
);

And here is the code for the warmup:

$classMetadataFactory = new ClassMetadataFactory(new AnnotationLoader(new AnnotationReader()));

$classMetadatas = [];

foreach([Category::class, Comment::class, Forum::class, Thread::class, User::class] as $class) {
    $classMetadatas[strtr($class, '\\', '_')] = $classMetadataFactory->getMetadataFor($class);
}

$adapter = new PhpArrayAdapter(__DIR__.'/../../metadata.cache', new NullAdapter());
$adapter->warmUp($classMetadatas);

I also switched from backfire run to blackfire/php-sdk to exclude the warmup from the profile.

@fbourigault
Copy link
Contributor Author

The examples you showed, it's all you need to use this in the stand-alone component?

Yes :)

When using this inside the Symfony Framework, do you need to enable/change something? Some config option? Some code?

This is not yet defined, but it's maybe something to enable. Scanning the whole src/Entity directory could be part of the recipe.

@nicolas-grekas nicolas-grekas added this to the next milestone Nov 8, 2018
@fbourigault
Copy link
Contributor Author

fbourigault commented Nov 8, 2018

I fixed all comments.

Now I want to write the CacheWarmer and the FrameworkBundle integration.

What about having something like:

framework:
    serializer:
        metadata:
            warmup:
                - directory: '%kernel.project_dir%/src/Entity'
                - class: App\Model\MyModelToWarmup
                - directory: '@SomeBundle/Model'

Third party bundles would be able to add entries to this list by prepending configuration.
Then the DI extension will build a Finder and find classes for directory entries.
Then a class list will be built and used as CacheWarmer argument.

I also have a few questions:

  • Where should this CacheWarmer live? The Serializer component or the FrameworkBundle?
  • Do we have some reusable code for scanning directories and finding classes?

@fbourigault
Copy link
Contributor Author

fbourigault commented Nov 13, 2018

Should I add the symfony/var-exporter as a requirement or a suggestion?

EDIT: I added it as suggestion.

@fbourigault
Copy link
Contributor Author

I rebased and fixed CS. I think this PR is ready now!

Copy link
Member

@dunglas dunglas left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you very much for this perf boost!

Copy link
Member

@nicolas-grekas nicolas-grekas left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes, that's it! just one minor comment: what about removing keys?

Copy link
Member

@nicolas-grekas nicolas-grekas left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great!

@nicolas-grekas
Copy link
Member

Is this still relevant now that many things moved on the topic?
If yes, can you please rebase?

@fabpot
Copy link
Member

fabpot commented Aug 11, 2020

Thank you @fbourigault.

@fabpot fabpot merged commit 7c522e2 into symfony:master Aug 11, 2020
@fabpot
Copy link
Member

fabpot commented Aug 11, 2020

@fbourigault Can you have a look at CompiledClassMetadataCacheWarmer::warmUp()? In 5.2, it should return an array of classes to preload in 7.4. I've returned an empty array for now, but we might do something better. Thank you.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants