Skip to content

[HttpKernel] Fix mem usage when stripping the prod container #18048

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Mar 9, 2016

Conversation

nicolas-grekas
Copy link
Member

Q A
Branch 2.3
Bug fix? yes
New feature? no
BC breaks? no
Deprecations? no
Tests pass? yes
Fixed tickets #18007
License MIT
Doc PR -

I propose to just replace doc comments by regular comments, so that the parser removes them and opcache doesn't have to keep them in memory, which is the target.

@dunglas
Copy link
Member

dunglas commented Mar 7, 2016

👍

@peteward
Copy link

peteward commented Mar 8, 2016

@nicolas-grekas, I ran some tests on my container after patching these changes into 2.8.

In each case I did a hard clear of cache, followed by an apache restart as opcache memory usage wasn't being reported consistently without it.

Dev

cache/dev/appDevDebugProjectContainer.php = 6,500,226 B
opcache memory consumption = 12.29 MB

Original - token_get_all

cache/prod/appProdProjectContainer.php = 5,435,490 B
opcache memory consumption = 11.12 MB
    Command being timed: "php app/console cache:clear --env=prod --no-optional-warmers"
    User time (seconds): 11.42
    System time (seconds): 4.75
    Percent of CPU this job got: 67%
    Elapsed (wall clock) time (h:mm:ss or m:ss): 0:23.84
    Average shared text size (kbytes): 0
    Average unshared data size (kbytes): 0
    Average stack size (kbytes): 0
    Average total size (kbytes): 0
    Maximum resident set size (kbytes): 425032

Mine - php_strip_whitespace

cache/prod/appProdProjectContainer.php = 4,827,701 B
opcache memory consumption = 11.12 MB
    Command being timed: "php app/console cache:clear --env=prod --no-optional-warmers"
    User time (seconds): 10.47
    System time (seconds): 4.49
    Percent of CPU this job got: 63%
    Elapsed (wall clock) time (h:mm:ss or m:ss): 0:23.49
    Average shared text size (kbytes): 0
    Average unshared data size (kbytes): 0
    Average stack size (kbytes): 0
    Average total size (kbytes): 0
    Maximum resident set size (kbytes): 162140

Proposed - single star opening comments

cache/prod/appProdProjectContainer.php = 6,486,201 B
opcache memory consumption = 11.52 MB
    Command being timed: "php app/console cache:clear --env=prod --no-optional-warmers"
    User time (seconds): 9.64
    System time (seconds): 4.96
    Percent of CPU this job got: 60%
    Elapsed (wall clock) time (h:mm:ss or m:ss): 0:24.33
    Average shared text size (kbytes): 0
    Average unshared data size (kbytes): 0
    Average stack size (kbytes): 0
    Average total size (kbytes): 0
    Maximum resident set size (kbytes): 166896

Summary

  • compilation time not an issue
  • raw stripped container file bigger (but then we knew it was going to be).
  • container compiling memory savings in-line with other methods so works 👍
  • slightly higher memory consumption in opcache - 11.12 MB vs 11.52 MB.

For this last point when I investigated the Container I saw various extra classes and methods with double-commented sections. It's possible that this could have been my patching of the changes but I found it much simpler to just do the following instead of your {$this->docStar} approach at the end of the dump() method:

        // Replace double star comments with single for non-debug container to strip for opcache
        return $options['debug'] ? $code : str_replace('/**', '/*', $code);

With this version I had no other negative impact, the container comment blocks were all /* and the opcache memory consumption was consistent at 11.12MB

In addition to these changes stripComments can I guess be removed along with testStripComments.

Do you want to update your PR or would you prefer me to make one?

@nicolas-grekas
Copy link
Member Author

Please make a new one after cherry-picking this one :)
I'm going to find where these additional doc comments come from

@nicolas-grekas
Copy link
Member Author

@peteward the delta you get is because this patch is for 2.3 and you apply it on 2.8, where a few more stars have to be removed.
I think we should now merge this one and merge it up to 2.8.

@peteward
Copy link

peteward commented Mar 8, 2016

See other PR - it's ProxyManager.

I created my PR from a new branch from 2.3.
My tests above were run on 2.8 but I'm pretty sure it's the same.

@nicolas-grekas
Copy link
Member Author

@peteward I added an str_replace on proxy-generated code. I now need to confirm this code can't contain arbitrary string values.

@nicolas-grekas
Copy link
Member Author

@Ocramius quick help needed plz: is it possible that a generated proxy class contains an arbitrary string, or are they all generated from well know templates so that the /** substring can't exist in generated proxies except for opening doc comments?

@peteward
Copy link

peteward commented Mar 8, 2016

The output looks to be formulaic from my container here...

@Ocramius
Copy link
Contributor

Ocramius commented Mar 8, 2016

I can't guarantee what you just asked for. str_replace()-based code manipulation is suicide. The ancients gave us AST: use it.

@nicolas-grekas
Copy link
Member Author

@Ocramius OK thanks,
then let's use the existing Kernel::stripComments for proxies.
PR updated.

@peteward
Copy link

peteward commented Mar 8, 2016

I was reluctant at reintroducing stripComments but running token_get_all in a loop on the individual generated ProxyFiles means the memory impact is almost negligible - there's no noticeable increase at all in total memory use on compiling my container and the opcache memory consumption is the same.

Looks like a complete fix, great work 👍

@nicolas-grekas
Copy link
Member Author

Cool, thanks checking!

@fabpot
Copy link
Member

fabpot commented Mar 9, 2016

👍

@fabpot
Copy link
Member

fabpot commented Mar 9, 2016

Thank you @nicolas-grekas.

@fabpot fabpot merged commit 4fa5844 into symfony:2.3 Mar 9, 2016
fabpot added a commit that referenced this pull request Mar 9, 2016
…er (nicolas-grekas)

This PR was merged into the 2.3 branch.

Discussion
----------

[HttpKernel] Fix mem usage when stripping the prod container

| Q             | A
| ------------- | ---
| Branch        | 2.3
| Bug fix?      | yes
| New feature?  | no
| BC breaks?    | no
| Deprecations? | no
| Tests pass?   | yes
| Fixed tickets | #18007
| License       | MIT
| Doc PR        | -

I propose to just replace doc comments by regular comments, so that the parser removes them and opcache doesn't have to keep them in memory, which is the target.

Commits
-------

4fa5844 [HttpKernel] Fix mem usage when stripping the prod container
@nicolas-grekas nicolas-grekas deleted the strip-by-part branch March 9, 2016 14:32
@fabpot fabpot mentioned this pull request Mar 13, 2016
fabpot added a commit that referenced this pull request Oct 4, 2017
This PR was merged into the 3.4 branch.

Discussion
----------

[DI] remove inheritdoc from dumped container

| Q             | A
| ------------- | ---
| Branch?       | 3.4
| Bug fix?      | no
| New feature?  | no <!-- don't forget to update src/**/CHANGELOG.md files -->
| BC breaks?    | no
| Deprecations? | no <!-- don't forget to update UPGRADE-*.md files -->
| Tests pass?   | yes
| Fixed tickets |
| License       | MIT
| Doc PR        |

The inheritdoc without adding text is pointless on it's own as any decent IDE will show that those methods are overwriting the base method. This removes it from the dumped container to remove memory and generation time. This goes in the same direction as #23673, #18048 and #24342
It does not affect phpdocs of the compiled container that might actually be useful (factory methods of services explaining if the service is private, shared etc.).

Commits
-------

d779845 [DI] remove inheritdoc from dumped container
fabpot added a commit that referenced this pull request Sep 24, 2023
This PR was squashed before being merged into the 6.4 branch.

Discussion
----------

Deprecate `Kernel::stripComments()`

| Q             | A
| ------------- | ---
| Branch?       | 6.4
| Bug fix?      | no
| New feature?  | no <!-- please update src/**/CHANGELOG.md files -->
| Deprecations? | no <!-- please update UPGRADE-*.md and src/**/CHANGELOG.md files -->
| Tickets       | Fix #... <!-- prefix each issue number with "Fix #", no need to create an issue if none exists, explain below instead -->
| License       | MIT
| Doc PR        | symfony/symfony-docs#... <!-- required for new features -->

This PR replace `method_exists` by `class_exists` in order to check if component is available. It harmonize codebase

~This PR remove checking if method `Kernel::setIgnore` exists.~

~This method was introduced in symfony 2.3 (#18048), so now it's always available~

Commits
-------

43c5038 Deprecate `Kernel::stripComments()`
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants