Skip to content

[Polyfill] A new component for portability across PHP versions and extensions #16240

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 1 commit into from

Conversation

nicolas-grekas
Copy link
Member

Q A
Bug fix? no
New feature? yes
BC breaks? no
Deprecations? yes
Tests pass? yes
Fixed tickets -
License MIT
Doc PR -

I know I come a bit late in the 2.8 releasing process, but I really hope we could merge this new component into 2.8, especially since merging it in 3.1 would remove a great deal of its value since the lowest PHP version there is 5.5, whereas this code has shims for pre-5.5 features.

So here we are:

  • polyfills for the mbstring
  • polyfills for the Normalizer class and grapheme_* functions
  • polyfills for some constants and functions introduced in 5.3, 5.4, 5.5, etc. borrowed from https://github.com/nicolas-grekas/Patchwork/tree/master/core/compat (yet an other code of mine)
  • polyfills for hash_equals, hash_pbkdf2 and ldap_escape moved out of existing private methods
  • utility class Binary to be used when mbstring.func_overload is enabled

This allows simplifying the full code base by removing many defined and function_exists checks.

For reference, here is the lists of e.g. functions introduced in 5.4:
http://www.php.net/migration54.functions.php

define('JSON_BIGINT_AS_STRING', 2);
define('JSON_UNESCAPED_SLASHES', 64);
define('JSON_PRETTY_PRINT', 128);
define('JSON_UNESCAPED_UNICODE', 256);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

-1 for this. This breaks libraries detecting whether the feature is detected (old versions of Symfony were defining the constant at the beginning of the file, and it was changed precisely for this reason).

constants should be defined only if their behavior is also polyfilled.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the same is true for the JsonSerializable polyfill btw

@stof
Copy link
Member

stof commented Oct 14, 2015

I don't think creating a component with many polyfills in it is a good idea. Someone wanting to have a random_bytes should not have to depend on the whole UTF-8 polyfills (especially given that such polyfills cannot be loaded on demand as they are for functions and constants, not for autoloadable classes)

So my vote is a -1 here.

If the random_bytes polyfill is buggy on Windows, send a PR to the existing library to improve the Windows support, and it will benefit everyone.

@nicolas-grekas
Copy link
Member Author

This needs some explanations: polyfill are autoloaded, they don't pollute anyone’s app.
This is possible thanks to the static classes proxies for polyfills: the bootstrap file is all that is required, with very minimal code to parse/load. Loading the implementation happens only when a polyfill is actually used.

I do think that one single component for all polyfills is a greater idea: polyfills have a finite upper limit (100% feature backporting). By collecting all efforts in one code base, we can concentrate efforts for everyone, instead of scattering one polyfill here, an other there, with varying implementation quality.

For random_bytes, my implementation is radically different so patching would mean removing most of the code. I'm not that rude.

Please reconsider your vote, or at least allow some discussion.

@@ -1147,7 +1147,7 @@ public static function closeOutputBuffers($targetLevel, $flush)
{
$status = ob_get_status(true);
$level = count($status);
$flags = defined('PHP_OUTPUT_HANDLER_REMOVABLE') ? PHP_OUTPUT_HANDLER_REMOVABLE | ($flush ? PHP_OUTPUT_HANDLER_FLUSHABLE : PHP_OUTPUT_HANDLER_CLEANABLE) : -1;
$flags = PHP_OUTPUT_HANDLER_REMOVABLE | ($flush ? PHP_OUTPUT_HANDLER_FLUSHABLE : PHP_OUTPUT_HANDLER_CLEANABLE);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is actually changing the behavior on PHP 5.3

@nicolas-grekas nicolas-grekas force-pushed the polyfill branch 2 times, most recently from f7d7ad0 to b0e3c21 Compare October 14, 2015 16:04
@nicolas-grekas
Copy link
Member Author

I removed constants definitions that did not match any shim implementation. I also added function_exists checks to the bootstrap file to enhance compat with existing shims.

if (!ini_get('session.entropy_file') && !ini_get('session.entropy_length')) {
ini_set('session.entropy_length', 32);

if (false !== @file_get_contents('/dev/urandom', false, null, -1, 1) ) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

windows here too

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

in fact I'm going to remove these ini_setting: it's reducing the entropy for auto conf, not a good

@mickaelandrieu
Copy link
Contributor

@nicolas-grekas why do we need this in Symfony 3 ? the requirements are php > 5.5

function mb_http_input($type = '') {return p\Mbstring::mb_http_input($type);}
}

if (!function_exists('utf8_encode')) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

when is it not defined ?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this function is provided by the xml extension, which is not always loaded

@stof
Copy link
Member

stof commented Oct 14, 2015

@nicolas-grekas discussion is allowed (this is even required to make a decider reconsider their vote btw).

@@ -0,0 +1,13 @@
<?php

interface Throwable
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Defining this interface while Exception does not implement it does not seem like a good idea to me

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fine by me, I'm removing the PHP7 exceptions related shims

@mickaelandrieu
Copy link
Contributor

Humm this component is a good idea for Symfony 2.

My only problem is that all components concerned have a new dependency on it, and this component will be - in most cases - useless because PHP 5.5 will become the standard for Symfony 3.

So, if the pull request is only for Symfony 2.3+ && < 3.0 it's 👍 else 👎

/**
* @covers Symfony\Component\Polyfill\Iconv::iconv_strlen
* @covers Symfony\Component\Polyfill\Iconv::strlen1
* @covers Symfony\Component\Polyfill\Iconv::strlen2
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we don't use @covers in Symfony

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes I know but this PR imports a code that existed previously, and removing the @covers annotations would only reduce the quality of the code (when running coverage tests). Which means I'd like we keep them.

@nicolas-grekas
Copy link
Member Author

@mickaelandrieu yes I'd like this component to land in 3.0. But of course, all shims for pre-5.5 features should be removed when merging.

@nicolas-grekas nicolas-grekas force-pushed the polyfill branch 2 times, most recently from 7768dfe to 5738021 Compare October 14, 2015 16:55
@mickaelandrieu
Copy link
Contributor

@nicolas-grekas ok then 👍 btw great job as always :)

@csarrazi
Copy link
Contributor

@nicolas-grekas Nice one. However, should this PR limit itself to functions, or also define constants if they are not available?

I'm talking about constants like LDAP_ESCAPE_DN or LDAP_ESCAPE_FILTER, for example.

@nicolas-grekas
Copy link
Member Author

@stof comments addressed, tests should be green on travis and appveyor
@csarrazi these constants are required since they are provided as arguments to the ldap_escape() shim.

@derrabus
Copy link
Member

@rparsi 👏

@nicolas-grekas
Copy link
Member Author

@rparsi thanks for your input but you misread the previous comments and the definition:

  • some raised compat concern, but none is proven, at least nobody refuted my arguments saying there is no compat issue.
  • the definition of a component does not prevent components being dependent on each others, which is of course already the case for many components. There is no difference.

...

@rparsi
Copy link

rparsi commented Oct 21, 2015

@nicolas-grekas @derrabus
That's why I put the disclaimer first :)

I think I can summarize it better:
The added dependency between Polyfill and other components itself is not the root concern, it's that the scope/depth of the dependency is considerable or extreme compared to the current Symfony framework/ecosystem.

So I think the real vote should be on whether or not to do something as @stof suggests.

By the way derrabus, I can't tell if your clap is sarcastic or earnest. :)

@mickaelandrieu
Copy link
Contributor

the definition of a component does not prevent components being dependent on each others, which is of course already the case for many components. There is no difference.

this argument sounds bad to me: this is not because we don't respect the promises we have done before that we have to continue this way.
Since 2.3, more and more components are coupled to anothers - and of course - this is bad for reuse. Symfony project is too much "Symfony Standard Edition" centric. I stopped fighting against it, doesn't means - like a lot of developers - that I think this is the good way to go.

@rparsi
Copy link

rparsi commented Oct 21, 2015

@mickaelandrieu Thanks, you captured my sentiments exactly.

compatibility PHP implementations for some extensions and functions. You should
use it when portability across PHP versions and extensions is desired.

Polyfills are provided are for:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Polyfills are provided for:

@derrabus
Copy link
Member

@rparsi earnest

@ircmaxell
Copy link
Contributor

One question: What does this have to do with Symfony at all? Why isn't this a generic PHP package (why does it have to be a Symfony component, or shipped by Symfony)?

@fabpot
Copy link
Member

fabpot commented Oct 21, 2015

@ircmaxell I'm not sure I understand your concern. Symfony2 has been a set of components from day one, so why not having one on polyfills? We already have similar components like the Intl one. Moreover, Symfony is not just this monolith repository as each component has its own separate repository (which is automatically synced in real-time with the main Symfony one) and Composer package. So, in that sense, that's no different from any other "generic PHP package" (not sure I understand what a generic PHP package is to be honest though). And of course, being under the Symfony umbrella brings a lot of "benefits" like LTS releases, the BC promise, documentation, 2 releases a year, and much more. Last, but not the least, it just happens that we need that "abstraction" in Symfony, that much of the code was already scattered in the different Symfony components (sometimes duplicated) and Nicolas, who is working on this topic is a member of the Symfony core team. Seems very natural to me that we are doing that component in the Symfony project itself and not elsewhere. It looks like I'm missing something here. Or does it has a negative connotation when something is part of Symfony? Is it a bad thing?

@paragonie-scott
Copy link

I think @ircmaxell was trying to suggest making one canonical framework-agnostic polyfill for new PHP features rather than having it be something that Symfony provides and maintains for Symfony exclusively, because if every framework does this, we end up with needless code duplication and bugs fixed in one place might not percolate to the rest of the community.

I don't think he meant to imply any negative connotation just because it's a part of Symfony.

@fabpot
Copy link
Member

fabpot commented Oct 21, 2015

@paragonie-scott I did understand that. But why the Symfony one couldn't be the canonical one? It would be an standalone Github repository with its own Packagist entry (no other Symfony deps). So, the only reason I can think of is the "Symfony" name. I thought that the PHP community stopped framework wars years ago and that nowadays people were more open to use components/libraries coming from many different frameworks. Or am I wrong?

@paragonie-scott
Copy link

But why the Symfony one couldn't be the canonical one?

I don't see why it couldn't.

I started an initiative to make a polyfill a while ago, but it's nowhere near as comprehensive. If Symfony\Polyfill is accepted by the community, I have no problem retiring mine in favor of this one.

It would be an standalone Github repository with its own Packagist entry (no other Symfony deps).

That's a good point in favor of this one.

I thought that the PHP community stopped framework wars years ago and that nowadays people were more open to use components/libraries coming from many different frameworks. Or am I wrong?

Well, this might be off-topic, but I still see pointless arguments about which framework is better almost every day, on Reddit, LinkedIn, Facebook, various forums, and IRC. Very rarely among professionals. It's hard to say if they're really over or not.

@Tobion
Copy link
Contributor

Tobion commented Oct 21, 2015

In general I think the Polyfill component is a good thing. It bundles all those fallback implementations in one place instead of having it spread across different components and each having to deal with workarounds.

I can think of one major argument why should live outside of symfony/symfony though. We increase minimum PHP requirements in symfony with major versions. But for the Polyfill it would not make sense to have the same release timeline and minimum PHP requirement as all the other symfony components. The Polyfill will probably have to provide the Polyfill for older PHP versions longer than symfony components because that is one of the major points of Polyfill. So it makes no sense to tie Polyfill and Components together and increase minimum PHP requirements at the same time. Otherwise it would mean we have to create a version 3 of Polyfill right now that removes all Polyfills for PHP < 5.5 which makes no sense already...

@Tobion
Copy link
Contributor

Tobion commented Oct 21, 2015

The release timeline was also one argument why we extracted several bundles/components from symfony/symfony in the past. And for Polyfill this is even more relevant IMO.

@nicolas-grekas
Copy link
Member Author

The 2.8 polyfill is for 5.3.9 minimum, and it will be supported in a LTS release cycle. 5.3 and 5.4 won't get any new feature. Which means the polyfill is feature full. Closed. Over. It won't receive anything more but bug fixes if any. So there is no issue at all. Target achieved, package published, job done, we can all move on and let the legacy there.
So I really don't see your point...

@ircmaxell
Copy link
Contributor

And of course, being under the Symfony umbrella brings a lot of "benefits" like LTS releases, the BC promise, documentation, 2 releases a year, and much more.

None of which benefit a polyfill. Polyfill versioning needs to be tied to the release cycle of the thing being polyfilled (as 7.1 is coming out, the polyfill would need to be updated for new patches), as well as for new extensions (which come at sporatic times). It also means that people using the LTS version of the framework (and hence the polyfill) will be coupled to the old version and not get any of the new functionality released since then...

As far as documentation, if it differs from php.net, it's not a polyfill (or that difference is a bug).

Last, but not the least, it just happens that we need that "abstraction" in Symfony

The difference though that I see is that every other component in Symfony is a relatively opinionated component. It's not aiming to be a canonical implementation, but instead an option. This is different and doesn't fit that paradigm. What we're talking about here is an unopinionated implementation of something that's directly tied to PHP itself.

It's not an abstraction at all. It's a polyfill. If you wanted to maintain an abstraction that included polyfill behavior, then I'd agree 100%. But that's not what's being discussed here. We're discussing a direct polyfill.

Ultimately do what you want, I'm not going to point fingers or anything. I'm just pointing out that this is unlike any of the other Symfony components, and by definition has different requirements (as I point out above in the versioning issue). It just feels "odd" to me. Call me crazy (I know, many do), but I'm just trying to point out that this is unlike anything else that you call a component (and calling it a component actually may work against you).

@rparsi
Copy link

rparsi commented Oct 21, 2015

@nicolas-grekas @fabpot From your comments you guys really want to merge this. I have a request that if implemented should help others detect/debug conflicts with other components (custom or otherwise).

Nicolas how much work would it be to add a console command that outputs the following information:

  • your environments php version
  • the list of functions that the polyfill thinks it should implement for your application (ie if you triggered a controller action), and for each one if it already exists before the Polyfill would implement it

@ircmaxell Thanks you explain it a lot better then I ever could. This Polyfill works like a component but as it's tied to PHP itself feels out of place as a Symfony component. Not sure if I'm making sense here so my apologies.

@Tobion
Copy link
Contributor

Tobion commented Oct 21, 2015

-1 to include it in symfony/symfony
+0 to create a separate repo like symfony/polyfill that bundles all polyfills
+1 to create a vendor/organisation just for polyfills where each polyfilled extension has it's own repo as stof suggested

@rparsi
Copy link

rparsi commented Oct 21, 2015

I know my vote doesn't count, so this just for myself :)
👍 to @Tobion @stof

@nicolas-grekas
Copy link
Member Author

Sleeping helping, I now understand what @Tobion says about minimum PHP versions and why that doesn't fit for a polyfill component.

@fabpot
Copy link
Member

fabpot commented Oct 22, 2015

Thank for the constructive discussion everyone, I really appreciate it. We heard your arguments, and we've decided to create a new repo for polyfills, symfony/polyfill which is going to have a different version numbering than Symfony itself and a different release cycle.

Go over at https://github.com/symfony/polyfill for the new implementation which should be ready soon.

fabpot added a commit that referenced this pull request Oct 28, 2015
This PR was merged into the 2.8 branch.

Discussion
----------

Rely on iconv and symfony/polyfill-*

| Q             | A
| ------------- | ---
| Bug fix?      | no
| New feature?  | no
| BC breaks?    | no
| Deprecations? | yes
| Tests pass?   | yes
| Fixed tickets | #16240
| License       | MIT
| Doc PR        | -

Status: needs work
symfony/polyfill-* packages are not ready yet. Still, review welcomed.

Commits
-------

303f05b Rely on iconv and symfony/polyfill-*
@edlington
Copy link

I don't see how polyfills result in loosely coupled code. What I'm seeing is that code rewritten to remove function_exists checks for backwards compatibility now will fail unless polyfill components are used. Above it says polyfills are autoloaded. They don't seem to be autoloading in my install. One definition of clean code is to minimize the number of conditions and loops in a function. That's great. Creating new components moves complexity from the file level to the directory level. Cleaner code, more confusing directory structure, more difficult to track down errors. For Symfony wizards, no problem. Solutions are always political and in this case I think the polyfills make the framework less friendly for those of us who are learners rather than experts.

@derrabus
Copy link
Member

@edlington If the polyfills don't work for you as expected, please open an issue explaining your problem. This gives others a chance to help you.

@stof
Copy link
Member

stof commented Dec 14, 2015

@edlington are you using the Composer autoloader, or are you using your own autoloading system ?

@edlington
Copy link

@stof thank you for that question. I think I see the problem is that I need to add an autoload field to my composer file.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.