[ci] Testing with UTC hides bugs #20551
Conversation
|
Hm, I would have expected more failures (at least the one being fixed in #20550). |
|
Wait until it's merged up to master :) |
05c5880 to
9132b34
Compare
|
Oh indeed, this feature was not present in in 2.7. :D |
|
Looks like a good idea! |
53b517e to
142cede
Compare
| $this->getDateTime(0, $formatter->getTimeZoneId())->format('M j, Y, g:i A'), | ||
| $formatter->format(0) | ||
| ); | ||
| //$this->assertEquals('Jan 1, 1970, 12:00 AM', $formatter->format(0)); |
There was a problem hiding this comment.
I need to understand why this fails when uncommented
142cede to
4d6a22f
Compare
d5d4f34 to
e0f9bda
Compare
This PR was merged into the 2.7 branch. Discussion ---------- [ci] Testing with UTC hides bugs | Q | A | ------------- | --- | Branch? | 2.7 | Bug fix? | no | New feature? | no | BC breaks? | no | Deprecations? | no | Tests pass? | yes | Fixed tickets | - | License | MIT | Doc PR | - As shown by #20550, we're having bugs because we don't in proper TZ conditions. Let's do Europe on Travis, and America on Windows. Commits ------- e0f9bda [ci] Testing with UTC hides bugs
|
This one breaks master for now because a test in the serializer component is set to default to UTC, while the code defaults to date_default_timezone_get(). ping @dunglas , who is right? the tests or the code ? Please open the PR. |
|
Fixed the tests for now by assuming the code is correct. |
| @@ -45,6 +45,8 @@ public function testFormatWithTimezoneFromEnvironmentVariable() | |||
|
|
|||
| protected function getDateFormatter($locale, $datetype, $timetype, $timezone = null, $calendar = IntlDateFormatter::GREGORIAN, $pattern = null) | |||
| { | |||
| IntlTestHelper::requireFullIntl($this, '55.1'); | |||
There was a problem hiding this comment.
Why is this hard coded to 55.1? Symfony is currently bundled with 57.1, but this change makes that tests can only be run against 55.1.
There was a problem hiding this comment.
What's the goal of this change?
Currently tests need to be run against the specific version of ICU (the one that matches the stub version). Some tests will fail with ICU 55.1 if stubs are at 57.1 due to differences introduced between versions. Example:
1) Symfony\Component\Intl\Tests\NumberFormatter\NumberFormatterTest::testFormatCurrencyWithCurrencyStyleCostaRicanColonsRounding with data set #0 (100, 'CRC',
'CRC', '%s100.00', 0, SplObjectStorage Object ())
Failed asserting that two strings are equal.
--- Expected
+++ Actual
@@ @@
-'CRC100.00'
+'CRC100'
There was a problem hiding this comment.
I missed replacing the comparison operator from != to >=.
The intend is to make tests run more frequently instead of being skipped, especially on my Ubuntu LTS that ships with 55.1, and where tests run fine.
Having that many tests be skipped almost all the time and especially on our CI or even locally for most users means they fail to achieve their goal of preventing regressions.
This is an attempt to make the situation better.
Would be great if you could look into it in your PR also.
There was a problem hiding this comment.
The intend is to make tests run more frequently instead of being skipped, especially on my Ubuntu LTS that ships with 55.1, and where tests run fine.
Ok. The problem is that these tests are currently very coupled to a specific ICU version, as data between versions changes quite a lot.
What I will try to do is to find examples that didn't change between recent ICU versions and update test cases. Otherwise we'll need to rollback your change and think of another way of running these tests.
This PR was squashed before being merged into the 2.7 branch (closes #21320). Discussion ---------- Fix ICU dependant tests | Q | A | ------------- | --- | Branch? | 2.7 | Bug fix? | no | New feature? | no | BC breaks? | no | Deprecations? | no | Tests pass? | yes | Fixed tickets | - | License | MIT | Doc PR | - #20551 changed the condition checking if the ICU version matches the stubbed data. The change intended to enable tests on more ICU versions, but it actually has limited them. I'm still not convinced it should've been done but let's at least fix the condition. Ideal solution would be to have the latest ICU data available on travis (still not there travis-ci/travis-ci#3616). I also needed to fix several tests. Currently skipped tests in components depending on ICU data: | | 4.8.1.1 | 54.1 | 55.1 | 57.1 | |---|---|--|--|--| | Intl | 488 | 488 | 7 | 6 | | Locale | 0 | 0 | 0 | 0 | | Translation| 0 | 0 | 0 | 0 | | Validator | 69 | 69 | 69 | 0 | | Form | 75 | 75 | 75 | 1 | Commits ------- d3b5d8e Fix tests with ICU 57.1 677d820 Fix the condition checking the minimum ICU version
As shown by #20550, we're having bugs because we don't in proper TZ conditions.
Let's do Europe on Travis, and America on Windows.