[Translator] Intl message converter #28486

Nyholm · 2018-09-17T01:43:08Z

Q	A
Branch?	master
Bug fix?	no
New feature?	yes
BC breaks?	no
Deprecations?	no
Tests pass?	yes
Fixed tickets
License	MIT
Doc PR

As discussed in #28375, we need a command to convert from "Symfony 3" plural format to Intl message format.

Before

<trans-unit id="xx0" resname="simple_key">
  <source>simple_key</source>
  <target>Hello %name%</target>
</trans-unit>
<trans-unit id="xx1" resname="plural_key">
  <source>plural_key</source>
  <target>{0} %name%, there are no apples|{1} %name%, there is one apple|]1,Inf] %name%, there are %count% apples</target>
</trans-unit>

{{ 'simple_key|trans({"%name%": "foo"}) }} <br>
{{ 'plural_key|transchoice(2, {"%name%": "foo"}) }}

After

<trans-unit id="xx0" resname="simple_key">
  <source>simple_key</source>
  <target>Hello {name}</target>
</trans-unit>
<trans-unit id="xx1" resname="plural_key">
  <source>plural_key</source>
  <target>{ COUNT, plural,
    =0 {{name}, there are no apples}
    =1 {{name}, there is one apple}
    other {{name}, there are # apples}
}</target>

{{ 'simple_key'|trans({"name": "foo"}) }} <br>
{{ 'plural_key'|trans({"COUNT":2, "name": "foo"}) }}

As you can see, the user needs to modify their twig templates, "%name%" => "name". To use "{%name%}" in the translation message is not valid syntax.

TODO

I need to add more tests of plural messages. Feel free to add a comment with a "Symfony style" plural string and I'll add it to the tests.

We also need a way to modify the the %name% placeholder. Users may use #name# or any other arbitrary string.

BenjaminRbt · 2018-09-21T14:06:39Z

src/Symfony/Component/Translation/Command/IntlConvertCommand.php

+        $kernel = $this->getApplication()->getKernel();
+
+        // Define Root Paths
+        $transPaths = $kernel->getProjectDir().'/translations';


$transPaths = $kernel->getProjectDir().'/translations';
🔽
$transPaths = $kernel->getProjectDir().DIRECTORY_SEPARATOR.'translations';
WDYT ?

BenjaminRbt · 2018-09-21T14:09:45Z

src/Symfony/Component/Translation/Util/IntlMessageConverter.php

+EOF;
+
+        $standardRules = array();
+        foreach ($parts as $part) {


What if $parts is empty ? I would have add a check not to continue if empty($parts) is true

I dont know.. When will parts be empty?

I adding a small fix.

nicolas-grekas · 2018-10-01T19:48:30Z

If #28375 goes well, we might close this one and save us some work. \o/

fabpot · 2018-10-10T09:59:35Z

Why could we close this one? We still need a way to move old messages to the new format, right?

nicolas-grekas · 2018-10-10T10:24:01Z

We don't need one because we don't plan to deprecate the current format. It could be nice to provide the command if ppl want to move to intl, but there is no real reason to do it so I'm not sure it's worth it.

fabpot · 2018-10-10T10:31:26Z

I get that, but I think people will want to migrate to the new format as it is more powerful.

stof · 2018-10-11T13:17:24Z

@nicolas-grekas at least for pluralized message, it is deprecated currently (as APIs consuming it are deprecated)

nicolas-grekas · 2018-10-11T13:32:28Z

@stof not really, see #28375 (comment)

nicolas-grekas

please rebase and move to short arrays.

fabpot · 2019-12-03T06:54:05Z

@Nyholm Do you have any plans to finish this one? I think it would be a very nice and useful addition.

Nyholm · 2019-12-03T07:35:15Z

Yes, I will make sure to complete this PR.

Thank you for the ping

welcoMattic · 2020-07-01T09:27:03Z

This PR could be very useful alongside #37462 to automate intl conversion of messages stored in a translation SaaS.
@Nyholm do you plan to finish it before 5.2 release (which is the release we target for #37462)?

Nyholm · 2020-07-01T10:30:26Z

Yes. I will

fabpot

Quick review (mostly about style) as a nice ping/reminder :)

fabpot · 2020-08-19T06:48:29Z

src/Symfony/Bundle/FrameworkBundle/Resources/config/console.xml

+            <argument type="service" id="translation.writer" />
+            <argument type="service" id="translation.reader" />
+            <tag name="console.command" command="translation:convert-to-intl-messages" />
+        </service>


Should be moved to the new PHP config file.

fabpot · 2020-08-19T06:48:56Z

src/Symfony/Component/Translation/Command/IntlConvertCommand.php

+    protected function configure()
+    {
+        $this
+            ->setDescription('Convert from Symfony 3 plural format to ICU message format.')


I would remove 3 here.

fabpot · 2020-08-19T06:49:25Z

src/Symfony/Component/Translation/Command/IntlConvertCommand.php

+        ;
+    }
+
+    protected function execute(InputInterface $input, OutputInterface $output)


fabpot · 2020-08-19T06:49:44Z

src/Symfony/Component/Translation/Command/IntlConvertCommand.php

+            }
+        }
+
+        $this->writer->write(new MessageCatalogue($locale, $updated), $input->getOption('output-format'), array('path' => $transPaths));


fabpot · 2020-08-19T06:50:03Z

src/Symfony/Component/Translation/Util/IntlMessageConverter.php

+    }
+
+    /**
+     * Get an ICU like array.


fabpot · 2020-08-19T06:50:21Z

src/Symfony/Component/Translation/Util/IntlMessageConverter.php

+namespace Symfony\Component\Translation\Util;
+
+/**
+ * Convert from Symfony 3's plural syntax to Intl message format.


I would remove the reference to 3 here as well.

fabpot · 2020-08-19T06:51:01Z

src/Symfony/Component/Translation/Util/IntlMessageConverter.php

+    public static function convert(string $message, string $variableDelimiter = '%'): string
+    {
+        $array = self::getMessageArray($message);
+        if (empty($array)) {


We try to avoid using empty in Symfony. if (!$array) seems enough here.

fabpot · 2020-08-19T06:51:14Z

src/Symfony/Component/Translation/Util/IntlMessageConverter.php

+            return self::replaceVariables($message, $variableDelimiter);
+        }
+
+        $icu = self::buildIcuString($array, $variableDelimiter);


temp var could be removed

fabpot · 2020-08-19T06:51:42Z

src/Symfony/Component/Translation/Util/IntlMessageConverter.php

+
+    private static function replaceVariables(string $message, string $variableDelimiter): string
+    {
+        $regex = sprintf('|%s(.*?)%s|s', $variableDelimiter, $variableDelimiter);


I would inline the regexp

nicolas-grekas · 2020-09-07T08:55:07Z

I'm closing here because this staled and apparently this is not critical, since this gathered very little feedback from the community. Let's reopen when someone actually needs this (I'd invite that person to adopt the patch here to make the new PR).

carsonbot added Status: Needs Review Translation Feature labels Sep 17, 2018

Adding IntlMessageConverter

6ac56d3

Nyholm force-pushed the icu-converter branch from 47846bb to 6ac56d3 Compare September 17, 2018 01:49

nicolas-grekas added this to the next milestone Sep 17, 2018

Nyholm mentioned this pull request Sep 17, 2018

[Translator] Deprecated transChoice and moved it away from contracts #28375

Merged

BenjaminRbt reviewed Sep 21, 2018

View reviewed changes

Nyholm added 2 commits September 22, 2018 11:25

minor fix

83bfcb6

Make sure "$parts" are never empty

e83e32e

Nyholm changed the title ~~[Translator] WIP: Intl message converter~~ [Translator] WIP: Icu message converter Sep 22, 2018

Nyholm added 3 commits September 22, 2018 12:06

It is called Icu message format. Not Intl

acbf7a0

Allow to specify delimiter

df69607

cs

9071df7

Nyholm changed the title ~~[Translator] WIP: Icu message converter~~ [Translator] Icu message converter Sep 22, 2018

Nyholm changed the title ~~[Translator] Icu message converter~~ [Translator] Intl message converter Sep 22, 2018

Renamed to Intl

6687d9a

nicolas-grekas reviewed Jan 27, 2019

View reviewed changes

javiereguiluz added the ⭐️ EU-FOSSA Hackathon label Apr 2, 2019

nicolas-grekas removed the ⭐️ EUFOSSA Hackathon label Apr 10, 2019

fabpot requested changes Aug 19, 2020

View reviewed changes

carsonbot added Status: Needs Work and removed Status: Needs Review labels Aug 19, 2020

nicolas-grekas closed this Sep 7, 2020

nicolas-grekas modified the milestones: next, 5.2 Oct 5, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Translator] Intl message converter #28486

[Translator] Intl message converter #28486

Nyholm commented Sep 17, 2018 •

edited

Loading

BenjaminRbt Sep 21, 2018

Nyholm Sep 22, 2018

BenjaminRbt Sep 21, 2018

Nyholm Sep 22, 2018

nicolas-grekas commented Oct 1, 2018

fabpot commented Oct 10, 2018

nicolas-grekas commented Oct 10, 2018

fabpot commented Oct 10, 2018

stof commented Oct 11, 2018

nicolas-grekas commented Oct 11, 2018

nicolas-grekas left a comment

fabpot commented Dec 3, 2019

Nyholm commented Dec 3, 2019

welcoMattic commented Jul 1, 2020

Nyholm commented Jul 1, 2020

fabpot left a comment

fabpot Aug 19, 2020

fabpot Aug 19, 2020

fabpot Aug 19, 2020

fabpot Aug 19, 2020

fabpot Aug 19, 2020

fabpot Aug 19, 2020

fabpot Aug 19, 2020

fabpot Aug 19, 2020

fabpot Aug 19, 2020

nicolas-grekas commented Sep 7, 2020

[Translator] Intl message converter #28486

[Translator] Intl message converter #28486

Conversation

Nyholm commented Sep 17, 2018 • edited Loading

Before

After

TODO

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

nicolas-grekas commented Oct 1, 2018

fabpot commented Oct 10, 2018

nicolas-grekas commented Oct 10, 2018

fabpot commented Oct 10, 2018

stof commented Oct 11, 2018

nicolas-grekas commented Oct 11, 2018

nicolas-grekas left a comment

Choose a reason for hiding this comment

fabpot commented Dec 3, 2019

Nyholm commented Dec 3, 2019

welcoMattic commented Jul 1, 2020

Nyholm commented Jul 1, 2020

fabpot left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

nicolas-grekas commented Sep 7, 2020

Nyholm commented Sep 17, 2018 •

edited

Loading