Skip to content

[RFC] [Translation] Remote Storages #36543

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
welcoMattic opened this issue Apr 23, 2020 · 9 comments · Fixed by #38475
Closed

[RFC] [Translation] Remote Storages #36543

welcoMattic opened this issue Apr 23, 2020 · 9 comments · Fixed by #38475

Comments

@welcoMattic
Copy link
Member

welcoMattic commented Apr 23, 2020

I open this issue in collaboration with @odolbeau, who has been working on php-translation with @Nyholm, @rvanlaak, me and other contributors for a long time.

Since a few weeks we have been thinking about integrating some of php-translation/symfony-bundle features directly into Symfony.

Why adding new features to the Translation component?

Nowadays, the Symfony Translation component is really useful for translation process at runtime. It reads a bunch of file formats, it extracts automatically most of translation keys from your code.

However, some features are still missing. They could actually be useful for modern applications. Among these features there is the storage of translations messages in dedicated SaaS to make translators (people) independent of your application.

Moreover, the Translation Panel's DX in Symfony Profiler can be enhanced a lot.

At php-translation we have been working on those features for a few years. We think that some of those can get their right place into Symfony.

What are the features that should be added, in our opinion?

Remote Storages

For us, the main feature to implement is Remote Storages. As I said, many modern applications use translation SaaS to manage their texts' localization.

Most of the time, those SaaS provide APIs with which developers can interact to push and pull translations.

This is very useful if your application needs to be translated by external translators. You might want to provide them a ready-to-use GUI tool and ease you to export their work once it is done. Furthermore, using these tools could avoid conflicts in translation files during development.

We would like to build Remote Storages system just as Transports in Symfony Messenger, or Adapters in Symfony Notifier. We will write an interface and let the developers write more Remote Storages if they need it.

Configuration

Let's imagine the following configuration:

framework:
    default_locale: en
    translator:
        default_path: '%kernel.project_dir%/translations'
        fallbacks:
            - en
        enabled: true
        logging: false
        formatter: translator.formatter.default
        cache_dir: '%kernel.cache_dir%/translations'
        paths: {  }
        
        ## Starting from here, it's new config
        remote_storages:
                # All translations in all locales are pushed and pulled to Loco
                loco:
                    dsn: loco://URL?params
                # Translations from domains "messages" & "validators" and only for locales "en" and "fr" are also pushed and pulled to transifex
                transifex:
                    dsn: transifex://URL/?params
                    domains: ['messages', 'validators']
                    locales: ['en', 'fr']

Commands

We will create at least 2 commands to let you interact with translation SaaS APIs.

Push command

bin/console translation:push <remote-storage-name> [--force] [--delete-obsolete] [--domains validators] [--locales en]

This command lets you send your local translations to a Remote Storage API.
By default, we send only new translations.
With --force we send all translations, new included. It will overwrite updated existing remote translations.
With --delete-obsolete we first fetch remote translations, we filter those which do not exist anymore in local files, remove those from remote storage(s), then we send all translations, new included.

Pull command

bin/console translation:pull <remote-storage-name> [--force] [--delete-obsolete] [--domains validators] [--locales en]

This command lets you fetch your remote translations from a Remote Storage API.
By default, we fetch only new translations.
With --force we fetch all translations, new included. It will overwrite updated existing your local translations.
With --delete-obsolete simply remove your local files and recreate those from fetched remote storage(s) data. By using this one, you must be sure that your local files are up to date. We will advert you with a warning message.

Both of these commands can filter messages on domains and/or locales.

@stof
Copy link
Member

stof commented Apr 23, 2020

AFAIK, we don't have any acting panel in the profiler right now. And that would require the webserver to have write access to the translation files, which might create permission issues.
And that becomes even more weird if the panel reflects the new translation after you save, as the page would not be displaying the data from the loaded profile anymore. That goes against the way the enter WebProfiler works. I fear it will create confusion.

Regarding remote storages, I understand that they would only be used by the syncing commands. Everything else would still read from local files. Am I right ? If yes, what is the benefit of having these commands in the core compared to having them in your existing bundle ?

@welcoMattic
Copy link
Member Author

In fact, we don't have any active panel for now. However, it's not the main feature and it's not mandatory to use. I understand that it could create confusion, especially on message editing and we can be agree to remove it. But I think that the one-click add of missing translations keys could really helps us, and avoid boring creation of new translation in XLIFF files. About permissions, I think that we could detect permissions on translation files and enable the feature according to the permissions.

About remote storages, you are right. At runtime we are still reading local files. Remote APIs are called only by commands to sync local files. The benefit to having these in the core is, IMHO, the same as Notifier or Mailer: let's use less external bundle for core features. That's exactly our point here, we would like to enhance the Translation component. That was the being reason of php-translation too, assess the need of these features.

May be we could split this RFC, one side with Profile panel, another with Remote Storages?

@stof
Copy link
Member

stof commented Apr 27, 2020

May be we could split this RFC, one side with Profile panel, another with Remote Storages?

I think that's indeed a good idea, as they are separate features.

@javiereguiluz
Copy link
Member

I think that adding support for API-based translation services would be OK (unless it complicates code a lot).

But I wouldn't add editing features to the Symfony Profiler. This would only complicate things and also, profiler is about debugging/inspecting ... so we should keep this separation of responsibilities.

@welcoMattic welcoMattic changed the title [RFC] [Translation] Remote Storages and Profiler panel improvements [RFC] [Translation] Remote Storages May 2, 2020
@welcoMattic
Copy link
Member Author

I just split the RFC for Profiler into #36663

@fabpot
Copy link
Member

fabpot commented May 3, 2020

Adding integration with third-party translators seems like a great idea!

@carsonbot
Copy link

Thank you for this suggestion.
There has not been a lot of activity here for a while. Would you still like to see this feature?

@carsonbot
Copy link

Hello? This issue is about to be closed if nobody replies.

@welcoMattic
Copy link
Member Author

No @carsonbot, we are working on the PR #38475 actively

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
6 participants