@fabpot seeing that you wrote this component: Any idea from your side about this issue?

@symfony/mergers anyone able to help with this one?

@dmaicher Python supports this too via the SMTPUTF8 extension.

Go supports it too:

// String formats the address as a valid RFC 5322 address.
// If the address's name contains non-ASCII characters
// the name will be rendered according to RFC 2047.

And that's the RFC that describes this: https://www.ietf.org/rfc/rfc2047.txt

@javiereguiluz as far as I understand rfc 2047 does not apply for email addresses?

+ An 'encoded-word' MUST NOT appear in any portion of an 'addr-spec'.

It would only apply for the "name" part of some recipient address header. But not the address itself.

@dmaicher I read too fast then. Sorry 😞

@javiereguiluz any idea how to move this forward? This issue keeps me from using the symfony/mailer unfortunately 😢

I think we can do this change, but only if we also forbid non-ASCII chars in the local part of the envelope sender. The reason is that there always must be a path to receive bounces, and the envelope-sender is their destination.

@nicolas-grekas first of all thank you for taking to time to look into this! 😊

I think we can do this change, but only if we also forbid non-ASCII chars in the local part of the envelope sender. The reason is that there always must be a path to receive bounces, and the envelope-sender is their destination.

Not sure I follow this comment. So currently this change should impact the envelope sender as well, right?

Seeing that the envelope sender is also an Address and thus will use the encoder internally: https://github.com/symfony/symfony/blob/master/src/Symfony/Component/Mime/Address.php#L72

So you mean we should change this behavior for the envelope sender?

We shouldn't necessarily change the behavior of going through the encoder, but we should add a check, in Mailer, to reject any local part with non-ASCII characters. This will ensure that any bounce message will be able to come back to the sender independently of whether SMTP hops are able to deal with UTF-8 or not. This looks like a critical property to me.

We shouldn't necessarily change the behavior of going through the encoder, but we should add a check, in Mailer, to reject any local part with non-ASCII characters. This will ensure that any bounce message will be able to come back to the sender independently of whether SMTP hops are able to deal with UTF-8 or not. This looks like a critical property to me.

Ok fair enough. That seems good to me 👍

How should we do this? Inside Mailer or inside Envelope? Should it throw Symfony\Component\Mime\Exception\AddressEncoderException?

Inside Envelope? this would allow reusing AddressEncoderException isn't it?

This is a new feature, so it must be rebased on master before merging.

@fabpot would have been great to see this change in 4.4 LTS... but ok. Why do you think this cannot be treated as a bugfix?

Anyway I rebased it 😉

Thank you @dmaicher.

It looks like this broke some tests for bridges, could you please have a look @dmaicher?

@nicolas-grekas ah sorry I missed those test fails.

@fabpot I could simply fix those test fails but not sure the change in behavior is really correct 😕

For example the test fail in MandrillApiTransportTest:

        $transport = new MandrillApiTransport('KEY', $client);

        $mail = new Email();
        $mail->subject('Hello!')
            ->to(new Address('saif.gmati@symfony.com', 'Saif Eddin'))
            ->from(new Address('fabpot@symfony.com', 'Fabien'))
            ->text('Hello There!');

        $message = $transport->send($mail);

Now with my changes the name "Fabien" is dropped from the sender address and before it was passed on to Mandrill as from_name.

EDIT: or maybe the mandrill transport (and others) should not use the sender as "from"? As per RFC the "Sender" can only be an email address without a name as @fabpot mentioned here: #36178 (comment)

EDIT: So for example

diff --git a/src/Symfony/Component/Mailer/Bridge/Mailchimp/Transport/MandrillApiTransport.php b/src/Symfony/Component/Mailer/Bridge/Mailchimp/Transport/MandrillApiTransport.php
index 50be4e02cb..027c67cd10 100644
--- a/src/Symfony/Component/Mailer/Bridge/Mailchimp/Transport/MandrillApiTransport.php
+++ b/src/Symfony/Component/Mailer/Bridge/Mailchimp/Transport/MandrillApiTransport.php
@@ -72,19 +72,25 @@ class MandrillApiTransport extends AbstractApiTransport
 
     private function getPayload(Email $email, Envelope $envelope): array
     {
+        $fromAddresses = $email->getFrom();
+        $from = reset($fromAddresses);
+
         $payload = [
             'key' => $this->key,
             'message' => [
                 'html' => $email->getHtmlBody(),
                 'text' => $email->getTextBody(),
                 'subject' => $email->getSubject(),
-                'from_email' => $envelope->getSender()->getAddress(),
                 'to' => $this->getRecipients($email, $envelope),
             ],
         ];
 
-        if ('' !== $envelope->getSender()->getName()) {
-            $payload['message']['from_name'] = $envelope->getSender()->getName();
+        if ($from) {
+            $payload['message']['from_email'] = $from->getAddress();
+
+            if ('' !== $from->getName()) {
+                $payload['message']['from_name'] = $from->getName();
+            }
         }
 
         foreach ($email->getAttachments() as $attachment) {

Dunno if it helps decide, but this affects deps=high on 4.4 also.

How does Mandrill use the from_email field? Is it used like the Sender or like the From header? If it's the latter, using the sender as the input for this attribute looks indeed wrong to me (same would apply to other APIs).

@xabbuh not sure how Mandrill is handling that but Mailgun seems to use it for the From header.

See https://documentation.mailgun.com/en/latest/api-sending.html#sending

And there we also use the envelope sender: https://github.com/symfony/symfony/blob/master/src/Symfony/Component/Mailer/Bridge/Mailgun/Transport/MailgunApiTransport.php#L94

But that means the "Sender customization feature" using EnvelopeListener would then not work anymore if we change that? 😕

@dmaicher see #37580

I'am sorry but i have to ask, will this "feature" be part of the 5.2 release or will it be an addition to 5.1.4?

Another Question is there currently any way to temporarily replace/bypass the \Symfony\Component\Mime\Encoder\IdnAddressEncoder or worse the \Symfony\Component\Mime\Address to create an workaround for this issue?

@MaSpeng this was merged as a feature in master branch (see 63f8827) which corresponds to the upcoming 5.2 version.

According to the Symfony Release Process minor versions (e.g. 5.1.4) only contains bug fixes and never new features. So, this will be included only in 5.2 and higher versions, but not in 5.1.*. The good news is that Symfony 5.2 release is just a few months ago (feature freeze starts in just 6 weeks!) and you can upgrade your app from 5.1 to 5.2 without breaking changes.

@javiereguiluz Many thanks for the explanations.