[Messenger] [Amqp-messenger] Support content encoding and compression #47592
Conversation
fb9ae8d to
278ff6f
Compare
src/Symfony/Component/Messenger/Bridge/Amqp/Compressor/CompressorFactory.php
Outdated
Show resolved
Hide resolved
|
What about allowing compression/decompression with any transports? I guess we could add a middleware to do so? |
9ac9934 to
fff81c6
Compare
|
Hi Nicolas! Thanks for your comments. I have worked on a draft. You can see it in the last commit. In the draft, I try to implement the (de)compression using a middleware. If the draft is close to what you had in mind, I think that we have a problem. The compression expects a string in order to compress it. In the opposite direction, the decompression expects a string to decompress it. When we send a message, the string is only available after the serialization. But the serialization process is inside the Transports. The Transports receive the message object, serialized it and send it. A similar problem arises when we consume the message. The stack of middlewares always has the message object. These are my ideas. What do you think about them? |
When I send a message using the AMQP transport together with the AmqpStamp and the content-encoding property, the message is not compressed.
The AMQP protocol, allow through the content-encoding property, to “indicate what additional content encodings have been applied to the application-data, and thus what decoding mechanisms need to be applied in order to obtain the media-type referenced by the content-type header field”.
The values admitted according to the protocol are: gzip, compress, deflate, identity, etc.
I expect that the body of the message will be compressed and decompressed respectively by the publisher
and the subscriber.
For example, we can send a messages:
and, With the current code the body message (payload) is not compress using gzip:
$ rabbitmqadmin get queue=messages -f pretty_json [ { "exchange": "messages", "message_count": 0, "payload": "{\"name\":\"Foo bar\"}", "payload_bytes": 18, "payload_encoding": "string", "properties": { "content_encoding": "gzip", "content_type": "application/json", "delivery_mode": 2, "headers": { "X-Message-Stamp-Symfony\\Component\\Messenger\\Stamp\\BusNameStamp": "[{\"busName\":\"messenger.bus.default\"}]", "type": "App\\Message\\DummyMessage" }, "timestamp": 1662922329 }, "redelivered": true, "routing_key": "" } ]but, if we are aware of the value of the content_encoding property we should compress the body message.
$ rabbitmqadmin get queue=messages -f pretty_json [ { "exchange": "messages", "message_count": 0, "payload": "H4sIAAAAAAAAA6tWysxNTE8NyC8u8UxRsrI0qwUAri6HchIAAAA=", "payload_bytes": 38, "payload_encoding": "base64", "properties": { "content_encoding": "gzip", "content_type": "application/json", "delivery_mode": 2, "headers": { "X-Message-Stamp-App\\Messenger\\UniqueIdStamp": "[{\"uniqueId\":\"631e4bdc3ad3b\"}]", "X-Message-Stamp-Symfony\\Component\\Messenger\\Stamp\\BusNameStamp": "[{\"busName\":\"messenger.bus.default\"}]", "type": "App\\Message\\AddPonkaToImage" }, "timestamp": 1662929884 }, "redelivered": true, "routing_key": "" } ]Finally, the subscriber must be able to uncompressed the body message. I made some changes in order to figure out how
this can be accomplished.
Thanks!, I should be grateful for any comments.
Postscriptum: I closed the PR 47545 by mistake :).