Thank you for this issue.

What if the message handler fails when you execute it for the 3rd time? Should the reschedule stop or continue?

This sounds very much that you want to implement a cronjob. I do like the idea and I would think it would be useful in a bundle/library, but I dont think it should be merged in to the core.

@Nyholm , I'm going to explain better what I mean...

This sounds very much that you want to implement a cronjob.

Yes and no...

It is LIKE a cronjob. But a cronjob has a problem of sovraposition between jobs and is not configurable in a finegrained way.

Imagine you have an app where users can connect their Google Drive account and upload n the app some files.

One user may have 10 files for a total amount of 100MB, while another may have 1000 files and 3GB of files.

Syncing the first account may require 10 seconds, while syncing the second account may require 30 minutes: same task but very different times.

Messenger solves this problem because you simply send the message and then the sync is executed (and you can decide where you want it to be executed: a lambda function, the same app, another server with the same code... it is a queue, not a cronjob).

With a cronjob, you have to implement all of this on yourself (duplicating the code that does something that Messenger already does and that it does very well).

What if the message handler fails when you execute it for the 3rd time?

This is something that is too much variable.

The causes of a failure are very variable and may depend on a wrong implementation, lack of connectivity, endpoint down, wrong credentials and many more...

I don't think this is something that Symfony should take into account: this is a responsibility of the developer that has to monitor his/her own app and fix errors.

Symfony simply provides a "helper" that make easier to reschedule the message.

Is the developer that has to choose if (s)he want to use the proposed middleware or has to use a more custom logic (after all, this is exactly the purpose of allowing for custom middlewares in the component, doesn't it?).

The proposed PR is a simple starting point that covers a basic need: for more advanced needs, a custom middleware is required anyway.

And we arrive at the third point:

Should the reschedule stop or continue?

The reschedule continues because this is what the middleware does: if the developer knows that the task that (s)he is performing can cause problems with "automatic" (because it is not so automatic, anyway: you have anyway to configure it), then (s)he will not use the middleware and will go with another solution/implementation).

It is anyway possible to read this information in the middleware, reading the RedeliveryStamp...

What to do is something we should discuss together, but it is possible to decide what to do...

I don't think it should be merged in to the core

I want to point out that it is in the core but not in the Core.

I mean that the Core classes (Worker, MessageBus, transports implementation, etc.) are not touched in any way.

We are speaking of simply adding two classes: the middleware and the stamp and they are not configured automatically, but has to be explicitly enabled by the developer that decides to use them.

They are like the DelayStamp: it is used only if the developer needs to delay the execution.

In the same way, if the developer needs to implement a re-scheduling mechanism, then (s)he activates the middleware and uses the corresponding RescheduleStamp: if (s)he don't need the functionality, then (s)he will not configure it.

As said, if the developer needs more custom or complex logic, then (s)he will go with a more custom implementation and will not use the one provided by the Messenger Component.

Conclusion

This is a basic implementation that covers basic needs: exactly as a component should do. For custom requirements a custom code is required.

Thank you for expanding on this.

Messenger solves this problem because you simply send the message and then the sync is executed

Correct, but I dont see how this argument is related to a RescheduleStamp.

What if the message handler fails when you execute it for the 3rd time?

I don't think this is something that Symfony should take into account: this is a responsibility of the developer that has to monitor his/her own app and fix errors.

I agree. That is why I dont think introducing a RescheduleStamp is a good idea.

What I take away form this is that if you need some code to run every 10 minutes. You should use a cron.

If you want to sync 1000 files, but maybe not in one "go", you could re-dispatch the command. See this handler:

class SyncFilesHandler implements MessageHandlerInterface {

  // ....
  public function __invoke(SyncFiles $command) {
     $files = $command->getFiles();
     $this->doSync(array_slice($files, 0, 10);
     // Dispatch a new message
     $this->commandBus->dispatch(new SyncFiles(array_slice($files, 10)));
  }
}

Im 👎

Correct, but I dont see how this argument is related to a RescheduleStamp.

It is just to explain that it is not a cronjob, but a rescheduling of a single message.

For example, new SynchGoogleDriveAccount($accountId).

The role of the cronjob is not to manage the syncing of a single account, but to manage the syncing of ALL the accounts... And for this task I prefer to use Messenger instead of a cronjob as Messenger gives me much more flexibility.

What I take away form this is that if you need some code to run every 10 minutes. You should use a cron.

Again, it is not some code that has to run every 10 minutes but is a Message.

There is a conceptual difference that has practical consequences: if I have to delete all the failed and not retried messages, then I will use a cron. But if I have to sync 1000 accounts each one of which has very different execution times, then I will go with Messenger because of its long running process: Messenger takes care of running, I take care of telling it what messages to execute, and each message (yes, with the same code) will be able to run for 3 seconds or for 3 hours...

A cronjbob doesn't have this flexibility: If I set it to run each 10 minutes, if my sync requires 3 minutes, I have 7 minutes of doing nothing... If my sync requires 3 hours, I have to stop and start it each 10 minutes to avoid overlaps (and managing this logic can be very very hard!).

I see you arguing for and against cronjobs. I also see you discussion the benefits of the messenger component. But I dont see you arguing for or against RescheduleStamp.

I see you arguing for and against cronjobs. I also see you discussion the benefits of the messenger component. But I dont see you arguing for or against RescheduleStamp.

RescheduleStamp is simply an helper that permits to reschedule the message...

Once we agree that cronjobs are not for all use cases (do we agree on this?), the continuation of the argument is this:

Once I've synced ONE Google Drive account one time, I want to sync it on a regular basis.

How can we do this?

Cronjobs are not the problem anymore: we have solved the problem of different time executions using Messenger instead of cronjobs.

So we have to schedule the next sync using Messenger: how do we do this?

We have two ways:

1. Implement the logic directly in the handler: once the sync is done, it schedules a new message in the future, using the DelayStamp
2. Centralise the re-scheduling logic in a middleware, so, as in two months we will need to sync also DropBox and we will use a different message/handler pair, we are not forced to duplicate the re-scheduling logic in two handlers.

And when, in three months, we will have to sync also Box accounts with a new pair of message/handler, we will not be forced to copy the logic in a third handler.

instead, using a middleware and a stamp, we will do something like this (extremely simplified):

switch ($cloudDrive) {
            case 'google_drive':
                $this->messengerBus->dispatch(new GoogleDriveSync($accountId), [new RescheduleStamp(1, RescheduleStamp::PERIOD_DAYS)]);
                break;
            case 'dropbox':
                $this->messengerBus->dispatch(new DropBoxSync($accountId), [new RescheduleStamp(1, RescheduleStamp::PERIOD_DAYS)]);
                break;
            case 'box':
                $this->messengerBus->dispatch(new BoxSync($accountId), [new RescheduleStamp(1, RescheduleStamp::PERIOD_DAYS)]);
                break;
        }

Once I've synced ONE Google Drive account one time, I want to sync it on a regular basis.

I understand this use case. To solve this you could use cronjobs or the kind of handler I gave an example for earlier.

I was reading also the PR #36512 ...

Reasoning about it, I'm arriving to this conclusion:

there are two use cases that all belong to the DelayStamp (that is used also in the middleware I'm proposing).

The use cases are those:

1. Create a DelayStamp using a future date
2. Create a DelayStamp using the syntax I showed in my previous comment ([Messenger] Implement a RescheduleStamp (and a crrsponding Middleware) #38463 (comment))

The first point is addressed by the PR

The second point, instead, could also be solved using another factory method in the DelayStamp and simply calling the dispatching of the message scheduling in the handler...

In this way there is basically no duplication of code...

WDYT?

Once I've synced ONE Google Drive account one time, I want to sync it on a regular basis.

I understand this use case. To solve this you could use cronjobs or the kind of handler I gave an example for earlier.

May be I'm imaging something different from what you are imagin, but when I tried with cronjobs it was literally a hell: a lot of errors and a lot of problems of overlaps 😩

Ok, thank you for this discussion: I will go with a new factory method executeIn()! 💪

Way more easy to manage, solves all the problems you have pointed out in your previous comments and solves also my own problem.

And removes a lot of uncertainty (like what you highlighted: "what happens if a message fails the 3d time").

Thank you!