Cancellation meeting and automatic refund processing of message queue

2020 year , Many industry exchange meetings have been cancelled due to the COVID-19 , This involves dealing with refunds , The ideal scenario is when we click the Cancel button in the system management interface , The refund of all paid users will be automatically processed in batch in the background , Then send the notification of meeting cancellation and refund success to the corresponding user .

Write a task class

For such time-consuming operations , It is obvious that message queuing can be used for asynchronous processing . We create a corresponding meeting cancellation task class  CancelConference, stay  handle  In the method , We are  each  Method to process refunds for each participant , And send an email notification ：

Processing queue task timed out

Inside this , If a third-party payment service is used , The refund needs to involve a network request , Sending mail is also a time-consuming operation , and Laravel The default timeout for a task class is 60s, Maybe not enough , You can specify  --timeout  Extend it ：

Here we set it to 300 minute , That is to say 5 Hours , The reason why the setting is so long , Because  CancelConference  All participants will be refunded and notified during one execution .

Of course , and  tries、--backoff  Same as other parameters , You can also set the timeout through the properties of the task class ：

This is recommended , Because only this task needs to set a long timeout , Other tasks don't take that long .

Avoid repeated execution of queue tasks

When getting tasks from the queue , The task is marked as  reserved, In this way, it can not be acquired by other processors , When the task is completed , Either remove from the queue , Or it is pushed to the queue again to try again .

However, if the queue processor process crashes during the execution of the queue task, the task execution is interrupted , Will always be marked as  reserved, Will no longer be executed .

To avoid this problem ,Laravel by  reserved  The status sets the timeout , The default is 90s, Can be in  queue.php  Configure this value in the configuration file ：

Because we are  CancelConference  The task class has set the timeout to 5 Hours , One problem this can cause is 90s after ,reserved  State failure , Other processors can get this task class to execute , Repeated execution of the same task occurs .

To solve this problem , Need to ensure  retry_after  The value of is greater than the timeout of all task classes ：

Make sure you have enough processors

If a task needs to be performed 5 Hours , If there is only one queue processor process , Other queue tasks will be blocked , To do this, you need to start multiple processor processes , But it is also possible that all processor processes are executing these time consuming processes at the same time 5 Hours of work , One solution is to push these tasks to separate queues ：

And allocate enough processor processes to it ：

Use separate connections to handle time-consuming tasks

Press and hold the gourd to lift the ladle , Only 1 Queue tasks take 5 Hours , With all the other tasks  reversed  The state requires 5 It takes more than hours to expire , It's not the right thing to do , To solve this problem , You can set up separate connections for this queue ：

Then specify the connection when pushing very time-consuming tasks to the queue ：

Or set the connection in the task class through the attribute value ：

Finally, we can specify the queue connection when starting the processor process ：

php artisan queue:work database-cancelations

A queue processor process that specifies a connection can only consume queue tasks in that connection .

Split a single task into multiple subtasks

our  CancelConference  Task classes may need to run for up to 5 Hours to complete all refunds and email , During this period, the queue processor process can only handle this task , If an exception occurs during execution , And start from scratch , And skip refunds that have already been processed , There are uncontrollable and high risks .

So , We can split this time-consuming single task distribution into multiple subtask distributions , It's not hard to realize , It is the implementation code that processes all refunds and notifications in the original task ：

Adjust to one task to process only one user's refund and notification , We build a new one  RefundAttendee  Task class to handle this work , Incoming from outside  RefundAttendee  No longer  Conference  example , It is  Attendee  The instance ：

thus , We can revert to using a single queue connection , There is no need to configure those extra timeout fields ：

here , We are actually working in a queue （CancelConference） Add another queue task （RefundAttendee） Push to message queue for processing .