Message queue avoiding repeated refund by idempotent design and atomic lock

Idempotent design

Let's review Last tutorial Created  RefundAttendee  The task class , It does two things —— Refund and email ：

If the queue task fails , Because of the configuration  tries  The value is 3, So I'll try again , There is a hidden danger here ： If the refund is successful , But the mail sending exception causes the task to fail and try again , There is a problem of repeated refunds .

When we design the execution of queue tasks , Due to the retry mechanism , So consider multiple executions Idempotency , For the current refund scenario , Is to execute many times  RefundAttendee, There will be no repeated refunds . To achieve this idempotence , Can be in  handle  Method first determines whether the user has refunded , Only non refunded users can initiate a refund operation ：

Double check

It should be noted that , Call the third-party payment service API A refund will initiate a network request , If the refund is successful , However, the return response failed due to network problems , Then the order will never be marked as refunded . The same problem may occur when the order refund status is updated after the refund .

In this case , Only third-party payment service providers are the only trusted data sources , Only they know whether the refund operation has been completed . therefore , We need to be based on the API To determine whether the refund is successful , Instead of local database fields ：

This point is in the development of third-party services API Interactive functions are very important , It is also the most prone to problems , But it is not convenient to check .

and  refund  The method is the same ,wasRefunded  The method will also pass HTTP Request the refund status from the service provider ：

Triple check and atomic lock

Although we do not subjectively treat the same  RefundAttendee  Push to queue multiple times , But accidents always happen .

If two queue processor processes get the two tasks that execute the same refund at the same time , There may be simultaneous sending HTTP To request and make a refund , Then there is the concurrent security problem ：

To avoid this problem , have access to Atomic lock To ensure that each refund operation is executed serially ：

Before doing anything , First, based on the cache key  refund.{id}  Get an atomic lock , Only by acquiring this lock , To perform the refund operation , Since the operation of allocating locks is atomic , When other concurrent processor processes execute here, they skip directly because the lock has been obtained by other processes .

When the task is completed , The lock will be automatically released .

Here we use a cache based atomic lock , This is a Laravel Functions provided by the bottom layer , besides , And based on Redis、 Implementation version of the database , For details, please refer to the be based on Redis Implement distributed locking and its application in Laravel The underlying implementation source code   This tutorial .

Retry through lock management

If the queue processor process crashes while executing a task after acquiring a lock , And the lock hasn't been released yet ,RefundAttendee  It will be considered successful , Even if the refund operation has not been performed —— Because in the subsequent retry , This lock is still there .

To solve this problem , You need to set the expiration time of the atomic lock and ensure that the lock has been released the next time you retry ：

We go through  Cache::lock  The second parameter of tells that the expiration time of the lock is 10s.

After the processor process crashes , The next retry is on 90s after （ Refer to the previous tutorial Avoid repeated execution of queue tasks In the introduction ）, The lock has expired at this time , You will get a new lock to perform this task .

Configure task delay

If an exception occurs during task execution , The lock will release automatically , But what if something goes wrong when you release the lock ？

We already know that the lock will be in 10s After expired , But in the meantime , When retrying the task, the lock is still there , Will not perform this task . To solve this problem , You can set a reasonable delay time in this task class ：

If the task fails , Will be in 11s After performing , such , This ensures that even if the lock release fails , You can also perform this task normally .

Be careful ： If the queue task fails ,11s Post retry , If the queue processor process crashes , It is 90s Post retry , The two retry times are different .