Django celery redis send email asynchronously

Django-Celery-Redis Asynchronous email

python == 3.7.6

django == 2.1.8

celery == 4.4.7

redis == 3.5.3

eventlet == 0.26.1

What is? celery

Celery Is a fully functional plug and play task queue . It makes it unnecessary for us to think about complicated problems , Very simple to use .celery It is suitable for asynchronous processing , When sending mail 、 Or file upload , Image processing and other time-consuming operations , We can execute it asynchronously , So users don't have to wait long , Improve user experience . celery Is characterized by ：

• Simple , Easy to use and maintain , There is a wealth of documentation .
• Efficient , Single celery Processes can handle millions of tasks per minute .
• flexible ,celery Almost every part of the can be custom extended .

celery The core

1、Task

Mission (Task) That's what you're going to do , For example, there are many tasks in a registration process , Sending verification email to users is a task , This time-consuming task can be entrusted to Celery To deal with , Another kind of task is a scheduled task , For example, regularly count the number of registered people on the website every day , This can also be handed over to Celery Periodic processing .

2、Broker

Broker The agent , Refers to the person who provides intermediary services for buyers and sellers in the market . stay Celery In, it is between producers and consumers , The data structure of this role is equivalent to that of the queue . For example, one Web In the system , Producers deal with the core business Web Program , There may be some time-consuming tasks in the business , Like text messages , The producer will send the task to Broker, Is to put this task in the queue temporarily , Waiting for the consumer to handle . The consumer is Worker, It is a background service specially used to perform tasks .Worker It will monitor whether there are new tasks in the queue in real time , If so, take it out and deal with it .Celery It does not provide queue service itself , It's usually used Redis perhaps RabbitMQ To play Broker Role

3、Worker

Worker The one who's been working backstage , Also known as task consumers , It will monitor whether there are tasks in the queue in real time , If so, take it out immediately and execute .

4、Beat

Beat Is a scheduled task scheduler , It will send the task to according to the configuration timing Broker, wait for Worker To consume .

5、Backend

Backend Used to save the execution result of the task , Each task has a return value , For example, the email service will tell us whether it has been sent successfully , The result is existence Backend in , Of course, we don't always care about the results of the task .

Broker choice

Celery We need a way to send and receive messages , We call this intermediate device used to store messages message broker, It can also be called a message broker . As a middleman , We have several options ：

RabbitMQ

RabbitMQ Is a fully functional , Stable and easy to install broker. It is the best choice in the production environment .

Use RabbitMQ For details, please refer to the following links ： http://docs.celeryproject.org/en/latest/getting-started/brokers/rabbitmq.html#broker-rabbitmq

Redis

Redis It is also a fully functional broker optional , But it is more likely to cause data loss due to accidental interruption or power failure .

Redis As Broker, Visit the following website ： http://docs.celeryproject.org/en/latest/getting-started/brokers/redis.html#broker-redis

take redis Publish subscribe mode is used for message queuing and rabbitmq The difference between ：

reliability

redis： There is no corresponding mechanism to ensure the reliable consumption of messages , If the publisher publishes a message , Without a corresponding subscriber , This message will be lost , Not in memory ;rabbbitmq： With message consumption confirmation mechanism , If you publish a message , No consumer has consumed the queue , Then this message will always be stored in the queue , Until a consumer consumes the message , This can ensure the reliable consumption of messages .

The real time

redis Real time high ,redis As an efficient cache server , All data exists in the server , So it has higher real-time consumer load balancing ;rabbitmq The queue can be monitored by multiple consumers at the same time , But each message can only be consumed once , because rabbitmq Consumption confirmation mechanism , Therefore, it can adjust its load according to the consumption ability of consumers ,redis Publish subscribe mode , A line It can be subscribed by multiple consumers at the same time , When a message arrives , The message is sent to each subscriber in turn ;

persistence

redis：redis Persistence of is for In the whole redis The contents of the cache , It has RDB and AOF Two persistence methods , You can put the whole redis The instance is persisted to disk , So as to make data backup , Prevent data loss under abnormal conditions .rabbitmq： queue 、 Messages can be selectively persisted , Persistence granularity is smaller , More flexible ; Queue monitoring rabbitmq The background monitoring platform is realized , You can see the details of all created queues on the platform , A good backstage management platform can facilitate our better use ;redis There is no so-called monitoring platform .

summary ：

redis： Lightweight 、 Low latency , High concurrency 、 Low reliability ;

rabbitmq： heavyweight 、 Highly reliable , asynchronous , Real time is not guaranteed

stay Django Use in celery

django Project name ：celery_test

stay django project celery_test/celery_test/ Create celery.py file , Configure the following ：

# celery.py file 
import os
from celery import Celery

#  hold celery and django Are combined , Identification and loading django Configuration file for 
os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'celery_test.settings')

#  establish celery example 
app = Celery('celery_test')

#  Appoint celery Configuration of message queue 
app.config_from_object('celery_test.config', namespace='CELERY')

#  From all of django-app Load tasks in 
app.autodiscover_tasks()


@app.task(bind=True)
def debug_task(self):
    print('Request: {0!r}'.format(self.request))

stay django project celery_test/celery_test/ Create config.py file , Configure the following ：

#  Message broker settings 
broker_url = 'redis://127.0.0.1:6379/15'
#  Result storage settings 
result_backend = 'redis://127.0.0.1:6379/14'

stay django project celery_test/celery_test/ Next __init__.py Write the following in ：

#  Absolutely quote , Make our celery The module will not be associated with the original celery Conflict 
from __future__ import absolute_import, unicode_literals
#  After the introduction of absolute , How to import the content under the current module ： from xx import xx as xx
from .celery import app as celery_app

__all__ = ('celery_app',)

stay settings.py Configure the content of sending email ：

#  Configure mail delivery 
EMAIL_BACKEND = 'django.core.mail.backends.smtp.EmailBackend'
EMAIL_HOST = 'smtp.qq.com'  #  If 163 mailbox , Set to smtp.163.com
EMAIL_PORT = 25  #  perhaps  465/587 It's set up  SSL  encryption 
#  E-mail address 
EMAIL_HOST_USER = '3735***[email protected]'
#  Client authorization password set in mailbox 
EMAIL_HOST_PASSWORD = 'xxxxxxxxxxxxx'  #  The authorized password used for third-party login 
EMAIL_USE_TLS = True  #  This must be  True, Otherwise, the sending will not succeed 
#  Sender seen by the recipient ,  Must be always and effective 
EMAIL_FROM = ' The moon on the sea <3735***[email protected]>'
DEFAULT_FROM_EMAIL = EMAIL_HOST_USER

stay django Of app Created in tasks.py file , Write asynchronous task content here ：

#  Absolutely quote , Make our celery The module will not be associated with the original celery Conflict 
from __future__ import absolute_import, unicode_literals
#  Import original celery Module shared_task from xx import xx
from celery import shared_task  
#  Use django Built in function to send mail 
from django.core.mail import send_mail
#  Import django Of settings
from django.conf import settings  


@shared_task
def send_mail_task(usernaem, email, token):
    """  Use django Built in function to send mail  """
    subject = " The moon on the sea "
    message = ""
    sender = settings.EMAIL_FROM
    recipient = [email]
    html_message = "<h1>{}, Welcome to register , Please click the following link to activate your email ：<a href='http://127.0.0.1:8000/api/v1/active/{}'> Click here to activate </a></h1>".format(username, token)
    send_mail(subject, message, sender, recipient, html_message=html_message)

stay django Of app Under the views.py Call in tasts Next task

from django.views import View
from django.http import JsonResponse
from .tasks import send_mail_task
from itsdangerous import TimedJSONWebSignatureSerializer
from django.conf import settings

#  Send E-mail 
token_serializer = TimedJSONWebSignatureSerializer(settings.SECRET_KEY, 600)


class SendMailView(View):
    def post(self, request):
        username = 'Edward'
        email = '3706***[email protected]'
        user_info = {
    'user_id': '1'}
        token = token_serializer.dumps(user_info).decode()
        send_mail_task.delay(username, email, token)
        return JsonResponse({
    'msg': 'OK'})

After the mail is sent successfully , adopt get To get the user's token

from django.views import View
from django.http import JsonResponse
from itsdangerous import TimedJSONWebSignatureSerializer, SignatureExpired
from django.conf import settings

#  Send E-mail 
token_serializer = TimedJSONWebSignatureSerializer(settings.SECRET_KEY, 600)


class ActiveView(View):
    """  Activate user  """
    def get(self, request, token):       
        try:
            user_info = token_serializer.loads(token)            
            return JsonResponse({
    'msg': ' Activation successful ', 'code': 200})
        except SignatureExpired:
            return JsonResponse({
    'msg': ' Activation link failed , Please re send email to activate ', 'code': 404})
        

start-up celery

open pycharm Under the Terminal, Go to the project root directory , Use eventlet start-up celery

celery -A celery_test worker -l info -P eventlet