Notes on the infrastructure of large websites

Notes on the basis of large website architecture

1. Architecture paradigm

• Layered architecture
• Distributed architecture
  • Distributed applications and services
  • Distributed static resources
  • Distributed data and storage
  • Distributed computing  Hadoop, MapReduce
• Business split
• Cluster deployment
• Cache deployment
  • CDN
  • Reverse proxy
  • Local cache
  • Distributed cache
• Asynchronism
• Resource redundancy
• automation

2. The five elements of Architecture

• performance
  • network performance   CDN, bandwidth
  • Server hardware performance   CPU, Memory , Mechanical drive ,SSD
  • Load balancing   LVS  four layers
• High performance services
  • Browser side     Browser cache , Page compression ,  Reduce Cookie
  • Application server   Local cache ,  Distributed cache , Asynchronous operations
  • Database server   Indexes , cache ,SQL performance optimization
• Usability
  • 7X24 Hours .
  • Resource redundancy
  • System availability during downtime
• Scalability
  • Function split , Physical separation , Achieve scaling
  • Single function clustering
• Extensibility
  • Event driven , Message queue
  • Distributed server framework , Call service mode separation
• Security
  • confidentiality
  • integrity
  • Authenticity
  • Possessiveness

3. Performance indicators

• response time
• Concurrency number
• throughput
• Health

4. cache

• memcached
• Redis
• Applicable to low frequency data modification
• Applicable to hotspot access data
• Data inconsistency
  • Through asynchronous threads
  • Solve through scheduled tasks
  • Solve through status verification
• Cache availability
  • Distributed
  • Copy
• Cache penetration problem
  • Data with an empty result is also cached
  • It must not exist key Use a Burong filter
• Distributed cache

5.CDN cache

6. Load balancing

• Seven layer load balancing  Nginx
• Four layer load balancing  LVS F5
• Reverse agent principle
  • Be a stand in for the internal server
  • As load balancing
• benefits
  • Improve internal server security
  • Improve responsiveness
  • save IP Address resources

7. Distributed database

• mysql
• advantage
  • Distributed data management , Increase performance
  • Economical and flexible
  • High reliability and availability
• Data cutting
  • Vertical segmentation
  • Transverse segmentation
• Read / write separation
• Master slave synchronization , Data integrity
  • Code based implementation
  • Use the intermediate layer
  • middleware
    • Cobar,MyCat,OneProxy,Viteness,KingShard,Atlas,MaxScale

8. Storage optimization

• SDD
• RAID
• HDFS
  • Hadoop2.0

9. Message queue

• A synchronous invocation
• Asynchronous call
• producer - Consumer model
• Two models ：
  • Point to point model
  • Subscription model
• Common products
  • RabbitMQ
  • ActiveMQ
  • RocketMQ
  • Kafka

10. Distributed service data consistency

• Two phase submission agreement
  • Preparation stage
  • Submission phase
• Three stage submission agreement
• TCC agreement
• Solution
  • Query mode
  • Compensation mode
  • Asynchronous assurance mode
  • Timed proofreading mode

11. Distributed log service

• EFK
  • elasticsreaarch
  • filebeat
  • kibana
  • logstash
  • kafka
  •