15. Website Statistics

UV（Unique Visitor）

• 独立访客,需通过用户IP排重统计数据
• 每次访问都要进行统计
• HyperLogLog,性能好,且存储空间小
  DAU（Dail Active User）
• 日活跃用户,需通过用户ID排重统计数据
• 访问过一次,则认为其活跃（自定义）
• Bitmap,性能好,and accurate results can be obtained

1、RedisKey

    // 单日UV
    public static String getUVKey(String date) {
    
        return PREFIX_UV + SPLIT + date;
    }

    // 区间UV
    public static String getUVKey(String startDate, String endDate) {
    
        return PREFIX_UV + SPLIT + startDate + SPLIT + endDate;
    }

    // 单日活跃用户
    public static String getDAUKey(String date) {
    
        return PREFIX_DAU + SPLIT + date;
    }

    // 区间活跃用户
    public static String getDAUKey(String startDate, String endDate) {
    
        return PREFIX_DAU + SPLIT + startDate + SPLIT + endDate;
    }

2、Service

由于使用Redis存储数据,So no access is requiredDAO层,直接在ServiceThe layer handles the data.
DataService.java

UV

1、将指定IP计入UV
通过new SimpleDateFormat(“yyyyMMdd”) Specify the date format first

@Service
public class DataService {
    

    @Autowired
    private RedisTemplate redisTemplate;

    private SimpleDateFormat df = new SimpleDateFormat("yyyyMMdd");

    // 将指定的IP计入UV
    public void recordUV(String ip) {
    
        String redisKey = RedisKeyUtil.getUVKey(df.format(new Date()));
        redisTemplate.opsForHyperLogLog().add(redisKey, ip);
    }

2、统计：Statistics for the specified date range
1）The date parameter passed in,是Data类
2）Because you want to count the data within the date range,So to generate a setRediskey：List< String> keyList,其中用到了CalendarThe class loops over dates
3）合并数据
4）调用 redisTemplate.opsForHyperLogLog().size() Get statistics

    // Statistics for the specified date range内的UV
    public long calculateUV(Date start, Date end) {
    
        if (start == null || end == null) {
    
            throw new IllegalArgumentException("参数不能为空!");
        }

        // 整理该日期范围内的key
        List<String> keyList = new ArrayList<>();
        Calendar calendar = Calendar.getInstance();
        calendar.setTime(start);
        while (!calendar.getTime().after(end)) {
    
            String key = RedisKeyUtil.getUVKey(df.format(calendar.getTime()));  // 单日UV
            keyList.add(key);
            calendar.add(Calendar.DATE, 1);                      // 日期+1
        }

        // 合并这些数据
        String redisKey = RedisKeyUtil.getUVKey(df.format(start), df.format(end));  // 生成区间UV的key
        redisTemplate.opsForHyperLogLog().union(redisKey, keyList.toArray());

        // 返回统计的结果
        return redisTemplate.opsForHyperLogLog().size(redisKey);
    }

DAU

1、根据 userId 将指定用户计入DAU

    public void recordDAU(int userId) {
    
        String redisKey = RedisKeyUtil.getDAUKey(df.format(new Date()));
        redisTemplate.opsForValue().setBit(redisKey, userId, true);
    }

2、Statistics for the specified date range内的DAU
与calculateUV() 方法类似,The difference is to be right data within the intervalOR操作,且connection.bitOp()要求传入RedisKey的Byte数组

    public long calculateDAU(Date start, Date end) {
    
        if (start == null || end == null) {
    
            throw new IllegalArgumentException("参数不能为空!");
        }

        // 整理该日期范围内的key
        List<byte[]> keyList = new ArrayList<>();	// 将RedisKey 转换成 byte数组
        Calendar calendar = Calendar.getInstance();
        calendar.setTime(start);
        while (!calendar.getTime().after(end)) {
    
            String key = RedisKeyUtil.getDAUKey(df.format(calendar.getTime()));
            keyList.add(key.getBytes());   
            calendar.add(Calendar.DATE, 1);
        }

        // 进行OR运算
        return (long) redisTemplate.execute(new RedisCallback() {
    
            @Override
            public Object doInRedis(RedisConnection connection) throws DataAccessException {
    
                String redisKey = RedisKeyUtil.getDAUKey(df.format(start), df.format(end));
                connection.bitOp(RedisStringCommands.BitOperation.OR,     // OR运算
                        redisKey.getBytes(), keyList.toArray(new byte[0][0]));
                return connection.bitCount(redisKey.getBytes());                 // Statistics aretrue的个数
            }
        });
    }
}

3、Controller

@DateTimeFormat(pattern = “yyyy-MM-dd”) ：Date start,is the handling of date parameters.
return “forward:/data”：forward请求转发.Declare the method to only handle half of it,Another method is required to proceed,请求转发到了 “/data” 路径,Because of the same request,所以"/data"Paths are also supported RequestMethod.POST 请求.

@Controller
public class DataController {
    

    @Autowired
    private DataService dataService;

    // 统计页面
    @RequestMapping(path = "/data", method = {
    RequestMethod.GET, RequestMethod.POST})
    public String getDataPage() {
    
        return "/site/admin/data";
    }

    // 统计网站UV
    @RequestMapping(path = "/data/uv", method = RequestMethod.POST)
    public String getUV(@DateTimeFormat(pattern = "yyyy-MM-dd") Date start,
                        @DateTimeFormat(pattern = "yyyy-MM-dd") Date end, Model model) {
    
        long uv = dataService.calculateUV(start, end);
        model.addAttribute("uvResult", uv);
        model.addAttribute("uvStartDate", start); // Put the date parameter inModel里,In order for the page to have the default value displayed
        model.addAttribute("uvEndDate", end);
        return "forward:/data";
    }

    // 统计活跃用户
    @RequestMapping(path = "/data/dau", method = RequestMethod.POST)
    public String getDAU(@DateTimeFormat(pattern = "yyyy-MM-dd") Date start,
                         @DateTimeFormat(pattern = "yyyy-MM-dd") Date end, Model model) {
    
        long dau = dataService.calculateDAU(start, end);
        model.addAttribute("dauResult", dau);
        model.addAttribute("dauStartDate", start);
        model.addAttribute("dauEndDate", end);
        return "forward:/data";
    }

}

4、拦截器

Data is logged for each request,So the interceptor is used here

@Component
public class DataInterceptor implements HandlerInterceptor {
    

    @Autowired
    private DataService dataService;

    @Autowired
    private HostHolder hostHolder;  // 获取当前登录用户

	// 在Controller之前执行
    @Override
    public boolean preHandle(HttpServletRequest request, HttpServletResponse response, Object handler) throws Exception {
    
        // 统计UV
        String ip = request.getRemoteHost();  // 获取IP
        dataService.recordUV(ip);        // 计入UV

        // 统计DAU
        User user = hostHolder.getUser();
        if (user != null) {
    
            dataService.recordDAU(user.getId());
        }

        return true;   // 请求继续向下执行
    }
}

配置 DataInterceptor,Intercept all requests except static resources
WebMvcConfig.java

    @Autowired
    private DataInterceptor dataInterceptor;

    @Override
    public void addInterceptors(InterceptorRegistry registry) {
    
    	...
        registry.addInterceptor(dataInterceptor)
                .excludePathPatterns("/**/*.css", "/**/*.js", "/**/*.png", "/**/*.jpg", "/**/*.jpeg");
    }