SQL and list de duplication

One 、sql Use scenarios 1： The last approval reason corresponding to the paging query task

1、 Query master sql： Use the main table task Link the audit record table resion

SELECT * FROM task t LEFT JOIN resion r ON t.id=r.task_id  LEFT JOIN xxxtable;

2、 Problem analysis ： One task corresponds to multiple approval records , Linked queries produce Cartesian products , Result set error . Yes sql To transform

SELECT
	* 
FROM
	task t
	LEFT JOIN (
	SELECT
		r.* 
	FROM
		resion r
		INNER JOIN ( SELECT task_id, max( create_time ) create_time FROM resion GROUP BY task_id ) a ON a.task_id = r.task_id 
	AND a.create_time = r.create_time 
	) e ON e.task_id = t.task_id

3、 Optimize ： When two audit records appear at the same time, data set errors cannot be avoided , Therefore, you can add additional conditions to ensure the uniqueness of the schedule

SELECT
	* 
FROM
	task t
	LEFT JOIN (
	SELECT
		r.* 
	FROM
		(
		SELECT
			a.*,
			count( 1 ) onlyValue 
		FROM
			resion a
			INNER JOIN resion b ON a.task_id = b.task_id 
			AND a.create_time <= b.create_time 
		GROUP BY
			a.resion_id 
		) r 
	WHERE
		r.onlyValue = 1 
	) e ON e.task_id = t.task_id

Two 、list To reprocess

1、 use TreeSet To and fro

1.1、 Two conversions

//  Here is the raw data without weight removal 
List<TaskInfo> taskList=...;  
//  according to id Attribute de duplication 
Set<TaskInfo> taskSet = new TreeSet<>(Comparator.comparing(TaskInfo::getId));
taskSet.addAll(taskList);
taskList=new ArrayList<TaskInfo>(taskSet);

1.2、 Flow conversion

 List<TaskInfo> newList = taskList.stream().collect(Collectors
  .collectingAndThen(
   Collectors.toCollection(() -> new TreeSet<>(Comparator.comparing(TaskInfo::getId))),
   ArrayList::new));

1.3、 Multi attribute joint de duplication

Set<TaskInfo> taskSet = new TreeSet<>(Comparator.comparing(o -> (o.getId() + "" + o.getDeptId())));
taskSet .addAll(taskList);
taskList=new ArrayList<TaskInfo>(taskSet);

2、 use map Handle

2.1、 filtration

List<TaskInfo> taskList = new ArrayList<>();
oldList.stream().filter(distinctByKey(t ->t.getId())).forEach(taskList::add);

static <T> Predicate<T> distinctByKey(Function<? super T, ?> keyExtractor) {
 Map<Object,Boolean> seen = new ConcurrentHashMap<>();
 return t -> seen.putIfAbsent(keyExtractor.apply(t), Boolean.TRUE) == null;
}

2.2、 The stream packet loop takes the first