Re19: Read the paper Paragraph-level Rationale Extraction through Regularization: A case study on European Court

诸神缄默不语-个人CSDN博文目录

论文名称：Paragraph-level Rationale Extraction through Regularization: A case study on European Court of Human Rights Cases
论文ArXiv下载地址：https://arxiv.org/abs/2103.13084
论文NAACL官方下载地址：https://aclanthology.org/2021.naacl-main.22/（The official video.This blog china-africa illustrations are intercepted from the papers from the video）
本文提出的ECtHR数据集的huggingface下载地址：ecthr_cases · Datasets at Hugging Face

本文是2021年NAACL论文.
任务：alleged violation predictionPredict the fact description text corresponding to violate the law of（highly skewed multi-label text classification task）+可解释性（rationale extraction）
rationale extraction：Extract the case is used to predict the results provide explanation in the text of the paragraph（paragraph-level）

为了提高alleged violation prediction任务中的可解释性（interpretability或explainablity）,From the perspective of the user center to the decision result provides reasonable explanation.本文通过rationale extraction来实现这一目标,Before the mainstream methods were focused onword-level rationales,And this article is frommulti-paragraph structured court casesExtract the paragraph.除此之外,This article also puts forward a markrationales的数据集ECtHR.This article USES the existingrationale constraints（ sparsity, continuity, and comprehensiveness）,And put forward the newsingularity,作为regularizers.

1. Background

1. rationalization by construction方法论：直接用constraint来正则化模型,The decision-making models based on the correctrationalesTo the condition of thereward,Instead of after the event can be interpreted according to the results of the model of decision-making reasoning
   the model is regularized to satisfy additional constraints that reward the model, if its decisions are based on concise rationales it selects, as opposed to inferring explanations from the model’s decisions in a post-hoc manner
2. Can be interpreted the meaning of：right to explanation
3. 执法过程：

2. 模型

2.1 Novelty

1. previous work on word-level rationales for binary classification→paragraph-level rationales
2. The end-to-end fine-tuning for the first trainingTransformer模型中应用rationale extraction的工作
3. Does not need manual annotationrationales

2.2 模型

constraint：以前就有的sparsity, continuity（Experimental results show invalid）, and comprehensiveness（需要根据multi-labelParadigm for correction）,In this paper, the proposedsingularity（能提升效果,And robust）

baseline HIERBERT-HA：text encoder→rationale extraction→prediction

In the video into the figure is：

Word level of regular
①Coding respectively each paragraph：context-unaware paragraph representations
②用2层transformer编码contextualized paragraph embeddings
③全连接层（激活函数selu）
K→用于分类
Q→用于rationale extraction→Each paragraph a full connection layer+sigmoid,得到soft attention
scores→binarize,得到hard attention scores
④得到hardmasked document representation（hard mask+max pooling）（不可微,So there is a trainingtrick）
⑤全连接层+sigmoid
baseline HIERBERT-ALL：不mask事实
constraint：
①Sparsity：Limit the number of selected fact
②Continuity：In this paper model is useless,But the experiment
③Comprehensiveness：Left in the paragraph generated much better results than to throw away,Compare the two paragraphs or cosine similarity
④Singularity：选出的maskIt is better than random
Rationales supervision：noisy rationale supervision

3. 实验

3.1 数据集

提出ECtHR数据集,English case text,silver/gold rationales,Events have time to order,Decisions including breach of law and citing precedent

3.2 实验设置

超参数：
网格搜索,Adam,学习率2e-5

贪心调参

LEGAL-BERT-SMALL：
50 paragraphs of 256 words

3.3 实验结果

指标：
micro-F1
Faithfulness: sufficiency and comprehensiveness
Rationale quality: Objective / subjective (mean R-Precision (mRP) [email protected])

4. 代码复现

等我服务器好了再说.