Re13: read the paper gender and racial stereotype detection in legal opinion word embeddings

The gods were silent - personal CSDN Blog Directory

Title of thesis ：Gender and Racial Stereotype Detection in Legal Opinion Word Embeddings
The paper ArXiv Download address ：https://arxiv.org/abs/2203.13369
The paper AAAI Official preprint download address ：https://www.aaai.org/AAAI22Papers/AISI-10870.MatthewsS.pdf
Official video ：https://aaai-2022.virtualchair.net/poster_aisi10870（ The author speaks English very fast , Popping ）

This article is about 2022 year AAAI The paper , Pay attention to the fairness of machine learning , American Civil Law for testing judicial opinions（ Legal opinion legal opinion） Gender and racial stereotypes embedded in trained words （stereotype,bias）.
This article focuses on historical and representation bias. Experiments will prove that historical factors （ The age of the case used for word embedding training ） Not the main influencing factor .

It feels strange to detect fairness problems , Manual work is more than machine work .

1. Background

Implicit Association Test (IAT) Measure human participants' response to the target word （ Flowers or insects ） And attribute terms （ Happy or unhappy ） Reaction time for classification .

Word Embedding Association Test (WEAT)： Measure target word grouping （ Such as men and women ） And attribute terms （ Such as positive or negative emotions ） The similarity of , For example, measure whether male related word embedding is closer to positive emotional word embedding . The method is to measure two groups of target words （ Such as typical male or female names ） And attribute terms （ Such as happy （love peace） Or unhappy （ugly hatred）） The similarity between embedded words （association, Cosine similarity ）

bias classification ：historical, representation, measurement, aggregation, evaluation, and deployment biases

2. Difficulties and corresponding solutions

1. The legal text is more formal , Use regular personal pronouns many times , The person's name 、 surname 、 Personal pronouns may be embedded bias, Only checking the person's name will lead to other bias The loss of .→ Use surnames related to race .
2. There is a shortage of women among legal workers , May cause gender-occupational stereotypes.
3. It cannot be directly used in the legal field open-domain Emotional vocabulary →WEAT The attribute vocabulary of the test uses a general vocabulary , add domain specific and expanded word lists（ Selected some iconic words as seed terms, Then use word embedding to generate expanded word lists（ Positive word ： And The vector difference between existing positive words and existing negative words The cosine similarity of this vector is high . Negative words are the opposite ）, Then manually review and delete words with obvious racial or gender characteristics ）
4. IAT The test mainly considers the positivity and negativity of attributes , But for legal issues , The impact of the results is greater → Use some measures to measure the impact of legal opinions on the results grant or deny To measure the positive and negative of the result .

Extract phrases （Idiomatic Phrase Extraction）→ Training skip-gram word2vec model Word embedding （ In all corpora 、 By time or legal topic The cut sub corpus is trained separately ）→ On gender and race WEAT testing
Gender ： Names and other typical demonstrative pronouns
race ： surname

Optimize ：

1. Idiomatic Phrase Extraction： In order to prevent n-gram dictionaries Too big , Only phrases with high frequency of common occurrence are considered , use Normalized Point-wise Mutual Information (NPMI) Indicators to select phrases added to the dictionary .
2. The last name may coincide with the company name ：
   1. Title cased the surnames to target proper nouns.
   2. Idiomatic phrase extraction Exclude some non human names .
   3. Centroid-based filtering to remove multi-sense words.（ Calculate the representation of all surnames , Calculate the representation of each surname and all surnames centroid Cosine similarity of , Delete 20% The last name with the lowest similarity ）（ Names are handled in a similar way ）

Experimental setup ：

1. Phrase extraction phase NPMI The threshold of is 0.5.
2. The dimension of word embedding is 300, The lowest frequency of words is 30,sampling threshold by $10^{-4}$, The learning rate is 0.05,window size by 10,negative samples by 10.
3. WEAT The standard deviation is calculated （by sub-sampling the word lists with a simple bootstrapping procedure）
   
   
4. Considering that there is a more serious problem of discrimination in American history , Therefore, the time factor is excluded （temporal effect）, But there are still unfair problems .
   practice ： Time segmented corpus , Train word embedding in different time periods , Conduct WEAT test
   
5. Gender stereotypes , Use different target words ：
   
6. Considering the difference legal topic： The corpus is divided into different topic Segmentation .（ To prevent low frequency effects , Deleted occurrence frequency less than 30 Attribute words of ）

3. Code reappearance

The paper does not give the public code , But it doesn't seem difficult to reproduce （ Just get the data set ）, When my server is ready and I have time, I will write a ！

4. Other practices related to fairness

LeSICiN¹ and ECHR² Is to name entities mask, To reduce demographic bias.

---

1. Re6： Reading papers LeSICiN: A Heterogeneous Graph-based Approach for Automatic Legal Statute Identification fro︎

2. Neural Legal Judgment Prediction in English︎