当前位置:网站首页>Financial risk control practice - decision tree rule mining template
Financial risk control practice - decision tree rule mining template
2022-07-07 05:58:00 【Grateful_ Dead424】
This data is the data of didi drivers using oil loans .
The bad debt rate of oil loan is as high as 5%, Very high , It must lose money . And can pass fraud detection .
The drivers who come to apply for oil loan already have A Stuck. ,A The card is rated A-F. Originally, there was only F Non lending , But in the oil loan, only the rating is A Only by lending can we not lose money .
Didi cooperates with many gas stations , The gas station will provide didi with driver data .
Import related packages
import pandas as pd
import numpy as np
# eliminate Warning
import warnings
warnings.filterwarnings("ignore")
Read the oil data set
data = pd.read_excel("oil_data_for_tree.xlsx")
data.head()
View information about the data
set(data.class_new)
#{'A', 'B', 'C', 'D', 'E', 'F'}
Calculate the bad debt rate
set(data.bad_ind)
#{0, 1}
data.bad_ind.sum()/data.bad_ind.count()
#0.01776363887846035
The characteristics are divided into 3 class
- org_lst No special transformation is required , Direct weight removal
- agg_lst Numerical variables are aggregated
- dstc_lst Text variables do cnt
org_lst = ['uid','create_dt','oil_actv_dt','class_new','bad_ind']
agg_lst = ['oil_amount','discount_amount','sale_amount','amount','pay_amount','coupon_amount','payment_coupon_amount']
dstc_lst = ['channel_code','oil_code','scene','source_app','call_source']
Data reorganization
# Copy , Check for missing
df = data[org_lst].copy()
df[agg_lst] = data[agg_lst].copy()
df[dstc_lst] = data[dstc_lst].copy()
Check for missing
df.isna().sum()
#uid 0
#create_dt 4944
#oil_actv_dt 0
#class_new 0
#bad_ind 0
#oil_amount 4944
#discount_amount 4944
#sale_amount 4944
#amount 4944
#pay_amount 4944
#coupon_amount 4944
#payment_coupon_amount 4946
#channel_code 0
#oil_code 0
#scene 0
#source_app 0
#call_source 0
#dtype: int64
On features creat_dt To complete
Yes creat_dt Complete , use oil_actv_dt To fill , And intercept 6 Months of data .
When constructing variables, you cannot directly accumulate all historical data .
Otherwise, over time , The distribution of variables will vary greatly .
def time_isna(x,y):
if str(x) == "NaT":
x = y
else:
x = x
return x
df2 = df.sort_values(["uid","create_dt"],ascending = False)
df2["create_dt"] = df2.apply(lambda x:time_isna(x.create_dt,x.oil_actv_dt),axis = 1)
df2["dtn"] = (df2.oil_actv_dt - df2.create_dt).apply(lambda x:x.days)
df = df2[df2["dtn"] < 180]
df
#()
On features org_list To deal with
Yes org_list Find the maximum interval of historical loan days , And go heavy
base = df[org_lst]
base["dtn"] = df["dtn"]
base = base.sort_values(['uid','create_dt'],ascending = False)
base = base.drop_duplicates(['uid'],keep = 'first') # Deduplication
base.shape
#(11099, 6)
Variable derivation ( a key )
# Variable derivation
gn = pd.DataFrame()
for i in agg_lst:
tp = pd.DataFrame(df.groupby('uid').apply(lambda df:len(df[i])).reset_index())
tp.columns = ['uid',i + '_cnt']
if gn.empty == True:
gn = tp
else:
gn = pd.merge(gn,tp,on = 'uid',how = 'left')
tp = pd.DataFrame(df.groupby('uid').apply(lambda df:np.where(df[i]>0,1,0).sum()).reset_index())
tp.columns = ['uid',i + '_num']
if gn.empty == True:
gn = tp
else:
gn = pd.merge(gn,tp,on = 'uid',how = 'left')
tp = pd.DataFrame(df.groupby('uid').apply(lambda df:np.nansum(df[i])).reset_index())
tp.columns = ['uid',i + '_tot']
if gn.empty == True:
gn = tp
else:
gn = pd.merge(gn,tp,on = 'uid',how = 'left')
tp = pd.DataFrame(df.groupby('uid').apply(lambda df:np.nanmean(df[i])).reset_index())
tp.columns = ['uid',i + '_avg']
if gn.empty == True:
gn = tp
else:
gn = pd.merge(gn,tp,on = 'uid',how = 'left')
tp = pd.DataFrame(df.groupby('uid').apply(lambda df:np.nanmax(df[i])).reset_index())
tp.columns = ['uid',i + '_max']
if gn.empty == True:
gn = tp
else:
gn = pd.merge(gn,tp,on = 'uid',how = 'left')
tp = pd.DataFrame(df.groupby('uid').apply(lambda df:np.nanmin(df[i])).reset_index())
tp.columns = ['uid',i + '_min']
if gn.empty == True:
gn = tp
else:
gn = pd.merge(gn,tp,on = 'uid',how = 'left')
tp = pd.DataFrame(df.groupby('uid').apply(lambda df:np.nanvar(df[i])).reset_index())
tp.columns = ['uid',i + '_var']
if gn.empty == True:
gn = tp
else:
gn = pd.merge(gn,tp,on = 'uid',how = 'left')
tp = pd.DataFrame(df.groupby('uid').apply(lambda df:np.nanmean(df[i])/max(np.nanvar(df[i]),1)).reset_index())
tp.columns = ['uid',i + '_bianyi']
if gn.empty == True:
gn = tp
else:
gn = pd.merge(gn,tp,on = 'uid',how = 'left')
Yes dstc_lst Variable seeking distinct Number
gc = pd.DataFrame()
for i in dstc_lst:
tp = pd.DataFrame(df.groupby('uid').apply(lambda df:len(set(df[i]))).reset_index())
tp.columns = ['uid',i + '_dstc']
if gc.empty == True:
gc = tp
else:
gc = pd.merge(gc,tp,on = 'uid',how = 'left')
Combine variables
fn = pd.merge(base,gn,on= 'uid')
fn = pd.merge(fn,gc,on= 'uid')
fn
fn = fn.fillna(0) # After combination, there will be many null values , Fill here with 0
Train with decision tree
x = fn.drop(['uid','oil_actv_dt','create_dt','bad_ind','class_new'],axis = 1)
y = fn.bad_ind.copy()
from sklearn import tree
dtree = tree.DecisionTreeRegressor(max_depth = 2,min_samples_leaf = 500,min_samples_split = 5000)
# Limit the maximum depth of the tree
# The leaf contains at least the number of samples
# The node must contain the number of training samples
dtree = dtree.fit(x,y) # utilize x Value , forecast y Whether it's a bad person
Output decision tree image , And make decisions
feature_names = x.columns
import matplotlib.pyplot as plt
plt.figure(figsize=(12,9),dpi=80)
tree.plot_tree(dtree,filled = True,feature_names = feature_names)
value Namely badrate
Recalculate the bad debt rate
sum(fn.bad_ind)/len(fn.bad_ind)
#0.04658077304261645
# Generation strategy
dff1 = fn.loc[(fn.amount_tot>48077.5)&(fn.coupon_amount_cnt>3.5)].copy()
dff1['level'] = 'oil_A'
dff2 = fn.loc[(fn.amount_tot>48077.5)&(fn.coupon_amount_cnt<=3.5)].copy()
dff2['level'] = 'oil_B'
dff3 = fn.loc[(fn.amount_tot<=48077.5)].copy()
dff3['level'] = 'oil_C'
dff1.head()
dff1 = dff1.append(dff2)
dff1 = dff1.append(dff3)
dff1 = dff1.reset_index(drop = True)
dff1.head()
last = dff1[['class_new','level','bad_ind','uid','oil_actv_dt','bad_ind']].copy()
last['oil_actv_dt'] = last['oil_actv_dt'] .apply(lambda x:str(x)[:7]).copy()
last.head(5)
边栏推荐
- 往图片添加椒盐噪声或高斯噪声
- What EDA companies are there in China?
- MFC BMP sets the resolution of bitmap, DPI is 600 points, and gdiplus generates labels
- Why does the data center need a set of infrastructure visual management system
- Bbox regression loss function in target detection -l2, smooth L1, IOU, giou, Diou, ciou, focal eiou, alpha IOU, Siou
- PTA ladder game exercise set l2-004 search tree judgment
- OpenSergo 即将发布 v1alpha1,丰富全链路异构架构的服务治理能力
- zabbix_ Get test database failed
- 产业金融3.0:“疏通血管”的金融科技
- Flask1.1.4 werkzeug1.0.1 source code analysis: start the process
猜你喜欢
随机推荐
Digital IC interview summary (interview experience sharing of large manufacturers)
往图片添加椒盐噪声或高斯噪声
Harmonyos practice - Introduction to development, analysis of atomized services
Message queue: how to deal with message backlog?
软件测试面试技巧
如果不知道这4种缓存模式,敢说懂缓存吗?
《ClickHouse原理解析与应用实践》读书笔记(6)
Red Hat安装内核头文件
Polynomial locus of order 5
Bbox regression loss function in target detection -l2, smooth L1, IOU, giou, Diou, ciou, focal eiou, alpha IOU, Siou
Flask1.1.4 Werkzeug1.0.1 源码分析:启动流程
[daily training -- Tencent selected 50] 235 Nearest common ancestor of binary search tree
[solved] record an error in easyexcel [when reading the XLS file, no error will be reported when reading the whole table, and an error will be reported when reading the specified sheet name]
zabbix_get测试数据库失败
zabbix_ Get test database failed
JVM the truth you need to know
Explication contextuelle du langage Go
Message queue: how to handle repeated messages?
async / await
Input of native applet switches between text and password types