当前位置：网站首页>随机森林、LGBM基于贝叶斯优化调参
随机森林、LGBM基于贝叶斯优化调参
2022-07-24 05:24:00 【羽星_s】
前言
- 本文基于孕妇吸烟与胎儿问题中数据集与前期处理
- 针对随机森林与LGBM模型网格搜索效率低，使用贝叶斯调参提高效率
- 有关于贝叶斯优化包相关参数说明详解可以看GitHub地址
- 将处理好的数据用
dill包进行封装，大家在尝试运行时，需要安装dill包 - 数据文件下载地址
数据导入
- 基于
jupyter notebook的魔术命令，如果不在jupyter notebook中运行，请将%号去掉
# 如果已经安装过dill包可以不要下面的魔术命令
%pip install dill
import dill
# 如果执行这步出错，请检查错误信息
# 是否有包未安装，比如bayes_opt包或lightgbm包
dill.load_session('C:/Users/lenovo/Desktop/data.pkl')
# 若未报错，则数据导入成功
安装贝叶斯优化包
- 基于
jupyter notebook的魔术命令，如果不在jupyter notebook中运行，请将%号去掉
%pip install bayesian-optimization
随机森林贝叶斯调参
导入包
# 贝叶斯调参优化随机森林
from sklearn.model_selection import cross_val_score
from bayes_opt import BayesianOptimization
构造黑盒函数
- 构造黑盒函数，即需要优化的目标，这里我选取的是10折交叉检验的分数值。当然你也可以在
RandomForestRegressor()后使用.fit(train_x, train_y).score(test_x, test_y)将模型在测试集上表现作为最优化目标 - 因为
bayes_opt库只支持最大值，所以最后的输出如果是求最小值，那么需要在前面加上负号，以转为最大值。这里使用neg_mean_squared_error作为最大化目标 - 由于
bayes优化只能优化连续超参数，因此要加上int()转为离散超参数
# 函数中包含需要调整的参数
def rf_cv(n_estimators, min_samples_split, max_features, max_depth, min_samples_leaf, max_leaf_nodes):
# 交叉检验，得到的评分为贝叶斯调参优化目标
val = cross_val_score(
# 由于bayes优化只能优化连续超参数，因此要加上int()转为离散超参数
RandomForestRegressor(n_estimators=int(n_estimators),
min_samples_split=int(min_samples_split),
min_samples_leaf = int(min_samples_leaf),
max_leaf_nodes = int(max_leaf_nodes),
max_features=int(max_features)
max_depth=int(max_depth),
random_state=42,
oob_score = 'True'),
X_train, y_train, scoring='neg_mean_squared_error', cv=10
).mean()
return val
确定域空间
- 确定参数搜索的范围，并打印迭代过程
# 规定各参数搜索范围
rf_bo = BayesianOptimization(rf_cv,
{
'n_estimators': (800, 1500),
'min_samples_split': (2, 20),
'max_features': (1, 6),
'max_depth': (3, 10),
'min_samples_leaf' : (2,20),
'max_leaf_nodes':(10,40)})
rf_bo.maximize()
输出：
| iter | target | max_depth | max_fe... | max_le... | min_sa... | min_sa... | n_esti... |
-------------------------------------------------------------------------------------------------
| 1 | -246.0 | 4.341 | 3.478 | 27.05 | 2.973 | 16.09 | 882.3 |
| 2 | -264.1 | 4.023 | 1.761 | 31.94 | 19.03 | 5.529 | 1.338e+0 |
| 3 | -245.4 | 5.505 | 4.8 | 23.88 | 4.304 | 12.89 | 1.147e+0 |
| 4 | -247.3 | 9.276 | 2.907 | 33.51 | 7.78 | 6.09 | 1.492e+0 |
| 5 | -257.1 | 6.966 | 1.406 | 25.3 | 14.24 | 11.24 | 925.4 |
| 6 | -245.3 | 5.946 | 4.281 | 26.73 | 5.724 | 13.45 | 1.147e+0 |
| 7 | -250.0 | 3.0 | 6.0 | 29.3 | 2.0 | 20.0 | 826.7 |
| 8 | -272.6 | 3.0 | 1.0 | 38.17 | 20.0 | 2.0 | 1.097e+0 |
| 9 | -248.6 | 8.698 | 5.645 | 23.75 | 2.0 | 20.0 | 1.182e+0 |
| 10 | -264.9 | 10.0 | 1.0 | 10.0 | 20.0 | 2.0 | 858.3 |
| 11 | -272.4 | 3.0 | 1.0 | 17.23 | 20.0 | 2.0 | 1.165e+0 |
| 12 | -247.0 | 4.871 | 5.969 | 31.59 | 2.493 | 16.71 | 1.134e+0 |
| 13 | -253.0 | 10.0 | 6.0 | 40.0 | 2.0 | 20.0 | 893.9 |
| 14 | -247.9 | 8.938 | 5.776 | 10.13 | 3.715 | 19.55 | 1.129e+0 |
| 15 | -246.0 | 8.643 | 4.938 | 16.8 | 12.07 | 17.0 | 1.48e+03 |
| 16 | -272.3 | 3.046 | 1.13 | 30.6 | 20.0 | 4.148 | 1.47e+03 |
| 17 | -247.3 | 10.0 | 5.957 | 19.54 | 4.089 | 17.66 | 1.495e+0 |
| 18 | -248.7 | 6.204 | 3.098 | 10.2 | 18.7 | 11.91 | 1.495e+0 |
| 19 | -249.2 | 3.0 | 5.068 | 39.19 | 2.0 | 20.0 | 865.9 |
| 20 | -248.0 | 6.247 | 5.639 | 30.41 | 16.99 | 19.59 | 1.499e+0 |
| 21 | -252.9 | 10.0 | 6.0 | 40.0 | 2.0 | 20.0 | 1.161e+0 |
| 22 | -249.2 | 10.0 | 6.0 | 22.95 | 19.86 | 20.0 | 1.136e+0 |
| 23 | -251.1 | 10.0 | 6.0 | 29.33 | 2.0 | 20.0 | 1.207e+0 |
| 24 | -265.7 | 10.0 | 1.0 | 10.0 | 2.0 | 2.0 | 1.487e+0 |
| 25 | -248.4 | 9.63 | 5.778 | 22.09 | 2.579 | 2.131 | 1.132e+0 |
| 26 | -250.4 | 3.485 | 5.665 | 34.65 | 19.73 | 17.61 | 881.4 |
| 27 | -248.1 | 3.569 | 4.336 | 11.09 | 7.701 | 17.22 | 892.6 |
| 28 | -272.5 | 3.0 | 1.0 | 22.71 | 2.0 | 2.0 | 892.6 |
| 29 | -248.5 | 6.826 | 2.856 | 17.2 | 10.92 | 19.96 | 879.0 |
| 30 | -247.6 | 6.9 | 4.121 | 39.17 | 3.153 | 19.32 | 1.49e+03 |
=================================================================================================
- 可以看到当迭代到30次时，就已经找到了较为理想的参数
最优参
# 使用max得到最优参
rf_bo.max
输出：
{
'target': -245.31420890294567,
'params': {
'max_depth': 5.946464811467048,
'max_features': 4.280789636350939,
'max_leaf_nodes': 26.726081689294052,
'min_samples_leaf': 5.724298965332711,
'min_samples_split': 13.447191575489041,
'n_estimators': 1147.3188404305388}}
- 使用最优参(记得该取整的要取整)再次训练模型，得到相对理想的模型。
使用最优参训练模型
# 基于sklearn封装的随机森林
from sklearn.ensemble import RandomForestRegressor
# 在RandomForestRegressor()括号中填入最优参
# 这里以上面我得到的最优参为例
rf_reg = RandomForestRegressor(max_depth = 5,
max_features = 4,
max_leaf_nodes = 26,
min_samples_leaf = 5,
min_samples_split = 13,
n_estimators = 1147)
# 在训练集上训练模型
rf_reg.fit(X_train, y_train)
# 查看模型在训练集上均方误差
print('train_mes : {:.3f}'.format(mean_squared_error(y_train, rf_reg.predict(X_train))))
LGBM贝叶斯调参
- 同样的，先构造黑箱函数，然后确定域空间，唯一不同的这次我们规定随机搜索的步数和贝叶斯优化的步数。这样会增加搜索的时间，但也许能比默认步数搜索到更好的参数
LGBM模型中L1、L2正则化等参数允许为浮点类型，所以不需要使用int()进行类型转换。
def lgbm_cv(n_estimators,max_depth,num_leaves,min_child_samples,reg_alpha,reg_lambda,subsample,colsample_bytree):
val = cross_val_score(
LGBMRegressor(learning_rate = 0.001,
n_estimators=int(n_estimators),
max_depth=int(max_depth),
num_leaves = int(num_leaves),
min_child_samples = int(min_child_samples),
reg_alpha = reg_alpha, # float
reg_lambda = reg_lambda,
subsample = subsample,
colsample_bytree = colsample_bytree),
X_train, y_train, scoring='neg_mean_squared_error', cv=10).mean()
return val
lgbm_bo = BayesianOptimization(lgbm_cv,
{
'n_estimators': (1500, 3000),
'max_depth': (2, 10),
'num_leaves': (5, 20),
'min_child_samples': (3, 100),
'reg_alpha' : (0.1,1),
'reg_lambda':(0.001,1),
'subsample':(0.8,1),
'colsample_bytree':(0.8,1)})
确定迭代次数
- 迭代次数由两部分组成，随机搜索的步数和贝叶斯优化的步数，贝叶斯优化的步数要多一点，步骤越多，就越有可能找到一个好的最大值。随机探索可以通过扩大探索空间而有所帮助。
# n_iter贝叶斯优化次数,init_points随机优化次数
lgbm_bo.maximize(n_iter = 100,init_points = 50)
输出：
| iter | target | colsam... | max_depth | min_ch... | n_esti... | num_le... | reg_alpha | reg_la... | subsample |
-------------------------------------------------------------------------------------------------------------------------
| 1 | -251.5 | 0.9299 | 6.878 | 82.45 | 2.147e+0 | 11.21 | 0.3167 | 0.2107 | 0.8781 |
| 2 | -252.4 | 0.8555 | 4.824 | 58.81 | 2.055e+0 | 11.71 | 0.7608 | 0.2855 | 0.8864 |
| 3 | -249.1 | 0.96 | 7.997 | 14.63 | 2.232e+0 | 19.55 | 0.1661 | 0.6375 | 0.8603 |
| 4 | -249.4 | 0.9018 | 8.336 | 20.44 | 2.155e+0 | 18.32 | 0.6761 | 0.1784 | 0.9618 |
| 5 | -249.9 | 0.8464 | 8.894 | 76.6 | 2.907e+0 | 15.3 | 0.1962 | 0.9202 | 0.9374 |
| 6 | -256.1 | 0.9289 | 9.386 | 70.46 | 1.546e+0 | 7.014 | 0.4163 | 0.9269 | 0.8498 |
| 7 | -250.1 | 0.8198 | 5.917 | 22.09 | 1.997e+0 | 8.863 | 0.2419 | 0.08677 | 0.8001 |
| 8 | -250.1 | 0.84 | 4.637 | 25.22 | 2.121e+0 | 13.66 | 0.7642 | 0.04577 | 0.9546 |
| 9 | -248.0 | 0.8454 | 8.498 | 15.58 | 2.894e+0 | 18.63 | 0.1802 | 0.4565 | 0.8858 |
| 10 | -255.2 | 0.8839 | 7.783 | 53.08 | 1.979e+0 | 5.731 | 0.4185 | 0.5306 | 0.9554 |
| 11 | -249.2 | 0.944 | 4.216 | 88.81 | 2.929e+0 | 10.36 | 0.5145 | 0.1131 | 0.9041 |
| 12 | -247.2 | 0.8847 | 5.88 | 12.0 | 2.059e+0 | 13.3 | 0.6635 | 0.09008 | 0.9531 |
| 13 | -250.2 | 0.8651 | 3.733 | 50.73 | 2.875e+0 | 16.17 | 0.542 | 0.847 | 0.9391 |
| 14 | -251.2 | 0.8654 | 8.812 | 92.12 | 2.522e+0 | 12.46 | 0.3778 | 0.6421 | 0.9782 |
| 15 | -251.3 | 0.9663 | 6.259 | 45.49 | 2.908e+0 | 17.26 | 0.3127 | 0.04984 | 0.9103 |
| 16 | -248.5 | 0.8887 | 6.157 | 79.92 | 2.83e+03 | 7.47 | 0.7573 | 0.1109 | 0.8294 |
| 17 | -254.0 | 0.9811 | 4.113 | 83.44 | 1.704e+0 | 8.561 | 0.787 | 0.1622 | 0.9373 |
| 18 | -250.7 | 0.9525 | 6.545 | 94.2 | 2.758e+0 | 8.907 | 0.6938 | 0.3916 | 0.9476 |
| 19 | -251.0 | 0.9972 | 6.941 | 46.37 | 2.577e+0 | 12.63 | 0.7774 | 0.9983 | 0.8211 |
| 20 | -246.4 | 0.8041 | 3.084 | 3.125 | 2.505e+0 | 17.59 | 0.4361 | 0.7433 | 0.9699 |
| 21 | -249.1 | 0.8621 | 8.701 | 65.99 | 2.938e+0 | 19.25 | 0.9019 | 0.2082 | 0.8993 |
| 22 | -249.5 | 0.9085 | 3.629 | 23.46 | 2.138e+0 | 12.78 | 0.99 | 0.6435 | 0.9545 |
| 23 | -259.9 | 0.8006 | 2.359 | 33.99 | 1.649e+0 | 16.87 | 0.9538 | 0.7214 | 0.8083 |
| 24 | -253.8 | 0.9543 | 9.966 | 76.27 | 1.817e+0 | 15.48 | 0.953 | 0.4465 | 0.8373 |
| 25 | -250.0 | 0.9024 | 9.932 | 87.61 | 2.563e+0 | 17.52 | 0.6365 | 0.3728 | 0.9358 |
| 26 | -247.7 | 0.9368 | 9.499 | 26.12 | 2.618e+0 | 6.258 | 0.5139 | 0.01136 | 0.8822 |
| 27 | -256.5 | 0.8403 | 2.212 | 72.82 | 1.975e+0 | 9.344 | 0.7336 | 0.9867 | 0.986 |
| 28 | -250.4 | 0.8356 | 6.36 | 25.71 | 2.362e+0 | 12.92 | 0.9792 | 0.1116 | 0.9806 |
| 29 | -250.4 | 0.967 | 6.722 | 72.78 | 2.598e+0 | 11.43 | 0.3715 | 0.7805 | 0.9167 |
| 30 | -253.8 | 0.9365 | 2.325 | 42.29 | 2.333e+0 | 6.774 | 0.7346 | 0.5393 | 0.9785 |
| 31 | -252.4 | 0.984 | 7.318 | 44.95 | 1.856e+0 | 11.53 | 0.364 | 0.06497 | 0.9884 |
| 32 | -255.7 | 0.895 | 4.489 | 41.63 | 1.858e+0 | 5.682 | 0.1547 | 0.7007 | 0.9882 |
| 33 | -248.9 | 0.9449 | 7.396 | 32.28 | 2.355e+0 | 14.05 | 0.8676 | 0.9371 | 0.8904 |
| 34 | -251.9 | 0.8728 | 7.131 | 58.31 | 2.399e+0 | 13.46 | 0.8022 | 0.9378 | 0.9732 |
| 35 | -252.0 | 0.859 | 5.741 | 43.33 | 1.914e+0 | 11.67 | 0.4837 | 0.1606 | 0.9066 |
| 36 | -256.2 | 0.9248 | 3.282 | 44.33 | 1.549e+0 | 17.83 | 0.2039 | 0.1553 | 0.8587 |
| 37 | -250.0 | 0.8103 | 5.894 | 43.89 | 2.872e+0 | 11.2 | 0.9715 | 0.1911 | 0.8768 |
| 38 | -247.4 | 0.9543 | 4.029 | 29.43 | 2.807e+0 | 18.67 | 0.6966 | 0.3294 | 0.8734 |
| 39 | -251.7 | 0.8035 | 9.248 | 49.61 | 2.935e+0 | 19.69 | 0.4769 | 0.7963 | 0.8527 |
| 40 | -253.8 | 0.8426 | 7.232 | 73.47 | 1.953e+0 | 9.829 | 0.4052 | 0.6666 | 0.8147 |
| 41 | -253.7 | 0.8878 | 9.484 | 77.37 | 1.981e+0 | 10.31 | 0.9634 | 0.37 | 0.8724 |
| 42 | -248.7 | 0.837 | 5.001 | 5.432 | 1.895e+0 | 10.73 | 0.4941 | 0.07088 | 0.9292 |
| 43 | -252.1 | 0.8235 | 2.448 | 9.134 | 2.106e+0 | 16.9 | 0.8737 | 0.3368 | 0.8999 |
| 44 | -252.3 | 0.927 | 7.326 | 91.65 | 2.058e+0 | 11.56 | 0.7419 | 0.4703 | 0.8103 |
| 45 | -250.4 | 0.9256 | 7.025 | 64.44 | 2.401e+0 | 16.9 | 0.107 | 0.7307 | 0.8434 |
| 46 | -248.8 | 0.9423 | 3.798 | 29.89 | 2.524e+0 | 12.89 | 0.7705 | 0.741 | 0.9162 |
| 47 | -258.6 | 0.9142 | 5.767 | 76.21 | 1.505e+0 | 6.561 | 0.2224 | 0.2147 | 0.9764 |
| 48 | -252.4 | 0.9271 | 8.811 | 6.559 | 1.796e+0 | 5.845 | 0.542 | 0.8878 | 0.9614 |
| 49 | -258.8 | 0.8216 | 7.332 | 98.04 | 1.61e+03 | 5.039 | 0.4765 | 0.09902 | 0.8323 |
| 50 | -251.2 | 0.8809 | 3.795 | 97.97 | 2.588e+0 | 5.977 | 0.9903 | 0.1333 | 0.8626 |
| 51 | -252.7 | 0.8713 | 3.811 | 55.85 | 2.138e+0 | 17.5 | 0.527 | 0.4825 | 0.8298 |
| 52 | -255.0 | 0.8587 | 9.695 | 56.51 | 1.657e+0 | 14.53 | 0.6904 | 0.6378 | 0.9473 |
| 53 | -254.0 | 0.971 | 2.733 | 91.54 | 2.319e+0 | 5.299 | 0.6559 | 0.9105 | 0.9984 |
| 54 | -252.5 | 0.915 | 9.869 | 92.64 | 2.222e+0 | 10.35 | 0.9431 | 0.4694 | 0.821 |
| 55 | -251.9 | 0.9004 | 7.716 | 73.5 | 2.288e+0 | 11.8 | 0.4452 | 0.4667 | 0.9693 |
| 56 | -251.1 | 0.9678 | 4.192 | 96.36 | 2.536e+0 | 14.14 | 0.1397 | 0.04324 | 0.8681 |
| 57 | -253.8 | 0.8579 | 4.479 | 72.29 | 1.916e+0 | 17.29 | 0.8099 | 0.4709 | 0.9209 |
| 58 | -251.7 | 0.8303 | 5.407 | 92.42 | 2.351e+0 | 9.32 | 0.1488 | 0.5252 | 0.8027 |
| 59 | -250.2 | 0.8626 | 9.889 | 37.49 | 2.543e+0 | 11.68 | 0.3817 | 0.1709 | 0.8627 |
| 60 | -250.7 | 0.9972 | 8.553 | 32.94 | 1.672e+0 | 15.26 | 0.3957 | 0.1086 | 0.8383 |
| 61 | -251.9 | 0.8949 | 5.338 | 26.2 | 1.942e+0 | 7.799 | 0.8794 | 0.1337 | 0.8964 |
| 62 | -246.9 | 0.8566 | 3.851 | 16.47 | 2.076e+0 | 18.73 | 0.5352 | 0.4207 | 0.9956 |
| 63 | -245.1 | 0.9937 | 4.547 | 3.305 | 2.867e+0 | 7.185 | 0.8233 | 0.4205 | 0.9417 |
| 64 | -250.5 | 0.9768 | 8.349 | 85.95 | 2.332e+0 | 14.78 | 0.3782 | 0.8485 | 0.9663 |
| 65 | -250.4 | 0.8401 | 9.422 | 32.77 | 2.428e+0 | 18.25 | 0.7231 | 0.7265 | 0.8794 |
| 66 | -247.4 | 0.9794 | 4.493 | 8.738 | 2.042e+0 | 16.57 | 0.6579 | 0.6725 | 0.844 |
| 67 | -248.6 | 0.946 | 4.873 | 34.24 | 2.044e+0 | 16.74 | 0.3182 | 0.2692 | 0.8474 |
| 68 | -254.3 | 0.8538 | 5.687 | 79.42 | 1.824e+0 | 18.33 | 0.7764 | 0.6676 | 0.8872 |
| 69 | -255.9 | 0.8359 | 8.635 | 76.64 | 1.688e+0 | 9.01 | 0.6837 | 0.9395 | 0.9813 |
| 70 | -252.5 | 0.9696 | 9.777 | 42.32 | 2.846e+0 | 14.44 | 0.4389 | 0.1839 | 0.9697 |
| 71 | -251.0 | 0.8293 | 7.864 | 97.97 | 2.745e+0 | 15.07 | 0.4306 | 0.3661 | 0.8145 |
| 72 | -254.7 | 0.844 | 2.044 | 82.31 | 2.124e+0 | 12.8 | 0.9643 | 0.9277 | 0.8329 |
| 73 | -248.6 | 0.9647 | 4.654 | 79.34 | 2.879e+0 | 14.25 | 0.3899 | 0.4517 | 0.8848 |
| 74 | -253.5 | 0.9759 | 9.409 | 34.56 | 1.831e+0 | 6.315 | 0.2175 | 0.9694 | 0.9046 |
| 75 | -249.8 | 0.9993 | 3.751 | 53.76 | 2.705e+0 | 18.52 | 0.9932 | 0.3771 | 0.9974 |
| 76 | -248.2 | 0.8568 | 7.918 | 26.47 | 2.832e+0 | 9.567 | 0.3081 | 0.2001 | 0.9637 |
| 77 | -250.6 | 0.9161 | 9.572 | 21.78 | 2.08e+03 | 17.8 | 0.659 | 0.2496 | 0.8105 |
| 78 | -256.7 | 0.8722 | 5.862 | 96.38 | 1.661e+0 | 17.3 | 0.8781 | 0.3713 | 0.8406 |
| 79 | -249.6 | 0.9668 | 3.886 | 67.11 | 2.591e+0 | 6.384 | 0.2177 | 0.1065 | 0.9837 |
| 80 | -249.0 | 0.8911 | 5.993 | 29.05 | 2.946e+0 | 13.13 | 0.902 | 0.1512 | 0.9985 |
| 81 | -249.2 | 0.8635 | 3.71 | 43.96 | 2.767e+0 | 18.02 | 0.9057 | 0.6716 | 0.8505 |
| 82 | -253.0 | 0.9688 | 3.429 | 79.2 | 2.066e+0 | 6.202 | 0.9512 | 0.6868 | 0.8608 |
| 83 | -250.9 | 0.952 | 4.57 | 75.44 | 2.293e+0 | 17.13 | 0.6878 | 0.9675 | 0.8407 |
| 84 | -254.5 | 0.9119 | 3.122 | 82.4 | 1.808e+0 | 12.66 | 0.9125 | 0.5803 | 0.9666 |
| 85 | -257.6 | 0.9048 | 3.93 | 95.5 | 1.624e+0 | 10.92 | 0.6581 | 0.2187 | 0.9817 |
| 86 | -257.4 | 0.8564 | 2.41 | 22.95 | 1.807e+0 | 8.449 | 0.3729 | 0.761 | 0.8282 |
| 87 | -248.5 | 0.8595 | 4.704 | 5.026 | 1.914e+0 | 17.77 | 0.5784 | 0.7281 | 0.8922 |
| 88 | -248.6 | 0.9925 | 2.062 | 13.15 | 2.681e+0 | 12.09 | 0.1668 | 0.6771 | 0.8767 |
| 89 | -247.2 | 0.9495 | 8.553 | 6.885 | 2.643e+0 | 14.56 | 0.8508 | 0.2082 | 0.9777 |
| 90 | -249.9 | 0.8889 | 5.049 | 75.7 | 2.95e+03 | 19.67 | 0.5517 | 0.6659 | 0.9488 |
| 91 | -252.1 | 0.8089 | 5.886 | 56.19 | 2.138e+0 | 7.085 | 0.1107 | 0.08426 | 0.8332 |
| 92 | -250.7 | 0.8242 | 8.274 | 80.49 | 2.32e+03 | 6.095 | 0.5902 | 0.6058 | 0.9501 |
| 93 | -257.2 | 0.9934 | 2.024 | 65.59 | 1.808e+0 | 8.934 | 0.1617 | 0.9746 | 0.8435 |
| 94 | -250.5 | 0.8985 | 2.545 | 84.7 | 2.795e+0 | 19.0 | 0.1595 | 0.3708 | 0.8454 |
| 95 | -250.3 | 0.8932 | 8.764 | 47.66 | 2.538e+0 | 19.69 | 0.5193 | 0.4982 | 0.8129 |
| 96 | -251.0 | 0.9709 | 7.095 | 48.09 | 2.861e+0 | 11.9 | 0.3469 | 0.7833 | 0.9071 |
| 97 | -252.4 | 0.9348 | 8.922 | 77.9 | 2.059e+0 | 15.69 | 0.8596 | 0.5381 | 0.9972 |
| 98 | -253.6 | 0.9101 | 2.922 | 11.67 | 1.967e+0 | 15.83 | 0.9068 | 0.7492 | 0.8623 |
| 99 | -250.2 | 0.8583 | 8.64 | 89.44 | 2.598e+0 | 18.2 | 0.1822 | 0.4276 | 0.8993 |
| 100 | -246.6 | 0.8264 | 5.264 | 10.81 | 2.47e+03 | 7.567 | 0.6361 | 0.3497 | 0.8002 |
| 101 | -250.1 | 0.8995 | 3.704 | 52.65 | 2.985e+0 | 19.72 | 0.11 | 0.1371 | 0.8817 |
| 102 | -248.8 | 0.8594 | 5.012 | 4.972 | 2.131e+0 | 8.508 | 0.3815 | 0.945 | 0.9192 |
| 103 | -247.6 | 0.9669 | 3.601 | 29.7 | 2.932e+0 | 18.49 | 0.3248 | 0.7928 | 0.8384 |
| 104 | -256.3 | 0.9432 | 9.293 | 54.1 | 1.761e+0 | 5.085 | 0.3883 | 0.5722 | 0.9298 |
| 105 | -257.2 | 0.9938 | 2.443 | 84.29 | 1.823e+0 | 18.0 | 0.3708 | 0.7673 | 0.9651 |
| 106 | -252.1 | 0.9587 | 8.62 | 42.49 | 2.116e+0 | 19.23 | 0.7527 | 0.4659 | 0.9647 |
| 107 | -253.4 | 0.8556 | 9.788 | 62.2 | 1.972e+0 | 19.38 | 0.9473 | 0.1454 | 0.8905 |
| 108 | -253.2 | 0.8099 | 7.696 | 50.18 | 2.162e+0 | 15.46 | 0.6554 | 0.827 | 0.8181 |
| 109 | -251.1 | 0.8175 | 8.761 | 13.57 | 1.69e+03 | 8.422 | 0.8813 | 0.9929 | 0.9977 |
| 110 | -251.1 | 0.8626 | 6.849 | 92.65 | 2.571e+0 | 8.158 | 0.758 | 0.8469 | 0.9977 |
| 111 | -252.6 | 0.8674 | 7.393 | 27.39 | 1.724e+0 | 18.88 | 0.7886 | 0.4438 | 0.8879 |
| 112 | -251.1 | 0.9263 | 4.146 | 78.4 | 2.257e+0 | 14.98 | 0.2221 | 0.145 | 0.9518 |
| 113 | -247.5 | 0.8534 | 6.048 | 10.58 | 2.222e+0 | 8.573 | 0.4608 | 0.3796 | 0.99 |
| 114 | -246.1 | 0.8489 | 3.597 | 6.778 | 2.394e+0 | 9.335 | 0.5718 | 0.1133 | 0.8493 |
| 115 | -250.9 | 0.9191 | 7.406 | 27.09 | 2.777e+0 | 19.52 | 0.6142 | 0.7267 | 0.8041 |
| 116 | -248.9 | 0.8929 | 7.28 | 38.2 | 2.463e+0 | 8.195 | 0.4663 | 0.3756 | 0.8455 |
| 117 | -252.0 | 0.9764 | 8.82 | 12.13 | 1.569e+0 | 7.788 | 0.7776 | 0.3066 | 0.8528 |
| 118 | -254.7 | 0.9845 | 6.606 | 88.3 | 1.692e+0 | 15.52 | 0.1237 | 0.8686 | 0.9593 |
| 119 | -249.9 | 0.9242 | 4.519 | 52.74 | 2.537e+0 | 19.79 | 0.4742 | 0.5048 | 0.9595 |
| 120 | -254.5 | 0.8294 | 4.186 | 32.16 | 1.826e+0 | 6.485 | 0.4382 | 0.6719 | 0.9082 |
| 121 | -257.4 | 0.8519 | 7.839 | 87.22 | 1.519e+0 | 13.29 | 0.3876 | 0.01006 | 0.9211 |
| 122 | -253.1 | 0.8557 | 4.863 | 99.88 | 2.196e+0 | 6.027 | 0.2864 | 0.08695 | 0.8477 |
| 123 | -255.3 | 0.9123 | 3.487 | 58.82 | 1.743e+0 | 16.01 | 0.981 | 0.2148 | 0.9191 |
| 124 | -252.9 | 0.8671 | 6.105 | 34.59 | 1.822e+0 | 7.959 | 0.5157 | 0.4601 | 0.9288 |
| 125 | -251.6 | 0.823 | 4.232 | 4.69 | 1.63e+03 | 17.58 | 0.3137 | 0.896 | 0.8394 |
| 126 | -251.1 | 0.9605 | 9.27 | 3.885 | 1.594e+0 | 9.155 | 0.3988 | 0.04495 | 0.8383 |
| 127 | -251.1 | 0.8553 | 8.43 | 92.19 | 2.549e+0 | 15.68 | 0.4312 | 0.3145 | 0.8269 |
| 128 | -253.7 | 0.8167 | 6.511 | 63.97 | 1.761e+0 | 18.79 | 0.4067 | 0.4545 | 0.8913 |
| 129 | -250.5 | 0.9509 | 3.603 | 86.18 | 2.351e+0 | 6.903 | 0.858 | 0.1375 | 0.887 |
| 130 | -251.3 | 0.8577 | 7.442 | 58.83 | 2.718e+0 | 18.86 | 0.3148 | 0.3542 | 0.8545 |
| 131 | -252.5 | 0.8522 | 5.962 | 44.5 | 1.682e+0 | 17.12 | 0.1616 | 0.242 | 0.8918 |
| 132 | -256.4 | 0.9798 | 4.611 | 95.35 | 1.608e+0 | 9.765 | 0.6692 | 0.7236 | 0.8428 |
| 133 | -251.3 | 0.9747 | 8.909 | 39.93 | 2.322e+0 | 13.09 | 0.7278 | 0.2057 | 0.9541 |
| 134 | -258.2 | 0.898 | 2.558 | 77.46 | 1.824e+0 | 17.6 | 0.184 | 0.882 | 0.8166 |
| 135 | -250.7 | 0.9765 | 8.294 | 36.04 | 1.986e+0 | 15.15 | 0.1793 | 0.0547 | 0.8536 |
| 136 | -256.1 | 0.8067 | 8.756 | 78.74 | 1.653e+0 | 9.253 | 0.4187 | 0.356 | 0.8482 |
| 137 | -255.0 | 0.9928 | 6.42 | 93.78 | 1.784e+0 | 18.29 | 0.3528 | 0.7969 | 0.9533 |
| 138 | -250.7 | 0.9377 | 9.27 | 93.17 | 2.997e+0 | 13.43 | 0.2546 | 0.3083 | 0.8694 |
| 139 | -259.6 | 0.8726 | 8.204 | 78.49 | 1.566e+0 | 5.512 | 0.6164 | 0.7091 | 0.8871 |
| 140 | -253.0 | 0.9226 | 2.858 | 49.18 | 2.565e+0 | 14.22 | 0.7601 | 0.5902 | 0.8829 |
| 141 | -255.2 | 0.8894 | 2.576 | 54.63 | 2.112e+0 | 16.73 | 0.9314 | 0.9692 | 0.9501 |
| 142 | -251.9 | 0.8662 | 2.44 | 42.66 | 2.677e+0 | 7.367 | 0.5093 | 0.6031 | 0.8568 |
| 143 | -246.4 | 0.8408 | 5.624 | 9.557 | 2.655e+0 | 17.48 | 0.6699 | 0.01308 | 0.9373 |
| 144 | -256.3 | 0.8831 | 4.472 | 72.16 | 1.638e+0 | 8.795 | 0.748 | 0.7101 | 0.8074 |
| 145 | -254.3 | 0.9842 | 5.52 | 54.41 | 1.66e+03 | 10.72 | 0.8502 | 0.954 | 0.8083 |
| 146 | -250.6 | 0.977 | 5.445 | 60.33 | 2.612e+0 | 18.05 | 0.5285 | 0.458 | 0.9856 |
| 147 | -249.9 | 0.9103 | 5.405 | 91.39 | 2.967e+0 | 12.09 | 0.6206 | 0.8223 | 0.812 |
| 148 | -252.6 | 0.9774 | 4.257 | 42.9 | 1.731e+0 | 15.42 | 0.9746 | 0.8842 | 0.8383 |
| 149 | -254.0 | 0.858 | 2.464 | 43.46 | 2.257e+0 | 13.9 | 0.3589 | 0.9985 | 0.8966 |
| 150 | -253.4 | 0.9733 | 9.152 | 55.79 | 2.118e+0 | 5.46 | 0.8561 | 0.062 | 0.8039 |
输出最优参
lgbm_bo.max
输出：
{
'target': -243.40837561015314,
'params': {
'colsample_bytree': 0.9035439141128341,
'max_depth': 3.167260637404743,
'min_child_samples': 9.324254670380586,
'n_estimators': 2816.120214269747,
'num_leaves': 17.035319354824765,
'reg_alpha': 0.33257959049618924,
'reg_lambda': 0.5556596408409464,
'subsample': 0.8898942099929366}}
- 使用最优参(记得该取整的要取整)再次训练模型，得到相对理想的模型。
写在最后
- 如果大家想更好的使用贝叶斯优化包，可以读一读优化包的GitHub说明，里面有基于经验范围的精密搜索、经验函数等一些参数的调整等，或许能提升优化器性能。
边栏推荐
- Sword finger offer jz10 Fibonacci sequence
- Restful API introduction
- Use of MySQL
- Write blog at leisure ~ briefly talk about let, VaR and Const
- Windows下Mysql5.7忘记root密码解决方法
- Several common problems of SQL server synchronization database without public IP across network segments
- Why can't index be the key of V-for?
- 【波形/信号发生器】基于 STC1524K32S4 for C on Keil
- object-oriented
- sql server 同步数据库 跨网段无公网ip几个常见小问题问题
猜你喜欢
Unable to boot after permanent mounting
Why can't index be the key of V-for?
Leetcode does not add, subtract, multiply, divide, and calculate the number of 1 in binary
【LVGL(5)】标签的(label)用法
Go environment construction and start
手动安装Apache
【LVGL(4)】对象的事件及事件冒泡
【LVGL】组件的样式的设置、更改、删除API函数
Data set and pre training model
Leetcode sword finger offer jz23: the entry node of the link in the linked list
随机推荐
【LVGL】【阶段总结1】
Customize MVC 3.0
NFS共享服务及实验
Leetcode sword finger offer JZ9 dual stack implementation queue
CentOS operating system security reinforcement
服务器硬件及RAID配置实战
ESP32超详细学习记录：NTP同步时间
Rsync (I): basic commands and usage
自定义zabbix agent rpm包
Backup MySQL database with bat script under Windows
Sword finger offer jz10 Fibonacci sequence
CentOS操作系统安全加固
迭代器与生成器
SSH Remote Access and control
MySQL Index & execution plan
Several common problems of SQL server synchronization database without public IP across network segments
Homework in the second week
Explain the event cycle mechanism and differences between browser and node in detail
Crud of MySQL
JS: why [] = =! [] return true?