超参数调优进阶：Optuna/Bayesian/Early Stopping

超参数调优进阶Optuna/Bayesian/Early Stopping1. 调优方法对比超参数调优方法 ├── 网格搜索Grid Search穷举所有组合慢但全面 ├── 随机搜索Random Search随机采样快但不保证最优 ├── 贝叶斯优化Bayesian基于历史结果智能搜索 └── 早停法Early Stopping训练中动态停止2. Optuna 调优importoptunafromsklearn.ensembleimportRandomForestClassifierfromsklearn.model_selectionimportcross_val_scoredefobjective(trial):params{n_estimators:trial.suggest_int(n_estimators,50,300),max_depth:trial.suggest_int(max_depth,3,15),min_samples_split:trial.suggest_int(min_samples_split,2,20),min_samples_leaf:trial.suggest_int(min_samples_leaf,1,10),max_features:trial.suggest_categorical(max_features,[sqrt,log2,None]),}modelRandomForestClassifier(**params,random_state42)scorescross_val_score(model,X_train,y_train,cv5,scoringaccuracy)returnscores.mean()studyoptuna.create_study(directionmaximize)study.optimize(objective,n_trials100,show_progress_barTrue)print(f最佳参数:{study.best_params})print(f最佳分数:{study.best_value:.4f})3. XGBoost Optunaimportoptunaimportxgboostasxgbdefobjective_xgb(trial):params{n_estimators:trial.suggest_int(n_estimators,50,500),max_depth:trial.suggest_int(max_depth,3,12),learning_rate:trial.suggest_float(learning_rate,0.01,0.3,logTrue),subsample:trial.suggest_float(subsample,0.6,1.0),colsample_bytree:trial.suggest_float(colsample_bytree,0.6,1.0),reg_alpha:trial.suggest_float(reg_alpha,1e-8,10.0,logTrue),reg_lambda:trial.suggest_float(reg_lambda,1e-8,10.0,logTrue),}modelxgb.XGBClassifier(**params,random_state42,use_label_encoderFalse)scorescross_val_score(model,X_train,y_train,cv5,scoringaccuracy)returnscores.mean()studyoptuna.create_study(directionmaximize)study.optimize(objective_xgb,n_trials200)4. Early Stoppingimportlightgbmaslgb train_datalgb.Dataset(X_train,labely_train)val_datalgb.Dataset(X_val,labely_val,referencetrain_data)params{objective:binary,metric:binary_logloss,learning_rate:0.05,num_leaves:31,}callbacks[lgb.early_stopping(stopping_rounds50),lgb.log_evaluation(period10),]modellgb.train(params,train_data,valid_sets[val_data],num_boost_round1000,callbackscallbacks,)总结方法速度精度推荐场景Grid Search慢高小参数空间Random Search快中快速探索Optuna中高复杂参数空间Early Stopping快高训练中使用