Release 1.1.1

New features

• Support building for Linux on aarch64 from sources using CMake (no prebuilt binaries or PyPI packages yet). #1981
• [C/C++ applier] Support embedding features. #2172
• [C/C++ applier] Add GetModelUsedFeaturesNames. #2204
• [Python] Add text features to utils.create_cd. #2193
• [Spark] Full support for Apache Spark 3.3
• [Spark] Read/write PySpark's DataFrame-like API for Pool. #2030
• [Spark] Allow to specify trainingDriver and worker listening ports. #2181

Bugfixes

• Fix prediction dimension check for RMSEWithUncertainty and MultiQuantile. #2155
• [C/C++ applier] Fix segmentation fault in prediction for multiple objects for multiple dimension models.
• [JVM applier] Fix catboost-common dependency version in catboost-prediction (Fixes JVM applier on macOS). #2121
• [Python] Update for pandas 1.5.0: iteritems -> items (Fixes annoying deprecation warning). #2179
• [Python] Fix segmentation fault when target is np.ndarray with dtype=object. #2201
• [Python] Fix specifying feature_names in utils.create_cd. #2211

Release 1.1

New features

• Multiquantile regression

  Now it's possible to train models with shared tree structure and multiple predicted quantile values in each leaf. Currently this approach doesn't give a strong guarantee for predicted quantile values consistency, but it still provides more consistency than training multiple independent models for each quantile. You can read short description in the documentation. Short example for Python: loss_function='MultiQuantile:alpha=0.2,0.4'. Supported only on CPU for now.

• Support text and embedding features for regression and ranking.

• Spark: Read/write Spark's Dataset-like API for Pool. #2030

• Support HashedCateg column type. This allows to use externally prehashed categorical features both in training and prediction.

• New option plot_file in Python functions with plot parameter allows to save plots to file. #758

• Add eval_fraction parameter. #1500

• Non-symmetric trees model summation.

• init_model parameter now works with non-symmetric trees.

• Partial support for Apache Spark 3.3 (only for Scala 2.12 and without PySpark).

Speedups

• 2x speedup DCG, nDCG and FilteredDCG metrics calculation for groups with >= 50 objects and with top=-1 (all objects from each group, default value)
• Fixed 2x slowdown of PairLogit and other ranking losses on CPU introduced in release 0.23

Bugfixes

• Fix for pandas integer array. #2096
• Save feature names to json format. #2102
• Fix feature weights on CPU
• Use feature weights on GPU
• Fix gradient calculation for QueryRMSE on GPU
• Fix ranking metrics with group weights in calc_metrics
• Fix JVM applier on data with text features. #2132

Release 1.0.6

New features

• Fixed splits for binary features on gpu for non-symmetric trees -- specify the set of splits to start each tree in the model with --fixed-binary-splits or fixed_binary_splits in Python package (by default, there are no fixed splits)

Documentation

• New sections on MultiRMSEWithMissingValues and LogCosh
• New section on get_embedding_feature_indices
• Add info on gpu support for metrics

Bug-fixes

• Fix warning about resetting logger when logging to sys.stdout & sys.stderr from different threads #1855
• Fix model summation in CatBoost for Apache Spark
• Fix performance and scalability of query auc for ranking (1m samples, query size 2, 8 cpu cores 0.55s -> 0.04s)
• Fix support for text features and embeddings in Java applier #2043
• Fix nan/inf split scores with yeti rank pairwise loss
• Fix nan/inf feature strengths in pair logit on cpu

Release 1.0.5

New features

• Support Apple Darwin arm64 architecture. #1526.
• Support feature tags in feature selection.
• Support for Apache Spark 3.2.
• Model sum in Apache Spark.

Python package

• Accommodate multiple target-platform arguments used to build universal binaries.
• Add grid creation function to utils.py
• Custom multilabel eval metrics by @ELitvinova
• Metrics plotter by @evgenabramov
• Fbeta score by @ELitvinova

Bugfixes

• Fix group weights in metrics calculation.
• Fix fit for PySpark estimators. #1976.
• Fix predict on GPU. #1901, #1923.
• Disable exact leafs calculation for MAE, MAPE, Quantile on GPU.
• Fix counter description for plotting. #1973.
• Allow weights in BrierScore. #1967.
• Disable AUC calculation for learn by default on GPU as well.
• Fix plot_tree example in documentation.
• Fix plots in cv.
• Fix ui32 overflows in pairwise losses on GPU.
• Fix for multiclass in nodejs evaluator. #1903.
• Fix CatBoost R package installation on Monterey. #1912.
• Fix CUDA error 700 caused by data race in mimalloc and CUDA driver.
• Fix slow compilation with CUDA 11.2+.
• Fix 2nd derivative in RMSEWithUncertainty.

New features

• Add sort param to FilteredDCG metric.
• Add StochasticRank for FilteredDCG.

Python package

• add is_max/minimizable methods. #1915
• Support custom metric in select_features #1920

R package

• Register functions from libcatboostr natively in R, removing one of CRAN notes.

Bugfixes

• Fix apply for models without main loss_function.
• Fix text calcer options specification. #1916
• Fix calc_feature_statistics
• Fix Multi-approx support in CLI calc_metrics mode.
• Fix processing for text options. #1930
• Fix snapshot saving in feature selection.
• Fix CatBoost models serialization inside pipeline models in PySpark. #1936

CatBoost for Apache Spark

• Fix incorrect Linux so files in deployed Maven artifacts for release 1.0.2 (no code changes)

CatBoost for Apache Spark

• PySpark: Fix python -> JVM datetime.timedelta conversion.
• Fix: proper handling of constant categorical features. #1867
• Fix SIGSEGV for for Multiclassification with Ctrs. #1886

New features

• Add is_min_optimal, is_max_optimal for BuiltinMetrics. #1890

R package

• Use libcatboostr-darwin.dylib instead of libcatboostr-darwin.so on macOS. #1834

Bugfixes

• Fix CatBoostError: (No such file or directory) bad new file name when using grid_search. #1893

:warning: PySpark support is broken in this release.. Please use release 1.0.3 instead.

CatBoost for Apache Spark

• More robust handling of CatBoost Master and Workers failures, avoid freezes.
• Fix for empty partitions. #1687
• Fix use-after-free. #1759 and other random errors.
• Support Spark 3.1.

Python package

• Support python 3.10. #1575

Breaking changes

• Use group weight for generated pairs in pairwise losses

Bugfixes

• Switch to mimalloc allocator on Linux and macOS to avoid problems with static TLS.
• Fix SEGFAULTs on macOS. #1877
• Fix: Distributed training: do not fail if worker contains only learn or test data
• Fix SEGFAULT on CPU with Depthwise training and rsm < 1.
• Fix calc_feature_statistics for cat features. #1882
• Fix result of Cross Validation if metric_period has been specified
• fix eval_metric for Multitarget training

In this release, we decided to increment the major version as we think that CatBoost is pretty stable and production-ready. We know, that CatBoost is used a lot in many different companies and individual projects, and we think, that all the features we added in the last year are worth incrementing major version. And of course, as many programmers, we love the magic of binary numbers and we want to celebrate 100₂ anniversary since CatBoost first release on Github 🥳

New losses

• We've implemented a multi-label multiclass loss function, that allows us to predict multiple labels for each object #1420
• Added LogCosh loss implementation #844

Fully distributed CatBoost for Apache Spark

• In this release Apache Spark package became truly distributed - in the previous version CatBoost stored test datasets in controller process memory. And now test datasets are split evenly by workers.

Major speedup on CPU

We've improved training speed on numeric datasets:

• 28% speedup on Higgs dataset: 1000 trees, binclass: on 16 cores Intel CPU: 405 seconds -> 315 seconds
• 20% speedup on the small numeric dataset with 480K rows, 60 features, 100 trees, binclass on 16 cores Intel CPU 3.7 seconds-> 2.9 seconds
• 53% speedup on sparse one-hot encoded airlines dataset: 1000 trees training time 381 seconds -> 249 seconds

R package

• Update C++ handles by reference to avoid redundant copies by @david-cortes
• Avoid calculating groupwise feature importance: do not calculate feature importance for groupwise metrics by default
• R tests clear environment after runs so they won't find temporary data from previous runs
• Fixed ignored features in R fail when single feature was ignored
• Fix feature_count attribute with ignored_features

CV improvements

• Added support for text features and embeddings in cross-validation mode
• We've changed the way cross-validation works - previously, CatBoost was training a small batch of trees on each fold and then switched to the next fold or next batch of trees. In 1.0.0 we changed this behavior and now CatBoost trains the full model on each fold. That allows us to reduce the memory and time overhead of starting a new batch - only one CPU to GPU memory copy is needed per fold, not per each batch of trees. Mean metric interactive plot became unavailable until the end of training on all folds.
• Important change From now on use_best_model and early stopping works independently on each fold, as we are trying to make single fold training as close to regular training as possible. If one model stops at iteration i we use the last value of metric in the mean score plot for points with [i+1; last iteration).

GPU improvements

• Fixed distributed training performance on Ethernet networks ~2x training time speedup. For 2 hosts, 8 v100/host, 10gigabit eth, 300 factors, 150m samples, 200 trees, 3300s -> 1700s
• We've found a bug in model-size-reg implementation in GPU that leaded to worse quality of the resulting model, especially in comparison to a model trained on CPU with equal parameters

Rust

• Enabled load model from the buffer for rust by @manavsah

Bugfixes

• Fix for model predictions with text and embedding features
• Switch to TBB local executor to limit TLS size and avoid memory leakage #1835
• Switch to tcmalloc under Linux x86_64 to avoid memory fragmentation bug in LFAlloc
• Fix for case of ignored text feature
• Fixed application of baseline in C++ code. Moved addition of that before application of activation functions and determining labels of objects.
• Fixes for scikit-learn compatibility validation #1783 and #1785
• Fix for thread_count = -1 in set_params(). Issue #1800
• Fix potential sigsegv in the model evaluator. Fixes #1809
• Fix slow (u)int8 & (u)int16 parsing as catfeatures. Fixes #718
• Adjust boost from average option before auto-learning rate
• Fix embeddings with CrossEntropy mode #1654
• Fix object importance #1820
• Fix data provider without target #1827

R package

• Supported text features in R package, thanks to @glemhel!
• Supported virtual Ensembles in R, thanks to @glemhel!

New features

• Thank @gmrandazzo for adding multiregression with missing values on targets - MultiRMSEWithMissingValues loss function
• Supported multiclass prediction in C++ wrapper for model inference C API

Bugfixes

• Renamed keyword parameter in predict_proba function from X to data, fixes #1785
• R feature importances: remove pool argument, fix #1438 and #1772
• Fix CUDA training on Windows, multiple issues. main issue with details #1735
• Issue #1728: don't dereference pointers when there is no features
• Fixed empty tree processing in feature strength calculation
• Fixed missing loss graph points in select_features, #1775
• Sort csr matrix indices, fixes #1749
• Fix error "active CatBoost worker is already present in the current process" after previous training interruption or failure. #1795.
• Fixed erroneous warnings from models validation after training with custom loss or custom error function. Fixes #873 Fixes #1169

New features

• #972. Add model evaluation on GPU. Thanks to @rakalexandra.
• Support Langevin on GPU
• Save class labels to models in cross validation
• #1524. Return models after CV. Thanks to @vklyukin
• [Python] #766. Add CatBoostRanker & pool.get_group_id_hash() for ranking. Thanks to @AnnaAraslanova
• #262. Make CatBoost widget work in jupyter lab. Thanks to @Dm17r1y
• [GPU only] Allow to add exponent to score aggregation function
• Allow to specify threshold parameter for binary classification model. Thanks to @Keksozavr.
• [C Model API] #503. Allow to specify prediction type.
• [C Model API] #1201. Get predictions for a specific class.

Breaking changes

• #1628. Use CUDA 11 by default. CatBoost GPU now requires Linux x86_64 Driver Version >= 450.51.06 Windows x86_64 Driver Version >= 451.82.

Losses and metrics

• Add MRR and ERR metrics on CPU.
• Add LambdaMart loss.
• #1557. Add survivalAFT base logic. Thanks to @blatr.
• #1286. Add Cox Proportional Hazards Loss. Thanks to @fibersel.
• #1595. Provide object-oriented interface for setting up metric parameters. Thanks to @ks-korovina.
• Change default YetiRank decay to 0.85 for better quality.

Python package

• #1372. Custom logging stream in python package. Thanks to @DianaArapova.
• #1304. Callback after iteration functionality. Thanks to @qoter.

R package

• #251. Train parameter synonyms. Thanks to @ebalukova.
• #252. Add eval_metrics. Thanks to @ebalukova.

Speedups

• [Python] Speed up custom metrics and objectives with numba (if available)
• [Python] #1710. Large speedup for cv dataset splitting by sklearn splitter

Other

• Use Exact leaves estimation method as default on GPU
• [Spark] #1632. Update version of Scala 2.11 for security reasons.
• [Python] #1695. Explicitly specify WHEEL 'Root-Is-Purelib' value

Bugfixes

• Fix default projection dimension for embeddings
• Fix use_weights for some eval_metrics on GPU - use_weights=False is always respected now
• [Spark] #1649. The earlyStoppingRounds parameter is not recognized
• [Spark] #1650. Error when using the autoClassWeights parameter
• [Spark] #1651. Error about "Auto-stop PValue" when using odType "Iter" and odWait
• Fix usage of pairlogit weights for CPU fallback metrics when training on GPU

Speedup

• Now CatBoost uses non-owning Numpy arrays for passing c++ data to user-defined metric and loss functions in Python. This opens lot's of speedup probabilities: using those vectors in numba.jitted code, in cython code or just using numpy vector functions. Thanks @micyril!

Bugfixes

• Fix #1620 - retrieval of R pointers by @david-cortes
• Fix EvalMetricsResult.get_metric() by @Roffild
• Fix multiclass AUC calculation #1615

CatBoost for Apache Spark

This release includes CatBoost for Apache Spark package that supports training, model application and feature evaluation on Apache Spark platform. We've prepared CatBoost for Apache Spark introduction and CatBoost for Apache Spark Architecture videos for introduction. More details available at CatBoost for Apache Spark home page.

Feature selection

CatBoost supports recursive feature elimination procedure - when you have lot's of feature candidates and you want to select only most influential features by training models and selecting only strongest by feature importance. You can look for details in our tutorial

New features

• Supported exact leaves estimation method for quantile, MAE and MAPE losses on GPU. You can enable it by setting leaf_estimation_method=Exact explicitly, in next releases we are planning to set it by default.
• Supported uncertainty prediction for multiclassification models
• #1568 Added support shap values calculation MultiRMSE models
• #1520 Added support for pathlib.Path in python package
• #1456 Added prehashed categorical features and text features to C API for model inference.

Losses and metrics

• Supported Huber and Tweedie losses in GPU training
• QueryAUC metric implemented by @fibersel

Breaking changes

• We changed NDCG calculation principle for groups without relevant docs to make our NDCG score fully compatible with XGBoost and LightGBM implementations. Now we calc dcg==1 when there is no relevant objects in group (when ideal DCG equals zero), later we used score==0 in that case.

Speedups

• With help of Intel developers team we switched our threading model implementation to Intel Threading Building Blocks. That gives us up to 20% speedup on 28 threads and around 2x speedup when training in 120 threads and largely improves scalability.
• Speed up rendering fstat plots.
• Slightly speed up string casting in python package during pool creation.

R package

• Added path expansion when saving/loading files in R by @david-cortes
• Added functionality to restore R handle after deserializing model by @david-cortes
• Retrieve R pointers outside loops to speed up scalar access by @david-cortes
• Multiple R documentation edits from @david-cortes and @jameslamb
• #1588 Added precision for converting params to json

Bugfixes

• #1525 Problem with missing exported functions in Windows R package dll
• #1315 Low CPU utilization in CPU cross-validation
• #785 Predict on single item with iloc fixed by @feeeper
• Segfaults due to null pointer in pool in R package fixed by @david-cortes
• #1553 Added check for baseline dimensions count in apply
• #1606 Allow to use CatBoost in AWS Lambda environment: fix bug with setting thread names
• #1609 and #1309 Print proper error message if all params in grid were invalid
• Ability to use docstrings in estimators added by @pawelopiela
• Allow extra space at the end of line for libsvm format

Thanks!

• We would like to recognize Intel software engineering team’s contributions to Catboost project.
• Many thanks to our individual contributors: @david-cortes @jameslamb @pawelopiela @feeeper @fibersel

Release 0.24.4

Speedup

• Major speedup asymmetric trees training time on CPU (2x speedup on Epsilon with 16 threads). We would like to recognize Intel software engineering team’s contributions to Catboost project.

New features

• From now on we are releasing Python 3.9 wheels. Related issues: #1491, #1509, #1510
• Allow boost_from_average for MultiRMSE loss. Issue #1515
• Add tag pairwise=False for sklearn compatibility. Fixes issue #1518

Bugfixes:

• Allow fstr calculation for datasets with embeddings
• Fix feature_importances_ for fstr with texts
• Virtual ensebles fix: use proper unshrinkage coefficients
• Fixed constants in RMSEWithUnceratainty loss function calculation to correspond values from original paper
• Allow shap values calculation for model with zero-weights and non-zero leaf values. Now we use sum of leaf weights on train and current dataset to guarantee non-zero weights for leafs, reachable on current dataset. Fixes issues #1512, #1284

Release 0.24.3

New functionality

• Support fstr text features and embeddings. Issue #1293

Bugfixes:

• Fix model apply speed regression introduced in 0.24.1
• Different fixes in embeddings support: fixed apply and model serialization, fixed apply on texts and embeddings
• Fixed virtual ensembles prediction - use proper scaling, fix apply (issue #1462)
• Fix score() method for RMSEWithUncertainty issue #1482
• Automatically use correct prediction_type in score()

Uncertainty prediction

• Supported uncertainty prediction for classification models.
• Fixed RMSEWithUncertainty data uncertainty prediction - now it predicts variance, not standard deviation.

New functionality

• Allow categorical feature counters for MultiRMSE loss function.
• group_weight parameter added to catboost.utils.eval_metric method to allow passing weights for object groups. Allows correctly match weighted ranking metrics computation when group weights present.
• Faster non-owning deserialization from memory with less memory overhead - moved some dynamically computed data to model file, other data is computed in lazy manner only when needed.

Experimental functionality

• Supported embedding features as input and linear discriminant analysis for embeddings preprocessing. Try adding your embeddings as new columns with embedding values array in Pandas.Dataframe and passing corresponding column names to Pool constructor or fit function with embedding_features=['EmbeddingFeaturesColumnName1, ...] parameter. Another way of adding your embedding vectors is new type of column in Column Description file NumVector and adding semicolon separated embeddings column to your XSV file: ClassLabel\t0.1;0.2;0.3\t....

Educational materials

• Published new tutorial on uncertainty prediction.

Bugfixes:

• Reduced GPU memory usage in multi gpu training when there is no need to compute categorical feature counters.
• Now CatBoost allows to specify use_weights for metrics when auto_class_weights parameter is set.
• Correctly handle NaN values in plot_predictions function.
• Fixed floating point precision drop releated bugs during Multiclass training with lots of objects in our case, bug was triggered while training on 25mln objects on single GPU card.
• Now average parameter is passed to TotalF1 metric while training on GPU.
• Added class labels checks
• Disallow feature remapping in model predict when there is empty feature names in model.

Uncertainty prediction

Main feature of this release is total uncertainty prediction support via virtual ensembles. You can read the theoretical background in the preprint Uncertainty in Gradient Boosting via Ensembles from our research team. We introduced new training parameter posterior_sampling, that allows to estimate total uncertainty. Setting posterior_sampling=True implies enabling Langevin boosting, setting model_shrink_rate to 1/(2*N) and setting diffusion_temperature to N, where N is dataset size. CatBoost object method virtual_ensembles_predict splits model into virtual_ensembles_count submodels. Calling model.virtual_ensembles_predict(.., prediction_type='TotalUncertainty') returns mean prediction, variance (and knowledge uncertrainty for models, trained with RMSEWithUncertainty loss function). Calling model.virtual_ensembles_predict(.., prediction_type='VirtEnsembles') returns virtual_ensembles_count predictions of virtual submodels for each object.

New functionality

• Supported non-owning model deserialization for models with categorical feature counters

Speedups

• We've done lot's of speedups for sparse data loading. For example, on bosch sparse dataset preprocessing speed got 4.5x speedup while running in 28 thread setting.

Bugfixes:

• Fixed target check for PairLogitPairwise on GPU. Issue #1217
• Supported n_features_in_ attribute required for using CatBoost in sklearn pipelines. Issue #1363

New functionality

• We've finally implemented MVS sampling for GPU training. Switched default bootstrap algorithm to MVS for RMSE loss function while training on GPU
• Implemented near-zero cost model deserialization from memory blob. Currently, if your model doesn't use categorical features CTR counters and text features you can deserialize model from, for example, memory-mapped file.
• Added ability to load trained models from binary string or file-like stream. To load model from bytes string use load_model(blob=b'....'), to deserialize form file-like stream use load_model(stream=gzip.open('model.cbm.gz', 'rb'))
• Fixed auto-learning rate estimation params for GPU
• Supported beta parameter for QuerySoftMax function on CPU and GPU

New losses and metrics

• New loss function RMSEWithUncertainty - it allows to estimate data uncertainty for trained regression models. The trained model will give you a two-element vector for each object with the first element as regression model prediction and the second element as an estimation of data uncertainty for that prediction.

Speedups

• Major speedups for CPU training: kdd98 -9%, higgs -18%, msrank -28%. We would like to recognize Intel software engineering team’s contributions to Catboost project. This was mutually beneficial activity, and we look forward to continuing joint cooperation.

Bugfixes:

• Fixed CatBoost model export as Python code
• Fixed AUC metric creation
• Add text features to model.feature_names_. Issue #1314
• Allow models, trained on datasets with NaN values (Min treatment) and without NaNs in model_sum() or as the base model in init_model=. Issue #1271

Educational materials

• Published new tutorial on categorical features parameters. Thanks @garkavem

New functionality

• Added plot_partial_dependence method in python-package (Now it works for models with symmetric trees trained on dataset with numerical features only). Implemented by @felixandrer.
• Allowed using boost_from_average option together with model_shrink_rate option. In this case shrinkage is applied to the starting value..
• Added new auto_class_weights option in python-package, R-package and cli with possible values Balanced and SqrtBalanced. For Balanced every class is weighted maxSumWeightInClass / sumWeightInClass, where sumWeightInClass is sum of weights of all samples in this class. If no weights are present then sample weight is 1. And maxSumWeightInClass - is maximum sum weight among all classes. For SqrtBalanced the formula is sqrt(maxSumWeightInClass / sumWeightInClass). This option supported in binclass and multiclass tasks. Implemented by @egiby.
• Supported model_size_reg option on GPU. Set to 0.5 by default (same as in CPU). This regularization works slightly differently on GPU: feature combinations are regularized more aggressively than on CPU. For CPU cost of a combination is equal to number of different feature values in this combinations that are present in training dataset. On GPU cost of a combination is equal to number of all possible different values of this combination. For example, if combination contains two categorical features c1 and c2, then the cost will be #categories in c1 * #categories in c2, even though many of the values from this combination might not be present in the dataset.
• Added calculation of Shapley values, (see formula (2) from https://arxiv.org/pdf/1802.03888.pdf). By default estimation from this paper (Algorithm 2) is calcucated, that is much more faster. To use this mode specify shap_calc_type parameter of CatBoost.get_feature_importance function as "Exact". Implemented by @LordProtoss.

Bugfixes:

• Fixed onnx converter for old onnx versions.

New functionality

• CatBoost model could be simply converted into ONNX object in Python with catboost.utils.convert_to_onnx_object method. Implemented by @monkey0head
• We now print metric options with metric names as metric description in error logs by default. This allows you to distinguish between metrics of the same type with different parameters. For example, if user sets weigheted average TotalF1 metric CatBoost will print TotalF1:average=Weighted as corresponding metric column header in error logs. Implemented by @ivanychev
• Implemented PRAUC metric (issue #737). Thanks @azikmsu
• It's now possible to write custom multiregression objective in Python. Thanks @azikmsu
• Supported nonsymmetric models export to PMML
• class_weights parameter accepts dictionary with class name to class weight mapping
• Added _get_tags() method for compatibility with sklearn (issue #1282). Implemented by @crazyleg
• Lot's of improvements in .Net CatBoost library: implemented IDisposable interface, splitted ML.NET compatible and basic prediction classes in separate libraries, added base UNIX compatibility, supported GPU model evaluation, fixed tests. Thanks @khanova
• In addition to first_feature_use_penalties presented in the previous release, we added new option per_object_feature_penalties which considers feature usage on each object individually. For more details refer the tutorial.

Breaking changes

• From now on we require explicit loss_function param in python cv method.

Bugfixes:

• Fixed deprecation warning on import (issue #1269)
• Fixed saved models logging_level/verbose parameters conflict (issue #696)
• Fixed kappa metric - in some cases there were integer overflow, switched accumulation types to double
• Fixed per float feature quantization settings defaults

Educational materials

• Extended shap values tutorial with summary plot examples. Thanks @azanovivan02

New functionality

• It is possible now to train models on huge datasets that do not fit into CPU RAM. This can be accomplished by storing only quantized data in memory (it is many times smaller). Use catboost.utils.quantize function to create quantized Pool this way. See usage example in the issue #1116. Implemented by @noxwell.
• Python Pool class now has save_quantization_borders method that allows to save resulting borders into a file and use it for quantization of other datasets. Quantization can be a bottleneck of training, especially on GPU. Doing quantization once for several trainings can significantly reduce running time. It is recommended for large dataset to perform quantization first, save quantization borders, use them to quantize validation dataset, and then use quantized training and validation datasets for further training. Use saved borders when quantizing other Pools by specifying input_borders parameter of the quantize method. Implemented by @noxwell.
• Training with text features is now supported on CPU
• It is now possible to set border_count > 255 for GPU training. This might be useful if you have a "golden feature", see docs.
• Feature weights are implemented. Specify weights for specific features by index or name like feature_weights="FeatureName1:1.5,FeatureName2:0.5". Scores for splits with this features will be multiplied by corresponding weights. Implemented by @Taube03.
• Feature penalties can be used for cost efficient gradient boosting. Penalties are specified in a similar fashion to feature weights, using parameter first_use_feature_penalties. This parameter penalized the first usage of a feature. This should be used in case if the calculation of the feature is costly. The penalty value (or the cost of using a feature) is subtracted from scores of the splits of this feature if feature has not been used in the model. After the feature has been used once, it is considered free to proceed using this feature, so no substruction is done. There is also a common multiplier for all first_use_feature_penalties, it can be specified by penalties_coefficient parameter. Implemented by @Taube03 (issue #1155)
• recordCount attribute is added to PMML models (issue #1026).

New losses and metrics

• New ranking objective 'StochasticRank', details in paper.
• Tweedie loss is supported now. It can be a good solution for right-skewed target with many zero values, see tutorial. When using CatBoostRegressor.predict function, default prediction_type for this loss will be equal to Exponent. Implemented by @ilya-pchelintsev (issue #577)
• Classification metrics now support a new parameter proba_border. With this parameter you can set decision boundary for treating prediction as negative or positive. Implemented by @ivanychev.
• Metric TotalF1 supports a new parameter average with possible value weighted, micro, macro. Implemented by @ilya-pchelintsev.
• It is possible now to specify a custom multi-label metric in python. Note that it is only possible to calculate this metric and use it as eval_metric. It is not possible to used it as an optimization objective. To write a multi-label metric, you need to define a python class which inherits from MultiLabelCustomMetric class. Implemented by @azikmsu.

Improvements of grid and randomized search

• class_weights parameter is now supported in grid/randomized search. Implemented by @vazgenk.
• Invalid option configurations are automatically skipped during grid/randomized search. Implemented by @borzunov.
• get_best_score returns train/validation best score after grid/randomized search (in case of refit=False). Implemented by @rednevaler.

Improvements of model analysis tools

• Computation of SHAP interaction values for CatBoost models. You can pass type=EFstrType.ShapInteractionValues to CatBoost.get_feature_importance to get a matrix of SHAP values for every prediction. By default, SHAP interaction values are calculated for all features. You may specify features of interest using the interaction_indices argument. Implemented by @IvanKozlov98.
• SHAP values can be calculated approximately now which is much faster than default mode. To use this mode specify shap_calc_type parameter of CatBoost.get_feature_importance function as "Approximate". Implemented by @LordProtoss (issue #1146).
• PredictionDiff model analysis method can now be used with models that contain non symmetric trees. Implemented by @felixandrer.

New educational materials

• A tutorial on tweedie regression
• A tutorial on poisson regression
• A detailed tutorial on different types of AUC metric, which explains how different types of AUC can be used for binary classification, multiclassification and ranking tasks.

Breaking changes

• When using CatBoostRegressor.predict function for models trained with Poisson loss, default prediction_type will be equal to Exponent (issue #1184). Implemented by @garkavem.

This release also contains bug fixes and performance improvements, including a major speedup for sparse data on GPU.

New features:

• The main feature of the release is the support of non symmetric trees for training on CPU. Using non symmetric trees might be useful if one-hot encoding is present, or data has little noise. To try non symmetric trees change grow_policy parameter. Starting from this release non symmetric trees are supported for both CPU and GPU training.
• The next big feature improves catboost text features support. Now tokenization is done during training, you don't have to do lowercasing, digit extraction and other tokenization on your own, catboost does it for you.
• Auto learning-rate is now supported in CPU MultiClass mode.
• CatBoost class supports to_regressor and to_classifier methods.

The release also contains a list of bug fixes.

New features:

• The main feature of this release is the Stochastic Gradient Langevin Boosting (SGLB) mode that can improve quality of your models with non-convex loss functions. To use it specify langevin option and tune diffusion_temperature and model_shrink_rate. See the corresponding paper for details.

Improvements:

• Automatic learning rate is applied by default not only for Logloss objective, but also for RMSE (on CPU and GPU) and MultiClass (on GPU).
• Class labels type information is stored in the model. Now estimators in python package return values of proper type in classes_ attribute and for prediction functions with prediction_type=Class. #305, #999, #1017. Note: Class labels loaded from datasets in CatBoost dsv format always have string type now.

Bug fixes:

• Fixed huge memory consumption for text features. #1107
• Fixed crash on GPU on big datasets with groups (hundred million+ groups).
• Fixed class labels consistency check and merging in model sums (now class names in binary classification are properly checked and added to the result as well)
• Fix for confusion matrix (PR #1152), thanks to @dmsivkov.
• Fixed shap values calculation when boost_from_average=True. #1125
• Fixed use-after-free in fstr PredictionValuesChange with specified dataset
• Target border and class weights are now taken from model when necessary for feature strength, metrics evaluation, roc_curve, object importances and calc_feature_statistics calculations.
• Fixed that L2 regularization was not applied for non symmetric trees for binary classification on GPU.
• [R-package] Fixed the bug that catboost.get_feature_importance did not work after model is loaded #1064
• [R-package] Fixed the bug that catboost.train did not work when called with the single dataset parameter. #1162
• Fixed L2 score calculation on CPU

Other:

• Starting from this release Java applier is released simultaneously with other components and has the same version.

Compatibility:

• Models trained with this release require applier from this release or later to work correctly.

New features:

• String class labels are now supported for binary classification
• [CLI only] Timestamp column for the datasets can be provided in separate files.
• [CLI only] Timesplit feature evaluation.
• Process groups of any size in block processing.

Bug fixes:

• classes_count and class_weight params can be now used with user-defined loss functions. #1119
• Form correct metric descriptions on GPU if use_weights gets value by default. #1106
• Correct model.classes_ attribute for binary classification (proper labels instead of always 0 and 1). #984
• Fix model.classes_ attribute when classes_count parameter was specified.
• Proper error message when categorical features specified for MultiRMSE training. #1112
• Block processing: It is valid for all groups in a single block to have weights equal to 0
• fix empty asymmetric tree index calculation. #1104

New features:

• Have leaf_estimation_method=Exact the default for MAPE loss
• Add CatBoostClassifier.predict_log_proba(), PR #1095

Bug fixes:

• Fix usability of read-only numpy arrays, #1101
• Fix python3 compatibility for get_feature_importance, PR #1090
• Fix loading model from snapshot for boost_from_average mode

New submodule for text processing! It contains two classes to help you make text features ready for training:

• Tokenizer -- use this class to split text into tokens (automatic lowercase and punctuation removal)
• Dictionary -- with this class you create a dictionary which maps tokens to numeric identifiers. You then use these identifiers as new features.

New features:

• Enabled boost_from_average for MAPE loss function

Bug fixes:

• Fixed Pool creation from pandas.DataFrame with discontinuous columns, #1079
• Fixed standalone_evaluator, PR #1083

Speedups:

• Huge speedup of preprocessing in python-package for datasets with many samples (>10 mln)

We also release precompiled packages for Python 3.8

New features:

• With this release we support Text features for classification on GPU. To specify text columns use text_features parameter. Achieve better quality by using text information of your dataset. See more in Learning CatBoost with text features
• MultiRMSE loss function is now available on CPU. Labels for the multi regression mode should be specified in separate Label columns
• MonoForest framework for model analysis, based on our NeurIPS 2019 paper. Learn more in MonoForest tutorial
• boost_from_average is now True by default for Quantile and MAE loss functions, which improves the resulting quality

Speedups:

• Huge reduction of preprocessing time for datasets loaded from files and for datasets with many samples (> 10 million), which was a bottleneck for GPU training
• 3x speedup for small datasets

New features:

• Now datasets.msrank() returns full msrank dataset. Previously, it returned the first 10k samples. We have added msrank_10k() dataset implementing the past behaviour.

Bug fixes:

• get_object_importance() now respects parameter top_size, #1045 by @ibuda

• The main feature of the release is huge speedup on small datasets. We now use MVS sampling for CPU regression and binary classification training by default, together with Plain boosting scheme for both small and large datasets. This change not only gives the huge speedup but also provides quality improvement!
• The boost_from_average parameter is available in CatBoostClassifier and CatBoostRegressor
• We have added new formats for describing monotonic constraints. For example, "(1,0,0,-1)" or "0:1,3:-1" or "FeatureName0:1,FeatureName3:-1" are all valid specifications. With Python and params-file json, lists and dictionaries can also be used

Bugs fixed:

• Error in Multiclass classifier training, #1040
• Unhandled exception when saving quantized pool, #1021
• Python 3.7: RuntimeError raised in StagedPredictIterator, #848

Bugs fixed:

• System of linear equations is not positive definite when training MultiClass on Windows, #1022
• Cat feature values could be taken from floating-point data. We have forbidden this
• Handling of numpy.ndarray features data with categorical features is corrected