API Reference

This page documents the public API of the linearboost package. Install it with pip install linearboost so that the classes below are available.

LinearBoostClassifier

class linearboost.LinearBoostClassifier(n_estimators=200, *, learning_rate=1.0, algorithm='SAMME.R', scaler='minmax', class_weight=None, loss_function=None, kernel='linear', gamma=None, degree=3, coef0=1, kernel_approx=None, n_components=256, early_stopping=False, validation_fraction=0.1, n_iter_no_change=5, tol=0.0001, subsample=1.0, shrinkage=1.0, boosting_type='adaboost')

Bases: _DenseAdaBoostClassifier

A LinearBoost classifier.

A LinearBoost classifier is a meta-estimator based on AdaBoost and SEFR. It is a fast and accurate classification algorithm built to enhance the performance of the linear classifier SEFR.

Parameters:

n_estimators (int, default=200) – The maximum number of SEFR classifiers at which boosting is terminated. In case of perfect fit, the learning procedure is stopped early. Values must be in the range [1, inf), preferably [10, 200].
learning_rate (float, default=1.0) – Weight applied to each SEFR classifier at each boosting iteration. A higher learning rate increases the contribution of each SEFR classifier. There is a trade-off between the learning_rate and n_estimators parameters. Values must be in the range (0.0, inf), preferably (0.0, 1.0).
algorithm ({'SAMME', 'SAMME.R'}, default='SAMME') – If ‘SAMME’ then use the SAMME discrete boosting algorithm. If ‘SAMME.R’ then use the SAMME.R real boosting algorithm (implemented from scikit-learn = 1.5). The SAMME.R algorithm typically converges faster than SAMME, achieving a lower test error with fewer boosting iterations.
scaler (str, default='minmax') –
Specifies the scaler to apply to the data. Options include:
- ’minmax’: Applies MinMaxScaler.
- ’quantile-uniform’: Uses QuantileTransformer with output_distribution=’uniform’.
- ’quantile-normal’: Uses QuantileTransformer with output_distribution=’normal’.
- ’normalizer-l1’: Applies Normalizer with norm=’l1’.
- ’normalizer-l2’: Applies Normalizer with norm=’l2’.
- ’normalizer-max’: Applies Normalizer with norm=’max’.
- ’standard’: Uses StandardScaler.
- ’power’: Applies PowerTransformer with method=’yeo-johnson’.
- ’maxabs’: Uses MaxAbsScaler.
- ’robust’: Applies RobustScaler.
kernel ({'linear', 'poly', 'rbf', 'sigmoid'} or callable, default='linear') – Specifies the kernel type to be used in the algorithm. If a callable is given, it is used to pre-compute the kernel matrix.
kernel_approx ({'rff', 'nystrom'} or None, default=None) –
Optional kernel approximation strategy for non-linear kernels.
- ’rff’: Use Random Fourier Features (RBFSampler). Only valid when kernel='rbf'. Approximates the RBF kernel via an explicit low-dimensional feature map.
- ’nystrom’: Use Nyström approximation (Nystroem). Can be used with ‘rbf’, ‘poly’, or ‘sigmoid’ kernels.
- None: Use exact kernel with full Gram matrix (O(n^2) memory).
n_components (int, default=256) – Dimensionality of the kernel feature map when using kernel approximation. Acts as the number of random features (for ‘rff’) or the rank of the approximation (for ‘nystrom’). Must be >= 1.

gammafloat, default=None

Kernel coefficient for ‘rbf’, ‘poly’ and ‘sigmoid’. If None, then it is set to 1.0 / n_features.

degreeint, default=3

Degree for ‘poly’ kernels. Ignored by other kernels.

coef0float, default=1

Independent term in kernel function. It is only significant in ‘poly’ and ‘sigmoid’.

class_weight{“balanced”}, dict or list of dicts, default=None

Weights associated with classes in the form {class_label: weight}. If not given, all classes are supposed to have weight one.

The “balanced” mode uses the values of y to automatically adjust weights inversely proportional to class frequencies in the input data as n_samples / (n_classes * np.bincount(y))

Note that these weights will be multiplied with sample_weight (passed through the fit method) if sample_weight is specified.

loss_functioncallable, default=None

Custom loss function for optimization. Must follow the signature:

loss_function(y_true, y_pred, sample_weight) -> float

where: - y_true: Ground truth (correct) target values. - y_pred: Estimated target values. - sample_weight: Sample weights (optional).

early_stoppingbool, default=False

Whether to use early stopping to terminate training when validation score is not improving. If True, it requires n_iter_no_change to be set.

If subsample < 1.0 (subsampling is enabled), Out-of-Bag (OOB) evaluation is automatically used instead of a fixed validation split. This is more data-efficient as it uses all training data while still providing validation feedback. OOB evaluation uses samples not included in each iteration’s subsample for validation.

validation_fractionfloat, default=0.1

The proportion of training data to set aside as validation set for early stopping. Must be between 0 and 1. Only used if early_stopping is True and subsample >= 1.0 (no subsampling). When subsampling is enabled, OOB evaluation is used instead and this parameter is ignored.

n_iter_no_changeint, default=5

Number of iterations with no improvement to wait before early stopping. Only used if early_stopping is True. Must be >= 1.

tolfloat, default=1e-4

Tolerance for the optimization. When the loss or score is not improving by at least tol for n_iter_no_change consecutive iterations, convergence is considered to be reached and training stops. Only used if early_stopping is True. Must be >= 0.

subsamplefloat, default=1.0

The fraction of samples to be used for fitting the individual base learners. If smaller than 1.0 this results in Stochastic Gradient Boosting. subsample interacts with the parameter n_estimators. Choosing subsample < 1.0 leads to a reduction of variance and an increase in bias. Values must be in the range (0, 1].

shrinkagefloat, default=1.0

Shrinkage parameter for regularization. Each estimator weight is multiplied by this factor. Values < 1.0 reduce the contribution of each base learner, helping to prevent overfitting and improve generalization. This is similar to the shrinkage used in gradient boosting methods.

If shrinkage = 1.0: no shrinkage (full weight)
If shrinkage < 1.0: apply shrinkage (e.g., 0.8 means 80% weight)

Values must be in the range (0, 1]. Typical values are in the range [0.8, 1.0] for moderate regularization or 1.0 for no regularization.

boosting_type{‘adaboost’, ‘gradient’}, default=’adaboost’

The type of boosting algorithm to use:

‘adaboost’: Use the AdaBoost algorithm (SAMME or SAMME.R) which reweights samples based on classification errors. This is the original LinearBoost approach.
‘gradient’: Use gradient boosting which fits each new estimator to the pseudo-residuals (negative gradient of log-loss). This can be more effective for complex non-linear patterns and provides smoother decision boundaries.

When boosting_type='gradient': - The algorithm parameter is ignored - Each estimator predicts pseudo-residuals instead of class labels - The ensemble prediction is the sum of estimator predictions - Better suited for XOR-like and highly non-linear patterns

Variables:

estimator (estimator) –
The base estimator (SEFR) from which the ensemble is grown.

Added in version scikit-learn: 1.2 base_estimator_ was renamed to estimator_.
base_estimator (estimator) –
The base estimator from which the ensemble is grown.

Deprecated since version scikit-learn: 1.2 base_estimator_ is deprecated and will be removed in scikit-learn 1.4. Use estimator_ instead.
estimators (list of classifiers) – The collection of fitted sub-estimators.
classes (ndarray of shape (n_classes,)) – The classes labels.
n_classes (int) – The number of classes.
estimator_weights (ndarray of floats) – Weights for each estimator in the boosted ensemble.
estimator_errors (ndarray of floats) – Classification error for each estimator in the boosted ensemble.
n_features_in (int) – Number of features seen during fit.
feature_names_in (ndarray of shape (n_features_in_,)) – Names of features seen during fit. Defined only when X has feature names that are all strings.
scaler (transformer) – The scaler instance used to transform the data.
X_fit (ndarray of shape (n_samples, n_features)) – The training data after scaling, stored when kernel != ‘linear’ for prediction purposes.
K_train (ndarray of shape (n_samples, n_samples)) – The precomputed kernel matrix on training data, stored when kernel != ‘linear’.
F (ndarray of shape (n_samples,)) – The raw prediction scores (log-odds) from gradient boosting. Only present when boosting_type='gradient'.
init_score (float) – The initial score (log-odds of class prior) for gradient boosting. Only present when boosting_type='gradient'.

Notes

This classifier only supports binary classification tasks.

Examples

>>> from linearboost import LinearBoostClassifier
>>> from sklearn.datasets import load_breast_cancer
>>> X, y = load_breast_cancer(return_X_y=True)
>>> clf = LinearBoostClassifier().fit(X, y)
>>> clf.predict(X[:2, :])
array([0, 0])
>>> clf.predict_proba(X[:2, :])
array([[0.88079708, 0.11920292],
       [0.88079708, 0.11920292]])
>>> clf.score(X, y)
0.97...

The main ensemble classifier. Supports AdaBoost and gradient boosting, kernels (linear, RBF, poly, sigmoid), kernel approximation (RFF, Nyström), early stopping, subsampling, shrinkage, and class weighting.

decision_function(X)

Compute the decision function of X.

Parameters:: X ({array-like} of shape (n_samples, n_features)) – The training input samples.
Returns:: score – The decision function of the input samples. The order of outputs is the same as that of the classes_ attribute. Binary classification is a special cases with k == 1, otherwise k==n_classes. For binary classification, values closer to -1 or 1 mean more like the first or second class in classes_, respectively.
Return type:: ndarray of shape of (n_samples, k)

property feature_importances_

The impurity-based feature importances.

The higher, the more important the feature. The importance of a feature is computed as the (normalized) total reduction of the criterion brought by that feature. It is also known as the Gini importance.

Warning: impurity-based feature importances can be misleading for high cardinality features (many unique values). See sklearn.inspection.permutation_importance() as an alternative.

Returns:: feature_importances_ – The feature importances.
Return type:: ndarray of shape (n_features,)

fit(X, y, sample_weight=None) → Self

Build a LinearBoost classifier from the training set (X, y).

Parameters:

X ({array-like} of shape (n_samples, n_features)) – The training input samples.
y (array-like of shape (n_samples,)) – The target values.
sample_weight (array-like of shape (n_samples,), default=None) – Sample weights. If None, the sample weights are initialized to 1 / n_samples.

Returns:

self – Fitted estimator.

Return type:

object

get_metadata_routing()

Raise NotImplementedError.

This estimator does not support metadata routing yet.

get_params(deep=True)

Get parameters for this estimator.

Parameters:: deep (bool, default=True) – If True, will return the parameters for this estimator and contained subobjects that are estimators.
Returns:: params – Parameter names mapped to their values.
Return type:: dict

predict(X)

Predict classes for X.

The predicted class of an input sample is computed as the weighted mean prediction of the classifiers in the ensemble.

Parameters:: X ({array-like} of shape (n_samples, n_features)) – The training input samples.
Returns:: y – The predicted classes.
Return type:: ndarray of shape (n_samples,)

predict_log_proba(X)

Predict class log-probabilities for X.

The predicted class log-probabilities of an input sample is computed as the weighted mean predicted class log-probabilities of the classifiers in the ensemble.

Parameters:: X ({array-like} of shape (n_samples, n_features)) – The training input samples.
Returns:: p – The class probabilities of the input samples. The order of outputs is the same of that of the classes_ attribute.
Return type:: ndarray of shape (n_samples, n_classes)

predict_proba(X)

Predict class probabilities for X.

The predicted class probabilities of an input sample is computed as the weighted mean predicted class probabilities of the classifiers in the ensemble.

Parameters:: X ({array-like} of shape (n_samples, n_features)) – The training input samples.
Returns:: p – The class probabilities of the input samples. The order of outputs is the same of that of the classes_ attribute.
Return type:: ndarray of shape (n_samples, n_classes)

score(X, y, sample_weight=None)

Return accuracy on provided data and labels.

In multi-label classification, this is the subset accuracy which is a harsh metric since you require for each sample that each label set be correctly predicted.

Parameters:

X (array-like of shape (n_samples, n_features)) – Test samples.
y (array-like of shape (n_samples,) or (n_samples, n_outputs)) – True labels for X.
sample_weight (array-like of shape (n_samples,), default=None) – Sample weights.

Returns:

score – Mean accuracy of self.predict(X) w.r.t. y.

Return type:

float

set_fit_request(*, sample_weight: bool | None | str = '$UNCHANGED$') → LinearBoostClassifier

Configure whether metadata should be requested to be passed to the fit method.

Note that this method is only relevant when this estimator is used as a sub-estimator within a meta-estimator and metadata routing is enabled with enable_metadata_routing=True (see sklearn.set_config()). Please check the User Guide on how the routing mechanism works.

The options for each parameter are:

True: metadata is requested, and passed to fit if provided. The request is ignored if metadata is not provided.
False: metadata is not requested and the meta-estimator will not pass it to fit.
None: metadata is not requested, and the meta-estimator will raise an error if the user provides it.
str: metadata should be passed to the meta-estimator with this given alias instead of the original name.

The default (sklearn.utils.metadata_routing.UNCHANGED) retains the existing request. This allows you to change the request for some parameters and not others.

Added in version 1.3.

Parameters:: sample_weight (str, True, False, or None, default=sklearn.utils.metadata_routing.UNCHANGED) – Metadata routing for sample_weight parameter in fit.
Returns:: self – The updated object.
Return type:: object

set_params(**params)

Set the parameters of this estimator.

The method works on simple estimators as well as on nested objects (such as Pipeline). The latter have parameters of the form <component>__<parameter> so that it’s possible to update each component of a nested object.

Parameters:: **params (dict) – Estimator parameters.
Returns:: self – Estimator instance.
Return type:: estimator instance

set_score_request(*, sample_weight: bool | None | str = '$UNCHANGED$') → LinearBoostClassifier

Configure whether metadata should be requested to be passed to the score method.

Note that this method is only relevant when this estimator is used as a sub-estimator within a meta-estimator and metadata routing is enabled with enable_metadata_routing=True (see sklearn.set_config()). Please check the User Guide on how the routing mechanism works.

The options for each parameter are:

True: metadata is requested, and passed to score if provided. The request is ignored if metadata is not provided.
False: metadata is not requested and the meta-estimator will not pass it to score.
None: metadata is not requested, and the meta-estimator will raise an error if the user provides it.
str: metadata should be passed to the meta-estimator with this given alias instead of the original name.

The default (sklearn.utils.metadata_routing.UNCHANGED) retains the existing request. This allows you to change the request for some parameters and not others.

Added in version 1.3.

Parameters:: sample_weight (str, True, False, or None, default=sklearn.utils.metadata_routing.UNCHANGED) – Metadata routing for sample_weight parameter in score.
Returns:: self – The updated object.
Return type:: object

staged_decision_function(X)

Compute decision function of X for each boosting iteration.

This method allows monitoring (i.e. determine error on testing set) after each boosting iteration.

Parameters:: X ({array-like} of shape (n_samples, n_features)) – The training input samples.
Yields:: score (generator of ndarray of shape (n_samples, k)) – The decision function of the input samples. The order of outputs is the same of that of the classes_ attribute. Binary classification is a special cases with k == 1, otherwise k==n_classes. For binary classification, values closer to -1 or 1 mean more like the first or second class in classes_, respectively.

staged_predict(X)

Return staged predictions for X.

The predicted class of an input sample is computed as the weighted mean prediction of the classifiers in the ensemble.

This generator method yields the ensemble prediction after each iteration of boosting and therefore allows monitoring, such as to determine the prediction on a test set after each boost.

Parameters:: X (array-like of shape (n_samples, n_features)) – The input samples.
Yields:: y (generator of ndarray of shape (n_samples,)) – The predicted classes.

staged_predict_proba(X)

Predict class probabilities for X.

The predicted class probabilities of an input sample is computed as the weighted mean predicted class probabilities of the classifiers in the ensemble.

This generator method yields the ensemble predicted class probabilities after each iteration of boosting and therefore allows monitoring, such as to determine the predicted class probabilities on a test set after each boost.

Parameters:: X ({array-like} of shape (n_samples, n_features)) – The training input samples.
Yields:: p (generator of ndarray of shape (n_samples,)) – The class probabilities of the input samples. The order of outputs is the same of that of the classes_ attribute.

staged_score(X, y, sample_weight=None)

Return staged scores for X, y.

This generator method yields the ensemble score after each iteration of boosting and therefore allows monitoring, such as to determine the score on a test set after each boost.

Parameters:

X ({array-like} of shape (n_samples, n_features)) – The training input samples.
y (array-like of shape (n_samples,)) – Labels for X.
sample_weight (array-like of shape (n_samples,), default=None) – Sample weights.

Yields:

z (float)

SEFR

class linearboost.SEFR(*, fit_intercept=True, kernel='linear', gamma=None, degree=3, coef0=1)

Bases: LinearClassifierMixin, BaseEstimator

A Scalable, Efficient, and Fast (SEFR) classifier.

SEFR is an ultra-low power binary linear classifier designed specifically for resource-constrained devices. It operates by computing feature weights based on class-wise averages, achieving linear time complexity in both training and inference. The algorithm provides comparable accuracy to state-of-the-art methods while being significantly more energy efficient.

Parameters:

fit_intercept (bool, default=True) – Specifies if a constant (a.k.a. bias or intercept) should be added to the decision function.
kernel ({'linear', 'poly', 'rbf', 'sigmoid', 'precomputed'} or callable, default='linear') – Specifies the kernel type to be used in the algorithm. If a callable is given, it is used to pre-compute the kernel matrix. If ‘precomputed’, X is assumed to be a kernel matrix.
gamma (float, default=None) – Kernel coefficient for ‘rbf’, ‘poly’ and ‘sigmoid’. If None, then it is set to 1.0 / n_features. Ignored when kernel=’precomputed’.
degree (int, default=3) – Degree for ‘poly’ kernels. Ignored by other kernels.
coef0 (float, default=1) – Independent term in kernel function. It is only significant in ‘poly’ and ‘sigmoid’.

Variables:

classes (ndarray of shape (n_classes, )) – A list of class labels known to the classifier.
coef (ndarray of shape (1, n_features) or (1, n_samples)) – Coefficient of the features in the decision function. When a kernel is used, the shape will be (1, n_samples).
intercept (ndarray of shape (1,)) –
Intercept (a.k.a. bias) added to the decision function.

If fit_intercept is set to False, the intercept is set to zero.
n_features_in (int) – Number of features seen during fit.
feature_names_in (ndarray of shape (n_features_in_,)) – Names of features seen during fit. Defined only when X has feature names that are all strings.
X_fit (ndarray of shape (n_samples, n_features)) – The training data, stored when a kernel is used (except for ‘precomputed’).

Notes

This classifier only supports binary classification tasks.

Examples

>>> from linearboost import SEFR
>>> from sklearn.datasets import load_breast_cancer
>>> X, y = load_breast_cancer(return_X_y=True)
>>> clf = SEFR(kernel='rbf').fit(X, y)
>>> clf.predict(X[:2, :])
array([0, 0])
>>> clf.score(X, y)
0.89...

The base binary linear classifier used by LinearBoost. Can be used standalone with linear or kernel (RBF, poly, sigmoid, precomputed) options. Supports fit_intercept and is fully scikit-learn compatible.

decision_function(X)

Predict confidence scores for samples.

The confidence score for a sample is proportional to the signed distance of that sample to the hyperplane.

Parameters:: X ({array-like, sparse matrix} of shape (n_samples, n_features)) – The data matrix for which we want to get the confidence scores.
Returns:: scores – Confidence scores per (n_samples, n_classes) combination. In the binary case, confidence score for self.classes_[1] where >0 means this class would be predicted.
Return type:: ndarray of shape (n_samples,) or (n_samples, n_classes)

fit(X, y, sample_weight=None) → Self

Fit the model according to the given training data.

Parameters:

X ({array-like, sparse matrix} of shape (n_samples, n_features) or (n_samples, n_samples)) – Training vector, where n_samples is the number of samples and n_features is the number of features. If kernel=’precomputed’, X should be a square kernel matrix.
y (array-like of shape (n_samples,)) – Target vector relative to X.
sample_weight (array-like of shape (n_samples,) default=None) – Array of weights that are assigned to individual samples. If not provided, then each sample is given unit weight.

Returns:

Fitted estimator.

Return type:

self

get_metadata_routing()

Get metadata routing of this object.

Please check User Guide on how the routing mechanism works.

Returns:: routing – A MetadataRequest encapsulating routing information.
Return type:: MetadataRequest

get_params(deep=True)

Get parameters for this estimator.

Parameters:: deep (bool, default=True) – If True, will return the parameters for this estimator and contained subobjects that are estimators.
Returns:: params – Parameter names mapped to their values.
Return type:: dict

predict(X)

Predict class labels for samples in X.

Parameters:: X ({array-like, sparse matrix} of shape (n_samples, n_features)) – The data matrix for which we want to get the predictions.
Returns:: y_pred – Vector containing the class labels for each sample.
Return type:: ndarray of shape (n_samples,)

predict_log_proba(X)

Predict logarithm of probability estimates.

The returned estimates for all classes are ordered by the label of classes.

Parameters:: X (array-like of shape (n_samples, n_features) or (n_samples, n_train_samples)) – Vector to be scored, where n_samples is the number of samples and n_features is the number of features. If kernel=’precomputed’, X should have shape (n_samples, n_train_samples).
Returns:: T – Returns the log-probability of the sample for each class in the model, where classes are ordered as they are in self.classes_.
Return type:: array-like of shape (n_samples, n_classes)

predict_proba(X)

Probability estimates.

The returned estimates for all classes are ordered by the label of classes.

Parameters:: X (array-like of shape (n_samples, n_features) or (n_samples, n_train_samples)) – Vector to be scored, where n_samples is the number of samples and n_features is the number of features. If kernel=’precomputed’, X should have shape (n_samples, n_train_samples).
Returns:: T – Returns the probability of the sample for each class in the model, where classes are ordered as they are in self.classes_.
Return type:: array-like of shape (n_samples, n_classes)

score(X, y, sample_weight=None)

Return accuracy on provided data and labels.

In multi-label classification, this is the subset accuracy which is a harsh metric since you require for each sample that each label set be correctly predicted.

Parameters:

X (array-like of shape (n_samples, n_features)) – Test samples.
y (array-like of shape (n_samples,) or (n_samples, n_outputs)) – True labels for X.
sample_weight (array-like of shape (n_samples,), default=None) – Sample weights.

Returns:

score – Mean accuracy of self.predict(X) w.r.t. y.

Return type:

float

set_fit_request(*, sample_weight: bool | None | str = '$UNCHANGED$') → SEFR

Configure whether metadata should be requested to be passed to the fit method.

Note that this method is only relevant when this estimator is used as a sub-estimator within a meta-estimator and metadata routing is enabled with enable_metadata_routing=True (see sklearn.set_config()). Please check the User Guide on how the routing mechanism works.

The options for each parameter are:

True: metadata is requested, and passed to fit if provided. The request is ignored if metadata is not provided.
False: metadata is not requested and the meta-estimator will not pass it to fit.
None: metadata is not requested, and the meta-estimator will raise an error if the user provides it.
str: metadata should be passed to the meta-estimator with this given alias instead of the original name.

The default (sklearn.utils.metadata_routing.UNCHANGED) retains the existing request. This allows you to change the request for some parameters and not others.

Added in version 1.3.

Parameters:: sample_weight (str, True, False, or None, default=sklearn.utils.metadata_routing.UNCHANGED) – Metadata routing for sample_weight parameter in fit.
Returns:: self – The updated object.
Return type:: object

set_params(**params)

Set the parameters of this estimator.

The method works on simple estimators as well as on nested objects (such as Pipeline). The latter have parameters of the form <component>__<parameter> so that it’s possible to update each component of a nested object.

Parameters:: **params (dict) – Estimator parameters.
Returns:: self – Estimator instance.
Return type:: estimator instance

set_score_request(*, sample_weight: bool | None | str = '$UNCHANGED$') → SEFR

Configure whether metadata should be requested to be passed to the score method.

Note that this method is only relevant when this estimator is used as a sub-estimator within a meta-estimator and metadata routing is enabled with enable_metadata_routing=True (see sklearn.set_config()). Please check the User Guide on how the routing mechanism works.

The options for each parameter are:

True: metadata is requested, and passed to score if provided. The request is ignored if metadata is not provided.
False: metadata is not requested and the meta-estimator will not pass it to score.
None: metadata is not requested, and the meta-estimator will raise an error if the user provides it.
str: metadata should be passed to the meta-estimator with this given alias instead of the original name.

The default (sklearn.utils.metadata_routing.UNCHANGED) retains the existing request. This allows you to change the request for some parameters and not others.

Added in version 1.3.

Parameters:: sample_weight (str, True, False, or None, default=sklearn.utils.metadata_routing.UNCHANGED) – Metadata routing for sample_weight parameter in score.
Returns:: self – The updated object.
Return type:: object