Skip to main content

Algorithms Used

AutoML uses four different models to make predictions.

For regression models:

For classification models:

XGBRegressor

For regression models, AutoML uses XGBoost’s XGBRegressorOpens in a new tab class.

The model hyperparameters are detailed below:

Hyperparameter Value
Max Depth 3
Learning Rate 0.1
Number of Estimators 100
Objective Squared Error
Booster Gbtree
Tree Method Auto
Number of Jobs 1
Gamma (min loss reduction for partition on leaf) 0
Min Child Weight 1
Max Delta Step 0
L2 Regularization Lambda 1
Scale Positive Weight 1
Base/Initial Score 0.5

Neural Network

For the Neural Network model, AutoML uses TensorFlow with KerasOpens in a new tab as a wrapper.

The input layer has its size based on the number of features. This layer is then densely connected to a single hidden layer composed of 100 neurons, which implement the ReLU Activation Function. This hidden layer is densely connected to the final output layer, which implements the Softmax Activation Function. The number of neurons in the output layer is equivalent to the number of classes present for classification.

The model hyperparameters are detailed below:

Hyperparameter Value
Optimizer (name) Adam
Beta_1 0.9
Beta_2 0.999
Epsilon 1e-07
Amsgrad False
Loss Sparse Categorical Crossentropy

Logistic Regression

For the Logistic Regression Model, AutoML uses SciKit Learn’s Logistic RegressionOpens in a new tab class.

The model hyperparameters are detailed below:

Hyperparameter Value
Penalty L2
Dual Formulation False
Tolerance 1e-4
C (Inverse Regularization Parameter) 1
Fit Intercept True
Intercept Scaling 1
Class Weight Balanced
Solver liblinear
Max Iterations 100
Multiclass One-vs-Rest
Warm Start False
Number of Jobs 1

Random Forest Classifier

For the Random Forest Classifier model, AutoML uses SciKit Learn’s Random Forest ClassifierOpens in a new tab class.

The model hyperparameters are detailed below:

Hyperparameter Value
Number of Estimators 100
Criterion Gini Impurity
Max Depth None
Min Samples to Split 2
Min Samples to be Leaf Node 1
Min Fraction of Total Sum of Weights to be Leaf 0
Max Features Square root of number of features
Max Leaf Nodes None
Min Impurity Decrease for Split 0
Bootstrap True
Number of Jobs 1
Warn Start False
Class Weight Balanced
FeedbackOpens in a new tab