Algorithms Used
AutoML uses four different models to make predictions.
For regression models:
For classification models:
XGBRegressor
For regression models, AutoML uses XGBoost’s XGBRegressorOpens in a new tab class.
The model hyperparameters are detailed below:
| Hyperparameter | Value |
|---|---|
| Max Depth | 3 |
| Learning Rate | 0.1 |
| Number of Estimators | 100 |
| Objective | Squared Error |
| Booster | Gbtree |
| Tree Method | Auto |
| Number of Jobs | 1 |
| Gamma (min loss reduction for partition on leaf) | 0 |
| Min Child Weight | 1 |
| Max Delta Step | 0 |
| L2 Regularization Lambda | 1 |
| Scale Positive Weight | 1 |
| Base/Initial Score | 0.5 |
Neural Network
For the Neural Network model, AutoML uses TensorFlow with KerasOpens in a new tab as a wrapper.
The input layer has its size based on the number of features. This layer is then densely connected to a single hidden layer composed of 100 neurons, which implement the ReLU Activation Function. This hidden layer is densely connected to the final output layer, which implements the Softmax Activation Function. The number of neurons in the output layer is equivalent to the number of classes present for classification.
The model hyperparameters are detailed below:
| Hyperparameter | Value |
|---|---|
| Optimizer (name) | Adam |
| Beta_1 | 0.9 |
| Beta_2 | 0.999 |
| Epsilon | 1e-07 |
| Amsgrad | False |
| Loss | Sparse Categorical Crossentropy |
Logistic Regression
For the Logistic Regression Model, AutoML uses SciKit Learn’s Logistic RegressionOpens in a new tab class.
The model hyperparameters are detailed below:
| Hyperparameter | Value |
|---|---|
| Penalty | L2 |
| Dual Formulation | False |
| Tolerance | 1e-4 |
| C (Inverse Regularization Parameter) | 1 |
| Fit Intercept | True |
| Intercept Scaling | 1 |
| Class Weight | Balanced |
| Solver | liblinear |
| Max Iterations | 100 |
| Multiclass | One-vs-Rest |
| Warm Start | False |
| Number of Jobs | 1 |
Random Forest Classifier
For the Random Forest Classifier model, AutoML uses SciKit Learn’s Random Forest ClassifierOpens in a new tab class.
The model hyperparameters are detailed below:
| Hyperparameter | Value |
|---|---|
| Number of Estimators | 100 |
| Criterion | Gini Impurity |
| Max Depth | None |
| Min Samples to Split | 2 |
| Min Samples to be Leaf Node | 1 |
| Min Fraction of Total Sum of Weights to be Leaf | 0 |
| Max Features | Square root of number of features |
| Max Leaf Nodes | None |
| Min Impurity Decrease for Split | 0 |
| Bootstrap | True |
| Number of Jobs | 1 |
| Warn Start | False |
| Class Weight | Balanced |
