The best method to extract the last juice out of your deep learning or machine learning models is to select the correct **hyperparameters. **With the right choice, you can tailor the behavior of the algorithm to your particular dataset. It’s important to note that **hyperparameter**s are different from parameters. The model estimates the parameters from the given data, for instance, the weights of a DNN (deep neural network). But the model can’t estimate **hyperparameters** from the given data. Rather, the practitioner specifies the **hyperparameter**s when configuring the model, such as the learning rate of a DNN (deep neural network).

Usually, knowing what values you should use for the **hyperparameter**s of a specific algorithm on a given dataset is challenging. That’s why you need to explore various strategies to tune **hyperparameter** values.

With

hyperparametertuning, you can determine the right mix ofhyperparameters that would maximize the performance of your model.

**Hyperparameter tuning**

The two best strategies in use for **hyperparameter** tuning are:

**1. GridSearch**

It involves creating a grid of probable values for **hyperparameter**s. Every iteration tries a set of **hyperparameter**s in a particular order from the grid of probable **hyperparameter** values. The **GridSearch **strategy will build several versions of the model with all probable combinations of **hyperparameter**s, and return the one with the best performance.

Since **GridSearch** goes through all the intermediate sets of **hyperparameter**s, it’s an extremely expensive strategy computationally.

**2. RandomizedSearch**

It also involves building a grid of probable values for **hyperparameter**s but here, every iteration tries a random set of **hyperparameter**s from the grid, documents the performance, and finally, returns the set of **hyperparameter**s that provided the best performance.

As **RandomizedSearch **moves through a fixed number of **hyperparameter** settings, it decreases unnecessary computations and the associated costs, thus offering a solution to overcome the drawbacks of **GridSearch**.

**Selecting the hyperparameters to tune**

The more **hyperparameter**s of an algorithm you want to tune, the slower would be the tuning process. This makes it important to choose a minimum subset of **hyperparameter**s to search or tune. But not all **hyperparameter**s are equally important. Also, you’ll find little universal advice on how to select the **hyperparameter**s that you should tune.

Having experience with the

machine learningtechnique you’re using could give you useful insights into the behavior of itshyperparameters, which could make your choice a bit easier. You may even turn tomachine learningcommunities to seek advice. But whatever your choice is, you should realize the implications.

Each **hyperparameter** that you select to tune will have the possibility of increasing the number of trials necessary for completing the tuning task successfully. And when you use an AI Platform Training to train your model, you’ll be charged for the task’s duration, which means choosing the **hyperparameter**s to tune carefully would decrease both the time and training cost of your model.

**Final words**

For a good start with **hyperparameter **tuning models, you can go with scikit-learn though there are better options too for **hyperparameter **tuning and optimization, such as Hyperopt, Optuna, Scikit-Optimize, and Ray-Tune, to name a few.

. . .

To learn more about variance and bias, * click here* and read our another article.