Try to find an optimizer which matches the characteristics of your dataset, training setup, and goal of the project. Certain optimizers perform extraordinarily well on data with sparse features [13] and others may perform better when the model is applied to previously unseen data [14].