FURI | Spring 2019

Alpha-Loss

Alpha-loss is a tunable loss function with a parameter alpha that bridges log-loss for (alpha = 1) and 0-1 loss for (alpha = infinity). In machine learning literature, the theoretically optimal loss function is the 0-1 loss function. Unfortunately, 0-1 loss is computationally intractable due to being non-differentiable and discontinuous. Thus, it will be extremely beneficial to continue exploring such a surrogate loss function and how well it can emulate 0-1 loss. Discovering such desirable properties of this loss function will provide engineers and machine learning enthusiast better performance and more variability in neural network designs.