Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

To be fair, AI researchers used strictly differentiable functions (which is required for back-propagation) until recently. For example lenet5 uses the logistic function.

Only in 2011, some smart-asses [1] :) experimented with rectifier units and discovered they're even better

[1] Xavier Glorot, Antoine Bordes and Yoshua Bengio - Deep sparse rectifier neural networks (2011)



Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: