My Brief Adventures in Neural Networks: Merely an interpolation system which cannot reliably extrapolate?
I used to think neural networks were a reasonably good technology, until I decided to try setting up a neural network myself. After some experimentation with neural networks, I dramatically reversed my stance. I now believe neural networks should not be in charge of any decisions such that could have life-or-death impact. Find out why below. Disclosure: These are my personal thoughts about neural networks based on my limited experience with them. I do not claim to be an expert in neural networks. First I'm going to propose a definition for a neural network: An interpolation system whose behavior becomes undefined when extrapolating beyond its training data. In simple language, "undefined" may be thought of as "random by human standards". OK, how did this definition come about? It all started when I desired to make a system for classifying fonts a year or two ago. Even Google Fonts had at the time very few parameters for manually finding suitable fonts. For pick
