Exploring Transfer Functions
For years many talented computer scientists have been working on a variety of neural networks. In general our basic notion has been that the signaling between nodes should be similar to biological models. And from our experience some of this in part is true, and in many ways limits how we think about networks and how to better use them to solve the greatest challenges of this century. New data on long term memory processes has revealed that many processes in nature are predictable, and therefore within our capability to understand. Included are a few new ideas on transfer functions, a primary function of signaling in a neural network.
One of the first ideas was to extend the response phase or build envelopes:
f0(x):= tanh(x)
f1(x):= (f0(x-1)+f0(x))/2, Lower Envelope
f2(x):= (f0(x)+f0(x+1))/2, Upper Envelope
In addition to adapting neural weights, the response functions can also change their frequency by increasing their range:
f3(x):= (f0(x)+f0(x+1))/2
f4(x):= (f0(x)+f0(x+1))/2.5
f5(x):= (f0(x)+f0(x+1))/3.5
These are just a few practical ideas to be used in breve.