Logic Journal of IGPL Advance Access published online on September 14, 2009
Logic Journal of IGPL, doi:10.1093/jigpal/jzp045
Semantic learning in autonomously active recurrent neural networks
Institute for Theoretical Physics, Goethe University Frankfurt, 60054 Frankfurt/Main, Germany.
E-mail: gros07{at}itp.uni-frankfurt.de
 |
Abstract |
|---|
The human brain is autonomously active, being characterized by a self-sustained neural activity which would be present even in the absence of external sensory stimuli. Here we study the interrelation between the self-sustained activity in autonomously active recurrent neural nets and external sensory stimuli.
There is no a priori semantical relation between the influx of external stimuli and the patterns generated internally by the autonomous and ongoing brain dynamics. The question then arises when and how are semantic correlations between internal and external dynamical processes learned and built up?
We study this problem within the paradigm of transient state dynamics for the neural activity in recurrent neural nets, i.e. for an autonomous neural activity characterized by an infinite time-series of transiently stable attractor states. We propose that external stimuli will be relevant during the sensitive periods, viz the transition period between one transient state and the subsequent semi-stable attractor. A diffusive learning signal is generated unsupervised whenever the stimulus influences the internal dynamics qualitatively.
For testing we have presented to the model system stimuli corresponding to the bars and stripes problem. We found that the system performs a non-linear independent component analysis on its own, being continuously and autonomously active. This emergent cognitive capability results here from a general principle for the neural dynamics, the competition between neural ensembles.
Key Words: recurrent neural networks • autonomous neural dynamics • transient state dynamics • emergent cognitive capabilities