Hopfield net

From Psychology Wiki

Community portal · Tasks to do · News · Help

Clinical · Educational · Ind&Org · Other fields · Professional · Transpersonal · World

Biological: Behavioural genetics · Evolutionary psychology · Neuroanatomy · Neurochemistry · Neuroendocrinology · Psychoneuroimmunology · Physiological Psychology · Psychopharmacology

A Hopfield net is a form of recurrent artificial neural network invented by John Hopfield. Hopfield nets serve as content-addressable memory systems with binary threshold units. They are guaranteed to converge to a stable state.

[edit] Structure

The units in Hopfield nets are binary threshold units, i.e. the units only take on two different values for their states and the value is determined by whether or not the units' input exceeds their threshold. Hopfield nets can either have units that take on values of 1 or -1, or units that take on values of 1 or 0. So, the two possible definitions for unit i's activation, , are:

(1)

(2)

Where:

is the connection weight from unit j to unit i.
is the state of unit j.
is the threshold of unit i.

The connections in a Hopfield net have two restrictions on them:

. (No unit has a connection with itself.)
. (All connections are symmetric.)

Hopfield nets have a scalar value associated with each state of the network referred to as the "energy", E, of the network, where:

WikiTeX: latex reported a failure, namely:

This is pdfeTeX, Version 3.141592-1.21a-2.2 (Web2C 7.5.4)
entering extended mode
(./5f3770e5f68a9314afed5e0f76615
LaTeX2e <2003/12/01>
Babel  and hyphenation patterns for american, french, german, ngerman, b
ahasa, basque, bulgarian, catalan, croatian, czech, danish, dutch, esperanto, e
stonian, finnish, greek, icelandic, irish, italian, latin, magyar, norsk, polis
h, portuges, romanian, russian, serbian, slovak, slovene, spanish, swedish, tur
kish, ukrainian, nohyphenation, loaded.
(/usr/share/texmf/tex/latex/base/article.cls
Document Class: article 2004/02/16 v1.4f Standard LaTeX document class
(/usr/share/texmf/tex/latex/base/size10.clo))
(/usr/share/texmf/tex/latex/amsfonts/amssymb.sty
(/usr/share/texmf/tex/latex/amsfonts/amsfonts.sty))
(/usr/share/texmf/tex/latex/amsmath/amsmath.sty
For additional information on amsmath, use the `?' option.
(/usr/share/texmf/tex/latex/amsmath/amstext.sty
(/usr/share/texmf/tex/latex/amsmath/amsgen.sty))
(/usr/share/texmf/tex/latex/amsmath/amsbsy.sty)
(/usr/share/texmf/tex/latex/amsmath/amsopn.sty))
(/usr/share/texmf/tex/latex/amsmath/amscd.sty)
(/usr/share/texmf/tex/latex/concmath/concmath.sty)
(./5f3770e5f68a9314afed5e0f76615.aux)
(/usr/share/texmf/tex/latex/concmath/ot1ccr.fd)
(/usr/share/texmf/tex/latex/concmath/omlccm.fd)
(/usr/share/texmf/tex/latex/concmath/omsccsy.fd)
(/usr/share/texmf/tex/latex/concmath/omxccex.fd)
(/usr/share/texmf/tex/latex/amsfonts/umsa.fd)
(/usr/share/texmf/tex/latex/amsfonts/umsb.fd)
! Missing } inserted.
 
                }
l.5 \begin{equation*}E = -\sum_{i\end{equation*}
                                                
[1] (./5f3770e5f68a9314afed5e0f76615.aux) )
(see the transcript file for additional information)
Output written on 5f3770e5f68a9314afed5e0f76615.dvi (1 page, 428 bytes).
Transcript written on 5f3770e5f68a9314afed5e0f76615.log.

This value is called the "energy" because the definition ensures that if units are randomly chosen to update their activations the network will converge to states which are local minima in the energy function (which is considered to be a Lyapunov function). Thus, if a state is a local minimum in the energy function it is a stable state for the network.

[edit] Training

Training a Hopfield net involves lowering the energy of states that the net should "remember". This allows the net to serve as a content addressable memory system, that is to say, the network will converge to a "remembered" state if it is given only part of the state. For example, if we train a Hopfield net with five units so that the state (1, 0, 1, 0, 1) is an energy minimum, and we give the network the state (1, 0, 0, 0, 1) it will converge to (1, 0, 1, 0, 1). Thus, the network is properly trained when the energy of states which the network should remember are local minima.

[edit] References

J. J. Hopfield, "Neural networks and physical systems with emergent collective computational abilities", Proceedings of National Academy of Sciences, vol. 79 no. 8 pp. 2554–2558, April 1982. PNAS Reprint (Abstract) PNAS Reprint (PDF)