Wikia

Psychology Wiki

Generalized Hebbian Algorithm

Talk0
34,117pages on
this wiki
Revision as of 01:31, July 2, 2013 by Dr Joe Kiff (Talk | contribs)

(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)

Assessment | Biopsychology | Comparative | Cognitive | Developmental | Language | Individual differences | Personality | Philosophy | Social |
Methods | Statistics | Clinical | Educational | Industrial | Professional items | World psychology |

Statistics: Scientific method · Research methods · Experimental design · Undergraduate statistics courses · Statistical tests · Game theory · Decision theory


The Generalized Hebbian Algorithm (GHA), also known in the literature as Sanger's rule, is a linear feedforward neural network model for unsupervised learning with applications primarily in principal components analysis. First defined in 1989,[1] it is similar to Oja's rule in its formulation and stability, except it can be applied to networks with multiple outputs.

TheoryEdit

GHA combines Oja's rule with the Gram-Schmidt process to produce a learning rule of the form

\,\Delta w_{ij} ~ = ~ \eta\left(y_j x_i - y_j \sum_{k=1}^j w_{ik} y_k \right),

where wij defines the synaptic weight or connection strength between the ith input and jth output neurons, x and y are the input and output vectors, respectively, and η is the learning rate parameter.

DerivationEdit

In matrix form, Oja's rule can be written

\,\frac{d w(t)}{d t} ~ = ~ w(t) Q - \mathrm{diag} [w(t) Q w(t)^{\mathrm{T}}] w(t),

and the Gram-Schmidt algorithm is

\,\Delta w(t) ~ = ~ -\mathrm{lower} [w(t) w(t)^{\mathrm{T}}] w(t),

where w(t) is any matrix, in this case representing synaptic weights, Q = η x xT is the autocorrelation matrix, simply the outer product of inputs, diag is the function that diagonalizes a matrix, and lower is the function that sets all matrix elements on or above the diagonal equal to 0. We can combine these equations to get our original rule in matrix form,

\,\Delta w(t) ~ = ~ \eta(t) \left(\mathbf{y}(t) \mathbf{x}(t)^{\mathrm{T}} - \mathrm{LT}[\mathbf{y}(t)\mathbf{y}(t)^{\mathrm{T}}] w(t)\right),

where the function LT sets all matrix elements above the diagonal equal to 0, and note that our output y(t) = w(t) x(t) is a linear neuron.[1]

Stability and PCAEdit

[2] [3]

ApplicationsEdit

GHA is used in applications where a self-organizing map is necessary, or where a feature or principal components analysis can be used. Examples of such cases include artificial intelligence and speech and image processing.

Its importance comes from the fact that learning is a single-layer process—that is, a synaptic weight changes only depending on the response of the inputs and outputs of that layer, thus avoiding the multi-layer dependence associated with the backpropagation algorithm. It also has a simple and predictable trade-off between learning speed and accuracy of convergence as set by the learning rate parameter η.[2]

See alsoEdit

ReferencesEdit

  1. 1.0 1.1 Sanger, Terence D. (1989). Optimal unsupervised learning in a single-layer linear feedforward neural network. Neural Networks 2 (6): 459–473.
  2. 2.0 2.1 Haykin, Simon (1998). Neural Networks: A Comprehensive Foundation, 2, Prentice Hall.
  3. Oja, Erkki (November 1982). Simplified neuron model as a principal component analyzer. Journal of Mathematical Biology 15 (3): 267–273. BF00275687.
This page uses Creative Commons Licensed content from Wikipedia (view authors).
Advertisement | Your ad here

Around Wikia's network

Random Wiki