Wikia

Psychology Wiki

Changes: Eigenvalue, eigenvector and eigenspace

Edit

Back to page

 
Line 1: Line 1:
 
{{StatsPsy}}
 
{{StatsPsy}}
[[Image:Mona Lisa with eigenvector.png|thumb|270px|Fig. 1. In this [[shear (mathematics)|shear]] transformation of the [[Mona Lisa]], the picture was deformed in such a way that its central vertical axis (red vector) was not modified, but the diagonal vector (blue) has changed direction. Hence the red vector is an '''eigenvector''' of the transformation and the blue vector is not. Since the red vector was neither stretched nor compressed, its '''eigenvalue''' is 1. All vectors with the same vertical direction - i.e. parallel to this vector - are also eigenvectors, with the same eigenvalue. Together with the zero-vector, they form the '''eigenspace''' for this eigenvalue.]]
+
[[Image:Mona Lisa with eigenvector.png|thumb|270px|Fig. 1. In this [[shear mapping]] of the Mona Lisa, the picture was deformed in such a way that its central vertical axis (red vector) was not modified, but the diagonal vector (blue) has changed direction. Hence the red vector is an '''eigenvector''' of the transformation and the blue vector is not. Since the red vector was neither stretched nor compressed, its '''eigenvalue''' is 1. All vectors with the same vertical direction - i.e., parallel to this vector - are also eigenvectors, with the same eigenvalue. Together with the zero-vector, they form the '''eigenspace''' for this eigenvalue.]]
   
In [[mathematics]], an {{Audio|De-eigenvector.ogg|'''eigenvector''' }} of a [[linear transformation|transformation]]<ref>In this context, only [[linear transformation]]s from a [[vector space]] to itself are considered.</ref> is a [[vector space|vector]] which, in the transformation, is multiplied by a constant factor, called the {{Audio|De-eigenvalue.ogg|'''eigenvalue'''}} of that vector. (See Fig. 1). Often, a transformation is completely described by its eigenvalues and eigenvectors. The '''eigenspace''' for a factor is the [[set (mathematics)|set]] of eigenvectors with that factor as eigenvalue.
+
In [[mathematics]], a [[Vector (spatial)|vector]] may be thought of as an arrow. It has a length, called its ''magnitude'', and it points in some particular ''direction''. A [[linear transformation]] may be considered to operate on a vector to change it, usually changing both its magnitude and its direction. An '''eigenvector''' of a given linear transformation is a vector which is multiplied by a constant called the '''eigenvalue''' during that transformation. The direction of the eigenvector is either unchanged by that transformation (for positive eigenvalues) or reversed (for negative eigenvalues).
   
In the specific case of [[linear algebra]], the ''eigenvalue problem'' is as follows. Given a matrix <math>A</math>, are there nonzero vectors <math>x</math> such that <math>Ax</math> is a multiple of <math>x</math>? If so, then the factor by which <math>x</math> is multiplied is called an ''eigenvalue'' of the matrix <math>A</math>, while <math>x</math> is called an ''eigenvector'' of <math>A</math> corresponding to this eigenvalue.
+
For example, an eigenvalue of +2 means that the eigenvector is doubled in length and points in the same direction. An eigenvalue of +1 means that the eigenvector is unchanged, while an eigenvalue of &minus;1 means that the eigenvector is reversed in direction. An '''eigenspace''' of a given transformation is the span of the eigenvectors of that transformation with the same eigenvalue, together with the zero vector (which has no direction). An '''eigenspace''' is an example of a [[linear subspace|subspace]] of a [[vector space]].
   
These concepts play a major role in several branches of both [[pure mathematics|pure]] and [[applied mathematics]] &mdash; appearing prominently in [[linear algebra]], [[functional analysis]], and to a lesser extent in [[nonlinear]] situations.
+
In [[linear algebra]], every linear transformation between finite-dimensional vector spaces can be given by a [[matrix (mathematics)|matrix]], which is a rectangular array of numbers arranged in rows and columns. Standard methods for finding '''eigenvalues''', '''eigenvectors''', and '''eigenspaces''' of a given matrix are discussed below.
   
It is common to prefix any natural name for the vector with ''eigen'' instead of saying ''eigenvector''. For example, ''eigenfunction'' if the eigenvector is a [[function (mathematics)|function]], ''eigenmode'' if the eigenvector is a [[harmonic mode]], ''eigenstate'' if the eigenvector is a [[quantum state]], and so on (e.g. the ''eigenface'' example below). Similarly for the eigenvalue, e.g. ''eigenfrequency'' if the eigenvalue is (or determines) a [[frequency]].
+
These concepts play a major role in several branches of both [[pure mathematics|pure]] and [[applied mathematics]] &mdash; appearing prominently in [[linear algebra]], [[functional analysis]], and to a lesser extent in [[nonlinear]] mathematics.
  +
  +
Many kinds of mathematical objects can be treated as vectors: [[function (mathematics)|functions]], [[harmonic|harmonic modes]], quantum states, and [[frequency|frequencies]], for example. In these cases, the concept of ''direction'' loses its ordinary meaning, and is given an abstract definition. Even so, if this abstract ''direction'' is unchanged by a given linear transformation, the prefix "eigen" is used, as in ''[[eigenfunction]]'', ''[[eigenmode]]'', ''eigenstate'', and ''eigenfrequency''.
   
 
==History==
 
==History==
Presently, eigenvalues are often introduced in the context of [[matrix theory]]. Historically, however, they arose in the study of [[quadratic form]]s and [[differential equation]]s.
+
Eigenvalues are often introduced in the context of [[linear algebra]] or [[matrix theory]]. Historically, however, they arose in the study of [[quadratic form]]s and [[differential equation]]s.
   
In the first half of the 18th century, [[Johann Bernoulli|Johann]] and [[Daniel Bernoulli]], [[Jean le Rond d'Alembert|d'Alembert]] and [[Leonhard Euler|Euler]] encountered eigenvalue problems when studying the motion of a rope, which they considered to be a weightless string loaded with a number of masses. [[Pierre-Simon Laplace|Laplace]] and [[Joseph Louis Lagrange|Lagrange]] continued their work in the second half of the century. They realized that the eigenvalues are related to the stability of the motion. They also used eigenvalue methods in their study of the [[solar system]].<ref>See Hawkins (1975), §2; Kline (1972), pp. 807+808.</ref>
+
Euler had also studied the rotational motion of a [[rigid body]] and discovered the importance of the [[moment of inertia|principal axes]]. As Lagrange realized, the principal axes are the eigenvectors of the inertia matrix.<ref>See Hawkins (1975), §2.</ref> In the early 19th century, [[Augustin Louis Cauchy|Cauchy]] saw how their work could be used to classify the [[quadric surface]]s, and generalized it to arbitrary dimensions.<ref name="hawkins3">See Hawkins (1975), §3.</ref> Cauchy also coined the term ''racine caractéristique'' (characteristic root) for what is now called ''eigenvalue''; his term survives in ''[[characteristic equation]]''.<ref name="kline807">See {{Harvnb|Kline|1972|loc=pp. 807-808}}</ref>
   
Euler had also studied the rotational motion of a [[rigid body]] and discovered the importance of the [[moment of inertia|principal axes]]. As Lagrange realized, the principal axes are the eigenvectors of the inertia matrix.<ref>See Hawkins (1975), §2.</ref> In the early 19th century, [[Augustin Louis Cauchy|Cauchy]] saw how their work could be used to classify the [[quadric surface]]s, and generalized it to arbitrary dimensions.<ref name="hawkins3">See Hawkins (1975), §3.</ref> Cauchy also coined the term ''racine caractéristique'' (characteristic root) for what is now called ''eigenvalue''; his term survives in ''[[characteristic equation]]''.<ref name="kline807">See Kline (1972), pp. 807+808.</ref>
+
[[Joseph Fourier|Fourier]] used the work of Laplace and Lagrange to solve the [[heat equation]] by [[separation of variables]] in his famous 1822 book ''Théorie analytique de la chaleur''.<ref>See {{Harvnb|Kline|1972|loc=p. 673}}</ref> [[Jacques Charles François Sturm|Sturm]] developed Fourier's ideas further and he brought them to the attention of Cauchy, who combined them with his own ideas and arrived at the fact that symmetric matrices have real eigenvalues.<ref name="hawkins3"/> This was extended by [[Charles Hermite|Hermite]] in 1855 to what are now called [[Hermitian matrix|Hermitian matrices]].<ref name="kline807"/> Around the same time, [[Francesco Brioschi|Brioschi]] proved that the eigenvalues of [[orthogonal matrix|orthogonal matrices]] lie on the unit circle,<ref name="hawkins3"/> and [[Alfred Clebsch|Clebsch]] found the corresponding result for [[skew-symmetric matrix|skew-symmetric matrices]].<ref name="kline807"/> Finally, [[Karl Weierstrass|Weierstrass]] clarified an important aspect in the [[stability theory]] started by Laplace by realizing that [[defective matrix|defective matrices]] can cause instability.<ref name="hawkins3"/>
   
[[Joseph Fourier|Fourier]] used the work of Laplace and Lagrange to solve the [[heat equation]] by [[separation of variables]] in his famous 1822 book ''Théorie analytique de la chaleur''.<ref>See Kline (1972), p. 673.</ref> [[Jacques Charles François Sturm|Sturm]] developed Fourier's ideas further and he brought them to the attention of Cauchy, who combined them with his own ideas and arrived at the fact that symmetric matrices have real eigenvalues.<ref name="hawkins3"/> This was extended by [[Charles Hermite|Hermite]] in 1855 to what are now called [[Hermitian matrix|Hermitian matrices]].<ref name="kline807"/> Around the same time, [[Francesco Brioschi|Brioschi]] proved that the eigenvalues of [[orthogonal matrix|orthogonal matrices]] lie on the unit circle,<ref name="hawkins3"/> and [[Alfred Clebsch|Clebsch]] found the corresponding result for [[skew-symmetric matrix|skew-symmetric matrices]].<ref name="kline807"/> Finally, [[Karl Weierstrass|Weierstrass]] clarified an important aspect in the stability theory started by Laplace by realizing that [[defective matrix|defective matrices]] can cause instability.<ref name="hawkins3"/>
+
In the meantime, [[Joseph Liouville|Liouville]] studied eigenvalue problems similar to those of Sturm; the discipline that grew out of their work is now called ''[[Sturm-Liouville theory]]''.<ref>See {{Harvnb|Kline|1972|loc=pp. 715-716}}</ref> [[Hermann Schwarz|Schwarz]] studied the first eigenvalue of [[Laplace's equation]] on general domains towards the end of the 19th century, while [[Henri Poincaré|Poincaré]] studied [[Poisson's equation]] a few years later.<ref>See {{Harvnb|Kline|1972|loc=pp. 706-707}}</ref>
   
In the meantime, [[Joseph Liouville|Liouville]] had studied similar eigenvalue problems as Sturm; the discipline that grew out of their work is now called ''[[Sturm-Liouville theory]]''.<ref>See Kline (1972), pp. 715+716.</ref> [[Hermann Schwarz|Schwarz]] studied the first eigenvalue of [[Laplace's equation]] on general domains towards the end of the 19th century, while [[Henri Poincaré|Poincaré]] studied [[Poisson's equation]] a few years later.<ref>See Kline (1972), pp. 706+707.</ref>
+
At the start of the 20th century, [[David Hilbert|Hilbert]] studied the eigenvalues of [[integral operator]]s by viewing the operators as infinite matrices.<ref>See {{Harvnb|Kline|1972|loc=p. 1063}}</ref> He was the first to use the [[German language|German]] word ''eigen'' to denote eigenvalues and eigenvectors in 1904, though he may have been following a related usage by [[Helmholtz]]. "Eigen" can be translated as "own", "peculiar to", "characteristic" or "individual"&mdash;emphasizing how important eigenvalues are to defining the unique nature of a specific transformation. For some time, the standard term in English was "proper value", but the more distinctive term "eigenvalue" is standard today.<ref>See Aldrich (2006).</ref>
   
At the start of the 20th century, [[David Hilbert|Hilbert]] studied the eigenvalues of [[integral operator]]s by considering them to be infinite matrices.<ref>See Kline (1972), p. 1063.</ref> He was the first to use the [[German language|German]] word ''eigen'' to denote eigenvalues and eigenvectors in 1904, though he may have been following a related usage by [[Helmholtz]]. "Eigen" can be translated as "own", "peculiar to", "characteristic" or "individual"&mdash;emphasizing how important eigenvalues are to defining the unique nature of a specific transformation. For some time, the standard term in English was "proper value", but the more distinctive term "eigenvalue" is standard today.<ref>See Aldrich (2006).</ref>
+
The first numerical algorithm for computing eigenvalues and eigenvectors appeared in 1929, when [[Richard Edler von Mises|Von Mises]] published the [[power method]]. One of the most popular methods today, the [[QR algorithm]], was proposed independently by [[J.G.F. Francis|Francis]] and [[Vera Kublanovskaya|Kublanovskaya]] in 1961.<ref> See {{Harvnb|Golub|van Loan|1996|loc=§7.3}}; {{Harvnb|Meyer|2000|loc=§7.3}}</ref>
   
The first numerical algorithm for computing eigenvalues and eigenvectors appeared in 1929, when [[Richard Edler von Mises|Von Mises]] published the [[power method]]. One of the most popular methods today, the [[QR algorithm]], was proposed independently by [[J.G.F. Francis|Francis]] and [[Vera Kublanovskaya|Kublanovskaya]] in 1961.<ref> See Golub and Van Loan (1996), §7.3; Meyer (2000), §7.3.</ref>
+
==Definitions: the eigenvalue equation==
 
==Definitions==
 
 
{{see also|Eigenplane}}
 
{{see also|Eigenplane}}
[[Linear transformation]]s of space&mdash;such as [[rotation]], [[Reflection (mathematics)|reflection]], [[stretching]], [[Physical compression|compression]], shear or any combination of these&mdash;may be visualized by the effect they produce on [[vector (spatial)|vector]]s. Vectors can be visualized as arrows pointing from one [[point (geometry)|point]] to another.
+
[[Linear transformation]]s of a vector space, such as [[Rotation (mathematics)|rotation]], [[Reflection (mathematics)|reflection]], [[stretching]], [[Physical compression|compression]], [[shear (mathematics)|shear]] or any combination of these, may be visualized by the effect they produce on [[Vector (spatial)|vector]]s. In other words, they are vector functions. More formally, in a vector space ''L'' a vector function ''A'' is defined if for each vector '''x''' of ''L'' there corresponds a unique vector '''y''' = ''A''('''x''') of ''L''. For the sake of brevity, the parentheses around the vector on which the transformation is acting are often omitted. A vector function ''A'' is ''linear'' if it has the following two properties:
  +
:[[additive function|''additivity'']] <math>\ A(\mathbf{x}+\mathbf{y})=A(\mathbf{x})+A(\mathbf{y})</math>
  +
:[[homogenous function|''homogeneity'']] <math>\ A(\alpha \mathbf{x})=\alpha A(\mathbf{x})</math>
  +
where '''x''' and '''y''' are any two vectors of the vector space ''L'' and α is any [[real number]]. Such a function is variously called a ''linear transformation'', ''[[linear operator]]'', or ''linear [[endomorphism]]'' on the space ''L''.
  +
{| class="wikitable"
  +
|
  +
Given a linear transformation ''A'', a non-zero vector '''x''' is defined to be an '''''eigenvector''''' of the transformation if it satisfies the '''eigenvalue equation''' <math>A \mathbf{x} = \lambda \mathbf{x}</math> for some [[scalar (mathematics)|scalar]] λ. In this situation, the scalar λ is called an '''''eigenvalue''''' of ''A'' corresponding to the eigenvector '''x'''.
  +
|}
  +
The key equation in this definition is the eigenvalue equation, ''A'''''x''' = λ'''x'''. Most vectors '''x''' will not satisfy such an equation. A typical vector '''x''' changes direction when acted on by ''A'', so that ''A'''''x''' is not a multiple of '''x'''. This means that only certain special vectors '''x''' are eigenvectors, and only certain special numbers λ are eigenvalues. Of course, if ''A'' is a multiple of the [[identity matrix]], then no vector changes direction, and all non-zero vectors are eigenvectors. But in the usual case, eigenvectors are few and far between. They are the "normal modes" of the system, and they act independently.<ref>See {{Harvnb|Strang|2006|loc=p. 249}}</ref>
  +
  +
The requirement that the eigenvector be non-zero is imposed because the equation ''A'''''0''' = λ'''0''' holds for every ''A'' and every λ. Since the equation is always trivially true, it is not an interesting case. In contrast, an eigenvalue can be zero in a nontrivial way. An eigenvalue can be, and usually is, also a [[complex number]]. In the definition given above, eigenvectors and eigenvalues do not occur independently. Instead, each eigenvector is associated with a specific eigenvalue. For this reason, an eigenvector '''x''' and a corresponding eigenvalue λ are often referred to as an ''eigenpair''. One eigenvalue can be associated with several or even with
  +
infinite number of eigenvectors. But conversely, if an eigenvector is given, the
  +
associated eigenvalue for this eigenvector is unique. Indeed, from the equality
  +
''A'''''x''' = λ'''x''' = λ''''x''' and from '''x''' ≠ '''0''' it follows that λ = λ'.<ref>See {{Harvnb|Sharipov|1996|loc=p. 66}}</ref>
  +
[[Image:Eigenvalue equation.svg|thumb|left|250px|Fig. 2. The eigenvalue equation as a homothety (similarity transformation) on the vector '''x'''.]]
  +
Geometrically (Fig. 2), the eigenvalue equation means that under the transformation ''A'' eigenvectors experience only changes in magnitude and sign &mdash; the direction of ''A'''''x''' is the same as that of '''x'''. This type of linear transformation is defined as [[homothety]] (dilatation<ref>See {{Harvnb|Bowen|Wang|1980|loc=p. 148}}</ref>, similarity transformation). The eigenvalue λ is simply the amount of "stretch" or "shrink" to which a vector is subjected when transformed by ''A''. If λ = 1, the vector remains unchanged (unaffected by the transformation). A transformation ''I'' under which a vector '''x''' remains unchanged, ''I'''''x''' = '''x''', is defined as [[identity transformation]]. If λ = –1, the vector flips to the opposite direction (rotates to 180°); this is defined as [[Reflection (linear algebra)|reflection]].
  +
  +
If '''x''' is an eigenvector of the linear transformation ''A'' with eigenvalue λ, then any vector '''y''' = α'''x''' is also an eigenvector of ''A'' with the same eigenvalue. From the homogeneity of the transformation ''A'' it follows that ''A'''''y''' = α(''A'''''x''') = α(λ'''x''') = λ(α'''x''') = λ'''y'''. Similarly, using the additivity property of the linear transformation, it can be shown that any linear combination of eigenvectors with eigenvalue λ has the same eigenvalue λ.<ref>For a proof of this lemma, see {{Harvnb|Shilov|1969|loc=p. 131}}, and [[Wikibooks:The Book of Mathematical Proofs/Algebra/Linear Transformations#Lemma for the eigenspace|Lemma for the eigenspace]]</ref> Therefore, any non-zero vector in the line through '''x''' and the zero vector is an eigenvector with the same eigenvalue as '''x'''. Together with the zero vector, those eigenvectors form a subspace of the vector space called an ''eigenspace''. The eigenvectors corresponding to different eigenvalues are linearly independent<ref name="Shilov_lemma">For a proof of this lemma, see {{Harvnb|Shilov|1969|loc=p. 130}}, {{Harvnb|Hefferon|2001|loc=p. 364}}, and [[Wikibooks:The Book of Mathematical Proofs/Algebra/Linear Transformations#Lemma for linear independence of eigenvectors|Lemma for linear independence of eigenvectors]]</ref> meaning, in particular, that in an ''n''-dimensional space the linear transformation ''A'' cannot have more than ''n'' eigenvectors with different eigenvalues.<ref>See {{Harvnb|Shilov|1969|loc=p. 131}}</ref> The vectors of the eigenspace generate a [[linear subspace]] of ''A'' which is invariant (unchanged) under this transformation.<ref>For proof, see {{Harvnb|Sharipov|1996|loc=Theorem 4.4 on p. 68}}</ref>
  +
  +
If a [[basis (linear algebra)|basis]] is defined in vector space L<sub>''n''</sub>, all vectors can be expressed in terms of [[Vector component|component]]s. Polar vectors can be represented as one-column matrices with ''n'' rows where ''n'' is the space dimensionality. Linear transformations can be represented with square matrices; to each linear transformation ''A'' of L<sub>''n''</sub> corresponds a square matrix of [[rank (linear algebra)|rank]] ''n''. Conversely, to each square matrix of rank ''n'' corresponds a linear transformation of L<sub>''n''</sub> at a given basis. Because of the additivity and homogeneity of the linear trasformation and the eigenvalue equation (which is also a linear transformation &mdash; homothety), those vector functions can be expressed in matrix form. Thus, in a the two-dimensional vector space ''L''<sub>2</sub> fitted with [[standard basis]], the eigenvector equation for a linear transformation ''A'' can be written in the following matrix representation:
  +
:<math> \begin{bmatrix} a_{11} & a_{12} \\ a_{21} & a_{22} \end{bmatrix} \begin{bmatrix} x \\ y \end{bmatrix} = \lambda \begin{bmatrix} x \\ y \end{bmatrix}, </math>
  +
where the juxtaposition of matrices means [[matrix multiplication]]. This is equivalent to a set of ''n'' linear equations, where ''n'' is the number of basis vectors in the [[basis (linear algebra)|basis set]]. In these equations both the eigenvalue λ and the components of '''x''' are unknown [[variable]]s.
  +
  +
The eigenvectors of ''A'' as defined above are also called ''right eigenvectors'' because they are column vectors that stand on the right side of the matrix ''A'' in the eigenvalue equation. If there exists a transposed matrix ''A''<sup>T</sup> that satifies the eigenvalue equation, that is, if ''A''<sup>T</sup>'''x''' = λ'''x''', then λ'''x'''<sup>T</sup> = (λ'''x''')<sup>T</sup> = (''A''<sup>T</sup>'''x''')<sup>T</sup> = '''x'''<sup>T</sup>''A'', or '''x'''<sup>T</sup>''A'' = λ'''x'''<sup>T</sup>. The last equation is similar to the eigenvalue equation but instead of the column vector '''x''' it contains its transposed vector, the row vector '''x'''<sup>T</sup>, which stands on the left side of the matrix ''A''. The eigenvectors that satisfy the eigenvalue equation '''x'''<sup>T</sup>''A'' = λ'''x'''<sup>T</sup> are called ''left eigenvectors''. They are row vectors.<ref> See {{Harvnb|Shores|2007|loc=p. 252}}</ref> In many common applications, only right eigenvectors need to be considered. Hence the unqualified term "eigenvector" can be understood to refer to a right eigenvector. Eigenvalue equations, written in terms of right or left eigenvectors (''A'''''x''' = λ'''x''' and '''x'''<sup>T</sup>''A'' = λ'''x'''<sup>T</sup>) have the same eigenvalue λ.<ref name="Weisstein">For a proof of this theorem, see Weisstein, Eric W. [http://mathworld.wolfram.com/Eigenvector.html Eigenvector] From MathWorld − A Wolfram Web Resource</ref>
  +
  +
An eigenvector is defined to be a ''principal'' or ''dominant eigenvector'' if it corresponds to the eigenvalue of largest magnitude (for real numbers, largest absolute value). Repeated application of a linear transformation to an arbitrary vector results in a vector proportional (collinear) to the principal eigenvector.<ref name="Weisstein"/>
  +
  +
The applicability the eigenvalue equation to general matrix theory extends the use of eigenvectors and eigenvalues to all matrices, and thus greatly extends the scope of use of these mathematical constructs not only to transformations in linear vector spaces but to all fields of science that use matrices: linear equations systems, optimization, vector and tensor calculus, all fields of physics that use matrix quantities, particularly quantum physics, relativity, and electrodynamics, as well as many engineering applications.
   
*An '''eigenvector''' of a linear transformation is a vector that is either left unaffected or simply multiplied by a [[scale factor]] after the transformation (the former corresponds to a scale factor of 1).
+
==Characteristic equation==
*The '''eigenvalue''' of a non-zero eigenvector is the scale factor by which it has been multiplied.
+
{{main|Characteristic equation}}
*An '''eigenvalue''' of a linear transformation is a factor for which it has a non-zero eigenvector with that factor as its eigenvalue.
+
{{main|Characteristic polynomial}}
*The '''eigenspace''' corresponding to a given eigenvalue of a linear transformation is the [[vector space]] of all eigenvectors with that eigenvalue.
+
The determination of the eigenvalues and eigenvectors is important in virtually all areas of physics and many engineering problems, such as stress calculations, stability analysis, oscillations of vibrating systems, etc. It is equivalent to [[matrix diagonalization]], and is the first step of orthogonalization, finding of invariants, optimization (minimization or maximization), analysis of linear systems, and many other common applications.
*The '''geometric [[multiplicity]]''' of an eigenvalue is the [[dimension]] of the associated eigenspace.
+
*The '''spectrum''' of a transformation on a finite dimensional [[vector space]] is the [[set]] of all its eigenvalues. (In the infinite-dimensional case, the concept of [[Spectrum (functional analysis)|spectrum]] is more subtle and depends on the [[topology]] on the vector space.)
+
The usual method of finding all eigenvectors and eigenvalues of a system is first to get rid of the unknown components of the eigenvectors, then find the eigenvalues, plug those back one by one in the eigenvalue equation in matrix form and solve that as a system of linear equations to find the components of the eigenvectors. From the identity transformation ''I'''''x''' = '''x''', where ''I'' is the [[identity matrix]], '''x''' in the eigenvalue equation can be replaced by ''I'''''x''' to give:
  +
:<math>A \mathbf{x} = \lambda I \mathbf{x}</math>
  +
The identity matrix is needed to keep matrices, vectors, and scalars straight; the equation (''A'' − λ) ''x'' = 0 is shorter, but mixed up since it does not differentiate between matrix, scalar, and vector.<ref>See {{Harvnb|Strang|2006|loc=footnote to p. 245}}</ref> The expression in the right hand side is transferred to left hand side with a negative sign, leaving 0 on the right hand side:
  +
:<math>A \mathbf{x} - \lambda I \mathbf{x} = 0</math>
  +
The eigenvector '''x''' is pulled out behind parentheses:
  +
:<math>(A - \lambda I) \mathbf{x} = 0</math>
  +
This can be viewed as a [[linear system of equations]] in which the coefficient matrix is the expression in the parentheses, the matrix of the unknowns is '''x''', and the right hand side matrix is zero. According to [[Cramer's rule]], this system of equations has non-trivial solutions (not all zeros, or not any number) if and only if its [[determinant]] vanishes, so the solutions of the equation are given by:
  +
:<math>\det(A - \lambda I) = 0 \,</math>
  +
This equation is defined as the ''[[characteristic equation]]'' (less often, [[secular equation]]) of ''A'', and the left-hand side is defined as the ''[[characteristic polynomial]]''. The eigenvector '''x''' or its components are not present in the characteristic equation, so at this stage they are dispensed with, and the only unknowns that remain to be calculated are the eigenvalues (the components of matrix ''A'' are given, ''i. e'', known beforehand). For a vector space ''L''<sub>2</sub>, the transformation ''A'' is a 2 × 2 square matrix, and the characteristic equation can be written in the following form:
  +
:<math>\begin{vmatrix} a_{11} - \lambda & a_{12}\\a_{21} & a_{22} - \lambda\end{vmatrix} = 0</math>
  +
Expansion of the determinant in the left hand side results in a characteristic polynomial which is a [[monic]] (its leading coefficient is 1) [[polynomial]] of the second [[degree of a polynomial|degree]], and the characteristic equation is the [[quadratic equation]]
  +
:<math>\lambda^2 - \lambda (a_{11} + a_{22}) + (a_{11} a_{22} - a_{12} a_{21}) = 0, \,</math>
  +
which has the following solutions ([[root (mathematics)|roots]]):
  +
:<math>\lambda_{1,2} = \frac{1}{2} \left [(a_{11} + a_{22}) \pm \sqrt{4a_{12} a_{21} + (a_{11} - a_{22})^2} \right ].</math>
  +
For real matrices, the coefficients of the characteristic polynomial are all real. The number and type of roots depends on the value of the [[discriminant]], Δ. For cases Δ = 0, Δ > 0, or Δ < 0, respectively, the roots are one real, two real, or two complex. If the roots are complex, they are also [[complex conjugate]]s of each other. When the number of roots is less than the degree of the characteristic polynomial (the latter is also the rank of the matrix, and the number of dimensions of the vector space) the equation has a ''multiple root''. In the case of a quadratic equation with one root, this root is a double root, or a root with [[multiplicity]] 2. A root with a multiplicity of 1 is a ''simple root''. A quadratic equation with two real or complex roots has only simple roots. In general, the ''algebraic [[multiplicity]]'' of an eigenvalue is defined as the [[Multiplicity#Multiplicity of a root of a polynomial|multiplicity of the corresponding root]] of the characteristic polynomial. The ''spectrum'' of a transformation on a finite dimensional [[vector space]] is defined as the [[set]] of all its eigenvalues. In the infinite-dimensional case, the concept of [[Spectrum (functional analysis)|spectrum]] is more subtle and depends on the [[topology]] of the vector space.
  +
  +
The general formula for the characteristic polynomial of an ''n''-square matrix is
  +
:<math>p(\lambda) = \sum_{k=0}^n (-1)^k S_k \lambda^{n-k},</math>
  +
where ''S''<sub>0</sub> = 1, ''S''<sub>1</sub> = tr(''A''), the [[trace (linear algebra)|trace]] of the transformation matrix ''A'', and ''S<sub>k</sub>'' with ''k'' > 1 are the sums of the principal [[minor (linear algebra)|minor]]s of order ''k''.<ref>For details and proof, see {{Harvnb|Meyer|2000|loc=p. 494-495}}</ref> The fact that eigenvalues are roots of an ''n''-order equation shows that a linear transformation of an ''n''-dimensional linear space
  +
has at most ''n'' different eigenvalues.<ref>See {{Harvnb|Greub|1975|loc=p. 118}}</ref> According to the [[fundamental theorem of algebra]], in a complex linear space, the characteristic polynomial has at least one zero. Consequently, every linear transformation of a complex linear space has at least one eigenvalue. <ref>See {{Harvnb|Greub|1975|loc=p. 119}}</ref><ref>For proof, see {{Harvnb|Gelfand|1971|loc=p. 115}}</ref> For real linear spaces, if the dimension is an odd number, the linear transformation has at least one eigenvalue; if the dimension is an even number, the number of eigenvalues depends on the determinant of the transformation matrix: if the determinant is negative, there exists at least one positive and one negative eigenvalue, if the determinant is positive nothing can be said about existence of eigenvalues.<ref>For proof, see {{Harvnb|Greub|1975|loc=p. 119}}</ref> The complexity of the problem for finding roots/eigenvalues of the characteristic polynomial increases rapidly with increasing the degree of the polynomial (the dimension of the vector space), ''n''. Thus, for ''n'' = 3, eigenvalues are roots of the [[cubic equation]], for ''n'' = 4 &mdash; roots of the [[quartic equation]]. For ''n'' > 4 there are no exact solutions and one has to resort to root-finding algorithms, such as [[Newton's method]] ([[Horner's method]]) to find numerical approximations of eigenvalues. For large symmetric [[sparse matrix|sparse matrices]], [[Lanczos algorithm]] is used to compute eigenvalues and eigenvectors.
  +
  +
In order to find the eigenvectors, the eigenvalues thus found as roots of the characteristic equations are plugged back, one at a time, in the eigenvalue equation written in a matrix form (illustrated for the simplest case of a two-dimensional vector space ''L''<sub>2</sub>):
  +
:<math>\left (\begin{bmatrix} a_{11} & a_{12}\\a_{21} & a_{22}\end{bmatrix} - \lambda \begin{bmatrix} 1 & 0\\0 & 1\end{bmatrix} \right ) \begin{bmatrix} x \\ y \end{bmatrix} = \begin{bmatrix} a_{11} - \lambda & a_{12}\\a_{21} & a_{22} - \lambda \end{bmatrix} \begin{bmatrix} x \\ y \end{bmatrix} = \begin{bmatrix} 0 \\ 0 \end{bmatrix},</math>
  +
where λ is one of the eigenvalues found as a root of the characteristic equation. This matrix equation is equivalent to a system of two linear equations:
  +
:<math>\begin{cases}
  +
\left ( a_{11} - \lambda \right ) x + a_{12} y = 0 \\
  +
a_{21} x + \left ( a_{22} - \lambda \right ) y = 0
  +
\end{cases}</math>
  +
The equations are solved for ''x'' and ''y'' by the usual algebraic or matrix methods. Often, it is possible to divide both sides of the equations to one or more of the coefficients which makes some of the coefficients in front of the unknowns equal to 1. This is called ''normalization'' of the vectors, and corresponds to choosing one of the eigenvectors (the normalized eigenvector) as a representative of all vectors in the eigenspace corresponding to the respective eigenvalue. The ''x'' and ''y'' thus found are the components of the eigenvector in the coordinate system used (most often Cartesian, or polar).
  +
  +
Using the [[Cayley-Hamilton theorem]] which states that every square matrix satisfies its own characteristic equation, it can be shown that (most generally, in the complex space) there exists at least one non-zero vector that satisfies the eigenvalue equation for that matrix.<ref>For details and proof, see {{Harvnb|Kuttler|2007|loc=p. 151}}</ref> As it was said in the Definitions section, to each eigenvalue correspond an infinite number of colinear (linearly dependent) eigenvectors that form the eigenspace for this eigenvalue. On the other hand, the dimension of the eigenspace is equal to the number of the linearly independent eigenvectors that it contains. The ''geometric multiplicity'' of an eigenvalue is defined as the [[dimension]] of the associated eigenspace. A multiple eigenvalue may give rise to a single eigenvector so that its algebraic multiplicity may be different than the geometric multiplicity.<ref>See {{Harvnb|Shilov|1969|loc=p. 134}}</ref> However, as already stated, different eigenvalues are paired with linearly independent eigenvectors.<ref name="Shilov_lemma"/> From the aforementioned, it follows that the geometric multiplicity cannot be greater than the algebraic multiplicity.<ref>See {{Harvnb|Shilov|1969|loc=p. 135 and Problem 11 to Chapter 5}}</ref>
   
 
For instance, an ''eigenvector'' of a rotation in three dimensions is a vector located within the [[axis of rotation|axis]] about which the rotation is performed. The corresponding ''eigenvalue'' is 1 and the corresponding ''eigenspace'' contains all the vectors along the axis. As this is a one-dimensional space, its ''geometric multiplicity'' is one. This is the only eigenvalue of the ''spectrum'' (of this rotation) that is a [[real number]].
 
For instance, an ''eigenvector'' of a rotation in three dimensions is a vector located within the [[axis of rotation|axis]] about which the rotation is performed. The corresponding ''eigenvalue'' is 1 and the corresponding ''eigenspace'' contains all the vectors along the axis. As this is a one-dimensional space, its ''geometric multiplicity'' is one. This is the only eigenvalue of the ''spectrum'' (of this rotation) that is a [[real number]].
   
 
==Examples==
 
==Examples==
  +
The examples that follow are for the simplest case of two-dimensional vector space ''L''<sub>2</sub> but they can easily be applied in the same manner to spaces of higher dimensions.
  +
===Homothety, identity, point reflection, and null transformation===
  +
[[Image:Homothety in two dim.svg|thumb|left|200px|Fig. 3. When a surface is stretching equally in homothety, any one of the radial vectors can be the eigenvector.]]
  +
As a one-dimensional vector space ''L''<sub>1</sub>, consider a rubber string tied to unmoving support in one end, such as that on a child's sling. Pulling the string away from the point of attachment stretches it and elongates it by some scaling factor λ which is a real number. Each vector on the string is stretched equally, with the same scaling factor λ, and although elongated it preserves its original direction. This type of transformation is called homothety (similarity transformation). For a two-dimensional vector space ''L''<sub>2</sub>, consider a rubber sheet stretched equally in all directions such as a small area of the surface of an inflating balloon (Fig. 3). All vectors originating at a fixed point on the balloon surface are stretched equally with the same scaling factor λ. The homothety transformation in two-dimensions is described by a 2 × 2 square matrix, acting on an arbitrary vector in the plane of the stretching/shrinking surface. After doing the matrix multiplication, one obtains:
  +
:<math>A \mathbf{x} = \begin{bmatrix}\lambda & 0\\0 & \lambda\end{bmatrix} \begin{bmatrix} x \\ y \end{bmatrix} = \begin{bmatrix}\lambda . x + 0 . y \\0 . x + \lambda . y\end{bmatrix} = \lambda \begin{bmatrix} x \\ y \end{bmatrix} = \lambda \mathbf{x},</math>
  +
which, expressed in words, means that the transformation is equivalent to multiplying the length of the vector by λ while preserving its original direction. The equation thus obtained is exactly the eigenvalue equation. Since the vector taken was arbitrary, in homothety any vector in the vector space undergoes the eigenvalue equation, ''i. e.'' any vector lying on the balloon surface can be an eigenvector. Whether the transformation is stretching (elongation, extension, inflation), or shrinking (compression, deflation) depends on the scaling factor: if λ > 1, it is stretching, if λ < 1, it is shrinking.
  +
  +
Several other transformations can be considered special types of homothety with some fixed, constant value of λ: in identity which leaves vectors unchanged, λ = 1; in reflection about a point which preserves length and direction of vectors but changes their orientation to the opposite one, λ = −1; and in null transformation which transforms each vector to the zero vector, λ = 0. The null transformation does not give rise to an eigenvector since the zero vector cannot be an eigenvector but it has eigenspace since eigenspace contains also the zero vector by definition.
  +
  +
===Unequal scaling===
  +
For a slightly more complicated example, consider a sheet that is stretched uneqally in two perpendicular directions along the coordinate axes, or, similarly, stretched in one direction, and shrunk in the other direction. In this case, there are two different scaling factors: ''k''<sub>1</sub> for the scaling in direction ''x'', and ''k''<sub>2</sub> for the scaling in direction ''y''. The transformation matrix is <math>\begin{bmatrix}k_1 & 0\\0 & k_2\end{bmatrix}</math>, and the characteristic equation is λ<sup>2</sup> − λ (''k''<sub>1</sub> + ''k''<sub>2</sub>) + ''k''<sub>1</sub>''k''<sub>2</sub> = 0. The eigenvalues, obtained as roots of this equation are λ<sub>1</sub> = ''k''<sub>1</sub>, and λ<sub>2</sub> = ''k''<sub>2</sub> which means, as expected, that the two eigenvalues are the scaling factors in the two directions. Plugging ''k''<sub>1</sub> back in the eigenvalue equation gives one of the eigenvectors:
  +
:<math>\begin{bmatrix}k_1 - k_1 & 0\\0 & k_2 - k_1\end{bmatrix} \begin{bmatrix} x \\ y\end{bmatrix} = \begin{cases}
  +
\left ( k_1 - k_1 \right ) x + 0 . y \\
  +
0 . x + \left ( k_2 - k_1 \right ) y
  +
\end{cases} = \left ( k_2 - k_1 \right ) y = 0.</math>
  +
Dividing the last equation by ''k''<sub>2</sub> − ''k''<sub>1</sub>, one obtains ''y'' = 0 which represents the ''x'' axis. A vector with lenght 1 taken along this axis represents the normalized eigenvector corresponding to the eigenvalue λ<sub>1</sub>. The eigenvector corresponding to λ<sub>2</sub> which is a unit vector along the ''y'' axis is found in a similar way. In this case, both eigenvalues are simple (with algebraic and geometric multiplicities equal to 1). Depending on the values of λ<sub>1</sub> and λ<sub>2</sub>, there are several notable special cases. In particular, if λ<sub>1</sub> > 1, and λ<sub>2</sub> = 1, the transformation is a stretch in the direction of axis ''x''. If λ<sub>2</sub> = 0, and λ<sub>1</sub> = 1, the transformation is a projection of the surface ''L''<sub>2</sub> on the axis ''x'' because all vectors in the direction of ''y'' become zero vectors.
  +
  +
Let the rubber sheet is stretched along the ''x'' axis (''k''<sub>1</sub> > 1) and simultaneously shrunk along the ''y'' axis (''k''<sub>2</sub> < 1). Then λ<sub>1</sub> = ''k''<sub>1</sub> will be the principal eigenvalue. Repeatedly applying this transformation of stretching/shrinking many times to the rubber sheet will turn the latter more and more similar to a rubber string. Any vector on the surface of the rubber sheet will be oriented closer and closer to the direction of the ''x'' axis (the direction of stretching), that is, it will become collinear with the principal eigenvector.
  +
 
===Mona Lisa===
 
===Mona Lisa===
 
[[Image:Mona Lisa with eigenvector.png|270px|right]]
 
[[Image:Mona Lisa with eigenvector.png|270px|right]]
For the example shown on the right, the matrix that would produce a shear transformation similar to this would be.
+
For the example shown on the right, the matrix that would produce a shear transformation similar to this would be
:<math>A=\begin{bmatrix}1 & 0\\ -\frac{1}{2} & 1\end{bmatrix}</math>
+
:<math>A=\begin{bmatrix}1 & 0\\ -\frac{1}{2} & 1\end{bmatrix}.</math>
   
We seek eigenvectors. That is vectors for which ''A'''''x''' = &lambda;'''x'''. The only effect of the matrix on these vectors will be to change their length, and possibly reverse their direction. So multiplying the right hand side by the [[Identity matrix]] ''I'', we have ''A'''''x''' = (&lambda;''I'')'''x''' therefore
+
The set of eigenvectors <math>\mathbf{x}</math> for <math>A</math> is defined as those vectors which, when multiplied by <math>A</math>, result in a simple scaling <math>\lambda</math> of <math>\mathbf{x}</math>. Thus,
  +
:<math>A\mathbf{x} = \lambda\mathbf{x}.</math>
  +
If we restrict ourselves to real eigenvalues, the only effect of the matrix on the eigenvectors will be to change their length, and possibly reverse their direction. So multiplying the right hand side by the [[Identity matrix]] ''I'', we have
  +
:<math>A\mathbf{x} = (\lambda I)\mathbf{x},</math>
  +
and therefore
 
:<math>(A-\lambda I)\mathbf{x}=0.</math>
 
:<math>(A-\lambda I)\mathbf{x}=0.</math>
   
In order for this equation to have non trivial solutions, we require the [[determinant]] det(''A'' &minus; &lambda;''I'') which called the [[characteristic polynomial]] of the matrix A to be zero. And so
+
In order for this equation to have non-trivial solutions, we require the [[determinant]] <math>\det(A - \lambda I)</math>, which is called the [[characteristic polynomial]] of the matrix A, to be zero. In our example we can calculate the determinant as
   
 
:<math>\det\!\left(\begin{bmatrix}1 & 0\\ -\frac{1}{2} & 1\end{bmatrix} - \lambda\begin{bmatrix}1 & 0\\ 0 & 1\end{bmatrix} \right)=(1-\lambda)^2,</math>
 
:<math>\det\!\left(\begin{bmatrix}1 & 0\\ -\frac{1}{2} & 1\end{bmatrix} - \lambda\begin{bmatrix}1 & 0\\ 0 & 1\end{bmatrix} \right)=(1-\lambda)^2,</math>
   
So we have obtained the [[characteristic polynomial]] (1 &minus; &lambda;)<sup>2</sup> of the matrix A. There is in this case only one distinct solution of the equation (1 &minus; &lambda;)<sup>2</sup> = 0, <math>\lambda=1</math>. This is the [[eigenvalue]] of the matrix A.
+
and now we have obtained the [[characteristic polynomial]] <math>(1-\lambda)^2</math> of the matrix A. There is in this case only one distinct solution of the equation <math>(1-\lambda)^2 = 0</math>, <math>\lambda=1</math>. This is the [[eigenvalue]] of the matrix A. As in the study of roots of polynomials, it is convenient to say that this eigenvalue has multiplicity 2.
   
Having found an eigenvalue, we can solve for the space of eigenvectors by finding the [[nullspace]] of <math>A-(1)I</math>. In other words by solving for vectors '''x''' which are solutions of
+
Having found an eigenvalue <math>\lambda=1</math>, we can solve for the space of eigenvectors by finding the [[Kernel_(matrix)|nullspace]] of <math>A-(1)I</math>. In other words by solving for vectors <math>\mathbf{x}</math> which are solutions of
   
 
:<math>\begin{bmatrix}1-\lambda & 0\\ -\frac{1}{2} & 1-\lambda \end{bmatrix}\begin{bmatrix}x_1\\ x_2\end{bmatrix}=0</math>
 
:<math>\begin{bmatrix}1-\lambda & 0\\ -\frac{1}{2} & 1-\lambda \end{bmatrix}\begin{bmatrix}x_1\\ x_2\end{bmatrix}=0</math>
Line 65: Line 84:
 
where ''c'' is an arbitrary constant. All vectors of this form, i.e. pointing straight up or down, are [[eigenvectors]] of the matrix A. The effect of applying the matrix A to these vectors is equivalent to multiplying them by their corresponding eigenvalue, in this case 1.
 
where ''c'' is an arbitrary constant. All vectors of this form, i.e. pointing straight up or down, are [[eigenvectors]] of the matrix A. The effect of applying the matrix A to these vectors is equivalent to multiplying them by their corresponding eigenvalue, in this case 1.
   
In general, 2-by-2 matrices will have two distinct eigenvalues, and thus two distinct eigenvectors. Whereas most vectors will have both their lengths and directions changed by the matrix, eigenvectors will only have their lengths changed, and will not change their direction, except perhaps to flip through the origin. Also, it is usually the case that the eigenvalue will be something other than 1, and so eigenvectors will be stretched, squashed and/or flipped through the origin by the matrix.
+
In general, 2-by-2 matrices will have two distinct eigenvalues, and thus two distinct eigenvectors. Whereas most vectors will have both their lengths and directions changed by the matrix, eigenvectors will only have their lengths changed, and will not change their direction, except perhaps to flip through the origin in the case when the eigenvalue is a negative number. Also, it is usually the case that the eigenvalue will be something other than 1, and so eigenvectors will be stretched, squashed and/or flipped through the origin by the matrix.
   
 
===Other examples===
 
===Other examples===
As the Earth rotates, every arrow pointing outward from the center of the Earth also rotates, except those arrows that lie on the axis of rotation. Consider the transformation of the Earth after one hour of rotation: An arrow from the center of the Earth to the Geographic [[South Pole]] would be an eigenvector of this transformation, but an arrow from the center of the Earth to anywhere on the [[equator]] would not be an eigenvector. Since the arrow pointing at the pole is not stretched by the rotation of the Earth, its eigenvalue is 1.
+
As the Earth rotates, every arrow pointing outward from the center of the Earth also rotates, except those arrows which are parallel to the axis of rotation. Consider the transformation of the Earth after one hour of rotation: An arrow from the center of the Earth to the Geographic [[South Pole]] would be an eigenvector of this transformation, but an arrow from the center of the Earth to anywhere on the [[equator]] would not be an eigenvector. Since the arrow pointing at the pole is not stretched by the rotation of the Earth, its eigenvalue is 1.
   
Another example is provided by a rubber sheet expanding omnidirectionally about a fixed point in such a way that the distances from any point of the sheet to the fixed point are doubled. This expansion is a transformation with eigenvalue 2. Every vector from the fixed point to a point on the sheet is an eigenvector, and the eigenspace is the set of all these vectors.
+
[[Image:Standing wave.gif|thumb|270px|Fig. 2. A standing wave in a rope fixed at its boundaries is an example of an eigenvector, or more precisely, an eigenfunction of the transformation giving the acceleration. As time passes, the [[standing wave]] is scaled by a [[sinusoid]]al oscillation whose frequency is determined by the eigenvalue, but its overall shape is not modified.]]
  +
However, three-dimensional geometric space is not the only vector space. For example, consider a stressed rope fixed at both ends, like the [[vibrating string]]s of a [[string instrument]] (Fig. 2). The distances of atoms of the vibrating rope from their positions when the rope is at rest can be seen as the [[Vector (spatial)|component]]s of a vector in a space with as many dimensions as there are [[atom]]s in the rope.
   
[[Image:Standing wave.gif|thumb|270px|Fig. 2. A standing [[wave]] in a rope fixed at its boundaries is an example of an eigenvector, or more precisely, an eigenfunction of the transformation giving the acceleration. As time passes, the [[standing wave]] is scaled by a [[sinusoid|sinusoidal]] oscillation whose frequency is determined by the eigenvalue, but its overall shape is not modified.]]
+
Assume the rope is a [[continuum mechanics|continuous medium]]. If one considers the equation for the [[acceleration]] at every point of the rope, its eigenvectors, or ''[[eigenfunction]]s'', are the [[standing wave]]s. The standing waves correspond to particular oscillations of the rope such that the acceleration of the rope is simply its shape scaled by a factor—this factor, the eigenvalue, turns out to be <math>-\omega^2</math> where <math>\omega</math> is the [[angular frequency]] of the oscillation. Each component of the vector associated with the rope is multiplied by a time-dependent factor <math>\sin(\omega t)</math>. If [[damping]] is considered, the [[amplitude]] of this oscillation decreases until the rope stops oscillating, corresponding to a [[complex number|complex]] ω. One can then associate a lifetime with the imaginary part of ω, and relate the concept of an eigenvector to the concept of [[resonance]]. Without damping, the fact that the acceleration operator (assuming a uniform density) is [[Hermitian operator|Hermitian]] leads to several important properties, such as that the standing wave patterns are [[orthogonal functions]].
However, three-dimensional geometric space is not the only vector space. For example, consider a stressed rope fixed at both ends, like the [[vibrating string]]s of a [[string instrument]] (Fig. 2). The distances of atoms of the vibrating rope from their positions when the rope is at rest can be seen as the [[vector (spatial)|component]]s of a vector in a space with as many dimensions as there are [[atom]]s in the rope.
 
   
Assume the rope is a [[continuum mechanics|continuous medium]]. If one considers the equation for the [[acceleration]] at every point of the rope, its eigenvectors, or ''[[eigenfunction]]s'', are the [[standing waves]]. The standing waves correspond to particular oscillations of the rope such that the acceleration of the rope is simply its shape scaled by a factor&mdash;this factor, the eigenvalue, turns out to be <math>-\omega^2</math> where &omega; is the [[angular frequency]] of the oscillation. Each component of the vector associated with the rope is multiplied by a time-dependent factor <math>\sin(\omega t)</math>. If [[damping]] is considered, the [[amplitude]] of this oscillation decreases until the rope stops oscillating, corresponding to a [[complex number|complex]] &omega;. One can then associate a [[lifetime]] with the imaginary part of &omega;, and relate the concept of an eigenvector to the concept of [[resonance]]. Without damping, the fact that the acceleration operator (assuming a uniform density) is [[Hermitian operator|Hermitian]] leads to several important properties, such as that the standing wave patterns are [[orthogonal functions]].
+
==Eigenfunctions==
+
However, it is sometimes unnatural or even impossible to write down the eigenvalue equation in a matrix form. This occurs for instance when the vector space is infinite dimensional, for example, in the case of the rope above. Depending on the nature of the transformation ''T'' and the space to which it applies, it can be advantageous to represent the eigenvalue equation as a set of [[differential equation]]s. If ''T'' is a [[differential operator]], the eigenvectors are commonly called '''eigenfunctions''' of the differential operator representing ''T''. For example, [[Differential calculus|differentiation]] itself is a linear transformation since
==Eigenvalue equation==
 
Suppose ''T'' is a [[linear transformation]] of a finite-dimensional space, that is <math>T(a\mathbf{v}+b\mathbf{w})=aT(\mathbf{v})+bT(\mathbf{w})</math> for all [[scalar (mathematics)|scalars]] ''a'', ''b'', and vectors '''v''', '''w''').
 
Then
 
<math>\mathbf{v}_\lambda</math> is an eigenvector and ''&lambda;'' the corresponding eigenvalue of ''T'' if the [[equation]]:
 
:<math>T(\mathbf{v}_\lambda)=\lambda\,\mathbf{v}_\lambda</math>
 
is true, where ''T''('''v'''<sub>''&lambda;''</sub>) is the vector obtained when applying the transformation ''T'' to '''v'''<sub>''&lambda;''</sub>.
 
 
Consider a [[basis (linear algebra)|basis]] of the vector space that ''T'' acts on. Then ''T'' and '''v'''<sub>''&lambda;''</sub> can be represented relative to that basis by a [[matrix (mathematics)|matrix]] ''A''<sub>''T''</sub>—a two-dimensional [[array]]—and respectively a column vector ''v''<sub>''&lambda;''</sub>—a one-dimensional vertical array. The eigenvalue equation in its matrix representation is written
 
:<math>A_T\,v_\lambda=\lambda\,v_\lambda</math>
 
where the juxtaposition is [[matrix multiplication]]. Since, once a basis is fixed, ''T'' and its matrix representation ''A''<sub>''T''</sub> are equivalent, we can often use the same symbol ''T'' for both the matrix representation and the transformation. This is equivalent to a set of ''n'' linear equations, where ''n'' is the number of basis vectors in the [[basis (linear algebra)|basis set]]. In this equation both the eigenvalue ''&lambda;'' and the ''n'' components of '''v'''<sub>''&lambda;''</sub> are [[variable|unknown]]s.
 
 
However, it is sometimes unnatural or even impossible to write down the eigenvalue equation in a matrix form. This occurs for instance when the vector space is infinite dimensional, for example, in the case of the rope above. Depending on the nature of the transformation ''T'' and the space to which it applies, it can be advantageous to represent the eigenvalue equation as a set of [[differential equation]]s. If ''T'' is a [[differential operator]], the eigenvectors are commonly called '''eigenfunctions''' of the differential operator representing ''T''. For example, [[differentiation]] itself is a linear transformation since
 
 
:<math> \displaystyle\frac{d}{dt}(af+bg) = a \frac{df}{dt} + b \frac{dg}{dt} </math>
 
:<math> \displaystyle\frac{d}{dt}(af+bg) = a \frac{df}{dt} + b \frac{dg}{dt} </math>
 
(''f''(''t'') and ''g''(''t'') are [[differentiable]] functions, and ''a'' and ''b'' are [[constant]]s).
 
(''f''(''t'') and ''g''(''t'') are [[differentiable]] functions, and ''a'' and ''b'' are [[constant]]s).
Line 81: Line 100:
 
Consider differentiation with respect to <math>t</math>. Its eigenfunctions ''h''(''t'') obey the eigenvalue equation:
 
Consider differentiation with respect to <math>t</math>. Its eigenfunctions ''h''(''t'') obey the eigenvalue equation:
 
:<math>\displaystyle\frac{dh}{dt} = \lambda h</math>,
 
:<math>\displaystyle\frac{dh}{dt} = \lambda h</math>,
where ''&lambda;'' is the eigenvalue associated with the function. Such a function of time is constant if <math>\lambda = 0</math>, grows proportionally to itself if <math>\lambda </math> is positive, and decays proportionally to itself if <math>\lambda </math> is negative. For example, an idealized population of rabbits breeds faster the more rabbits there are, and thus satisfies the equation with a positive lambda.
+
where ''λ'' is the eigenvalue associated with the function. Such a function of time is constant if <math>\lambda = 0</math>, grows proportionally to itself if <math>\lambda </math> is positive, and decays proportionally to itself if <math>\lambda </math> is negative. For example, an idealized population of rabbits breeds faster the more rabbits there are, and thus satisfies the equation with a positive lambda.
   
The solution to the eigenvalue equation is <math>g(t)= \exp (\lambda t)</math>, the [[exponential function]]; thus that function is an eigenfunction of the differential operator ''d/dt'' with the eigenvalue ''&lambda;''. If ''&lambda;'' is [[Negative and non-negative numbers|negative]], we call the evolution of ''g'' an [[exponential decay]]; if it is [[Negative and non-negative numbers|positive]], an [[exponential growth]]. The value of ''&lambda;'' can be any [[complex number]]. The spectrum of ''d/dt'' is therefore the whole [[complex plane]]. In this example the vector space in which the operator ''d/dt'' acts is the space of the [[differentiable]] functions of one [[variable]]. This space has an [[infinite]] dimension (because it is not possible to express every differentiable function as a [[linear combination]] of a finite number of [[basis function]]s). However, the eigenspace associated with any given eigenvalue ''&lambda;'' is one dimensional. It is the set of all functions <math>g(t)= A \exp (\lambda t)</math>, where ''A'' is an arbitrary constant, the initial population at ''t=0''.
+
The solution to the eigenvalue equation is <math>g(t)= \exp (\lambda t)</math>, the [[exponential function]]; thus that function is an eigenfunction of the differential operator ''d/dt'' with the eigenvalue ''λ''. If ''λ'' is [[Negative and non-negative numbers|negative]], we call the evolution of ''g'' an [[exponential decay]]; if it is [[Negative and non-negative numbers|positive]], an [[exponential growth]]. The value of ''λ'' can be any [[complex number]]. The spectrum of ''d/dt'' is therefore the whole [[complex plane]]. In this example the vector space in which the operator ''d/dt'' acts is the space of the [[differentiable]] functions of one [[variable]]. This space has an [[infinite]] dimension (because it is not possible to express every differentiable function as a [[linear combination]] of a finite number of [[basis function]]s). However, the eigenspace associated with any given eigenvalue ''λ'' is one dimensional. It is the set of all functions <math>g(t)= A \exp (\lambda t)</math>, where ''A'' is an arbitrary constant, the initial population at ''t=0''.
   
 
==Spectral theorem==
 
==Spectral theorem==
Line 92: Line 111:
 
:<math>\mathcal{T}(\mathbf{v})= \lambda_1 (\mathbf{v}_1 \cdot \mathbf{v}) \mathbf{v}_1 + \lambda_2 (\mathbf{v}_2 \cdot \mathbf{v}) \mathbf{v}_2 + \cdots </math>
 
:<math>\mathcal{T}(\mathbf{v})= \lambda_1 (\mathbf{v}_1 \cdot \mathbf{v}) \mathbf{v}_1 + \lambda_2 (\mathbf{v}_2 \cdot \mathbf{v}) \mathbf{v}_2 + \cdots </math>
 
 
where <math>\mathbf{v}_1, \mathbf{v}_2, \dots</math> and <math>\lambda_1, \lambda_2, \dots</math> stand for the eigenvectors and eigenvalues of <math>\mathcal{T}</math>. The simplest case in which the theorem is valid is the case where the linear transformation is given by a [[real number|real]] [[symmetric matrix]] or [[complex number|complex]] [[Hermitian matrix]]; more generally the theorem holds for all [[normal matrix|normal matrices]].
+
where <math>\mathbf{v}_1, \mathbf{v}_2, \dots</math> and <math>\lambda_1, \lambda_2, \dots</math> stand for the eigenvectors and eigenvalues of <math>\mathcal{T}</math>. The theorem is valid for all self-adjoint linear transformations (linear transformations given by [[real number|real]] [[symmetric matrices]] and [[Hermitian matrices]]), and for the more general class of ([[complex number|complex]]) [[normal matrix|normal matrices]].
   
If one defines the ''n''th power of a transformation as the result of applying it ''n'' times in succession, one can also define [[polynomial]]s of transformations. A more general version of the theorem is that any polynomial ''P'' of <math>\mathcal{T}</math> is equal to:
+
If one defines the ''n''th power of a transformation as the result of applying it ''n'' times in succession, one can also define [[polynomial]]s of transformations. A more general version of the theorem is that any polynomial ''P'' of <math>\mathcal{T}</math> is given by
   
:<math>P(\mathcal{T})(\mathbf{v})= P(\lambda_1) (\mathbf{v}_1 \cdot \mathbf{v}) \mathbf{v}_1 + P(\lambda_2) (\mathbf{v}_2 \cdot \mathbf{v}) \mathbf{v}_2 + \cdots </math>
+
:<math>P(\mathcal{T})(\mathbf{v}) = P(\lambda_1) (\mathbf{v}_1 \cdot \mathbf{v}) \mathbf{v}_1 + P(\lambda_2) (\mathbf{v}_2 \cdot \mathbf{v}) \mathbf{v}_2 + \cdots </math>
   
The theorem can be extended to other functions of transformations like [[analytic function]]s, the most general case being [[Measurable function|Borel functions]].
+
The theorem can be extended to other functions of transformations, such as [[analytic function]]s, the most general case being [[Measurable function|Borel functions]].
   
==Eigenvalues and eigenvectors of matrices==
+
==Eigendecomposition==
===Computing eigenvalues and eigenvectors of matrices===
+
{{main|Eigendecomposition (matrix)}}
Suppose that we want to compute the eigenvalues of a given matrix. If the matrix is small, we can compute them symbolically using the [[characteristic polynomial]]. However, this is often impossible for larger matrices, in which case we must use a [[numerical analysis|numerical method]].
+
The [[spectral theorem]] for matrices can be stated as follows. Let <math>\mathbf{A}</math> be a square (<math>n\times n</math>) matrix. Let <math>\mathbf{q}_1 ... \mathbf{q}_k</math> be an eigenvector basis, i.e. an indexed set of ''k'' [[linearly independent]] eigenvectors, where ''k'' is the dimension of the space spanned by the eigenvectors of <math>\mathbf{A}</math>. If ''k''=''n'', then <math>\mathbf{A}</math> can be written
+
:<math>\mathbf{A}=\mathbf{Q}\mathbf{\Lambda}\mathbf{Q}^{-1} </math>
====Symbolic computations====
+
where <math>\mathbf{Q}</math> is the square (<math>n\times n</math>) matrix whose ''i''<sup>th</sup> column is the basis eigenvector <math>\mathbf{q}_i</math> of <math>\mathbf{A}</math> and <math>\mathbf{\Lambda}</math> is the [[diagonal matrix]] whose diagonal elements are the corresponding eigenvalues, i.e. <math>\Lambda_{ii}=\lambda_i</math>.
{{details|symbolic computation of matrix eigenvalues}}
 
 
;Finding eigenvalues
 
An important tool for describing eigenvalues of square matrices is the [[characteristic polynomial]]: saying that ''&lambda;'' is an eigenvalue of ''A'' is equivalent to stating that the [[system of linear equations]] (''A'' &ndash; ''&lambda;I'') ''v'' = 0 (where ''I'' is the [[identity matrix]]) has a non-zero solution ''v'' (an eigenvector), and so it is equivalent to the [[determinant]]:
 
 
:<math>\det(A - \lambda I) = 0 \!\ </math>
 
 
The function ''p''(''&lambda;'') = det(''A'' &ndash; ''&lambda;I'') is a [[polynomial]] in ''&lambda;'' since determinants are defined as sums of products.
 
This is the '''characteristic polynomial''' of ''A'': the eigenvalues of a matrix are the zeros of its [[characteristic polynomial]].
 
 
All the eigenvalues of a matrix ''A'' can be computed by solving the equation <math> p_A(\lambda) = 0 </math>.
 
If ''A'' is an ''n''&times;''n'' matrix, then <math>p_A</math> has degree ''n'' and ''A'' can therefore have at most ''n'' eigenvalues.
 
If the matrix is over an algebraically closed field, such as the complex numbers, then the [[fundamental theorem of algebra]] says that the characteristic equation has exactly ''n'' [[Root (mathematics)|root]]s (zeroes), counted with multiplicity. Therefore, any matrix over the complex numbers has an eigenvalue. All real polynomials of odd degree have a real number as a root, so for odd n, every real matrix has at least one real eigenvalue. However, if ''n'' is even, a matrix with real entries may not have any real eigenvalues. For any ''n'', the non-real eigenvalues of a real matrix will come in [[conjugate transpose#Properties of the conjugate transpose|conjugate pairs]], just as the roots of a polynomial with real coefficients do.
 
 
;Finding eigenvectors
 
Once the eigenvalues &lambda; are known, the eigenvectors can then be found by solving:
 
 
:<math> (A - \lambda I) v = 0 \!\ </math>
 
where v is in the [[null space]] of <math>A-\lambda I</math>
 
 
An example of a matrix with no real eigenvalues is the 90-degree clockwise rotation:
 
 
:<math>\begin{bmatrix}0 & 1\\ -1 & 0\end{bmatrix}</math>
 
whose characteristic polynomial is <math>\lambda^2+1</math> and so its eigenvalues are the pair of complex conjugates ''i'', -''i''. The associated eigenvectors are also not real.
 
 
====Numerical computations====
 
{{details|eigenvalue algorithm}}
 
 
In practice, eigenvalues of large matrices are not computed using the characteristic polynomial. Computing the polynomial becomes expensive in itself, and exact (symbolic) roots of a high-degree polynomial can be difficult to compute and express: the [[Abel–Ruffini theorem]] implies that the roots of high-degree (5 and above) polynomials cannot be expressed simply using <math>n</math>th roots. Effective numerical algorithms for approximating roots of polynomials exist, but small errors in the eigenvalues can lead to large errors in the eigenvectors. Therefore, general algorithms to find eigenvectors and eigenvalues, are [[iterative method|iterative]]. The easiest method is the [[power method]]: a [[random]] vector <math>v</math> is chosen and a sequence of [[unit vector]]s is computed as
 
: <math>\frac{Av}{\|Av\|}</math>, <math>\frac{A^2v}{\|A^2v\|}</math>, <math>\frac{A^3v}{\|A^3v\|}</math>, ...
 
This [[sequence]] will almost always converge to an eigenvector corresponding to the eigenvalue of greatest magnitude. This algorithm is easy, but not very useful by itself. However, popular methods such as the [[QR algorithm]] are based on it.
 
 
===Properties===
 
====Algebraic multiplicity====
 
The '''algebraic [[multiplicity]]''' of an eigenvalue &lambda; of ''A'' is the [[degree of a polynomial|order]] of &lambda; as a zero of the characteristic polynomial of ''A''; in other words, if &lambda; is one [[root (mathematics)|root]] of the polynomial, it is the number of factors (''t'' &minus; ''&lambda;'') in the characteristic polynomial after [[factorization]]. An ''n''&times;''n'' matrix has ''n'' eigenvalues, counted according to their algebraic multiplicity, because its characteristic polynomial has degree ''n''.
 
 
An eigenvalue of algebraic multiplicity 1 is called a "simple eigenvalue".
 
 
In an article on [[matrix theory]], a statement like the one below might be encountered:
 
:"the eigenvalues of a matrix ''A'' are 4,4,3,3,3,2,2,1,"
 
meaning that the algebraic multiplicity of 4 is two, of 3 is three, of 2 is two and of 1 is one. This style is used because algebraic multiplicity is the key to many [[mathematical proof]]s in matrix theory.
 
 
Recall that above we defined the ''geometric'' multiplicity of an eigenvector to be the dimension of the associated eigenspace, the nullspace of λI − ''A''. The algebraic multiplicity can also be thought of as a dimension: it is the dimension of the associated ''generalized eigenspace'' (1st sense), which is the nullspace of the matrix (λI − ''A'')<sup>''k''</sup> for ''any sufficiently large k''. That is, it is the space of ''generalized eigenvectors'' (1st sense), where a generalized eigenvector is any vector which ''eventually'' becomes 0 if λI − ''A'' is applied to it enough times successively. Any eigenvector is a generalized eigenvector, and so each eigenspace is contained in the associated generalized eigenspace. This provides an easy proof that the geometric multiplicity is always less than or equal to the algebraic multiplicity. The first sense should not to be confused with generalized eigenvalue problem as stated below.
 
 
For example:
 
:<math> A=\begin{bmatrix} 1 & 1 \\ 0 & 1 \end{bmatrix}. </math>
 
It has only one eigenvalue, namely &lambda; = 1. The characteristic polynomial is <math>(\lambda-1)^2</math>, so this eigenvalue has algebraic multiplicity 2. However, the associated eigenspace is the axis usually called the ''x'' axis, [[Linear span|span]]ned by the unit vector <math> \begin{bmatrix} 1 \\ 0 \end{bmatrix} </math>, so the geometric multiplicity is only 1.
 
 
Generalized eigenvectors can be used to calculate the [[Jordan normal form]] of a matrix (see discussion below). The fact that Jordan blocks in general are not diagonal but [[nilpotent]] is directly related to the distinction between eigenvectors and generalized eigenvectors.
 
 
====Decomposition theorems for general matrices====
 
The '''decomposition theorem''' is a version of the spectral theorem in the particular case of matrices. This theorem is usually introduced in terms of coordinate transformation. If ''U'' is an [[invertible matrix]], it can be seen as a transformation from one coordinate system to another, with the columns of ''U'' being the components of the new basis vectors within the old basis set. In this new system the coordinates of the vector <math>v</math> are labeled <math>v'</math>. The latter are obtained from the coordinates ''v'' in the original coordinate system by the relation <math>v'=Uv</math> and, the other way around, we have <math>v=U^{-1}v'</math>. Applying successively <math>v'=Uv</math>, <math>w'=Uw</math> and <math>U^{-1}U=I</math>, to the relation <math>Av=w</math> defining the [[matrix multiplication]] provides <math>A'v'=w'</math> with <math>A'=UAU^{-1}</math>, the representation of ''A'' in the new basis. In this situation, the matrices ''A'' and <math>A'</math> are said to be [[Similarity (mathematics)#Linear algebra|similar]].
 
 
The decomposition theorem states that, if one chooses as columns of <math>U^{-1}</math> ''n'' [[linearly independent]] eigenvectors of ''A'', the new matrix <math>A'=UAU^{-1}</math> is diagonal and its diagonal elements are the eigenvalues of ''A''. If this is possible the matrix ''A'' is ''[[diagonalizable matrix|diagonalizable]]''. An example of non-diagonalizable matrix is given by the matrix ''A'' [[Eigenvalue, eigenvector, and eigenspace#Algebraic multiplicity|above]]. There are several generalizations of this decomposition which can cope with the non-diagonalizable case, suited for different purposes:
 
* the [[Schur decomposition|Schur triangular form]] states that any matrix is unitarily equivalent to an [[upper triangular]] one;
 
* the [[singular value decomposition]], <math>A=U \Sigma V^*</math> where <math>\Sigma</math> is diagonal with ''U'' and ''V'' unitary matrices. The diagonal entries of <math>A=U \Sigma V^*</math> are nonnegative; they are called the singular values of ''A''. This can be done for non-square matrices as well;
 
* the [[Jordan normal form]], where <math>A=X \Lambda X^{-1}</math> where <math>\Lambda</math> is not diagonal but block-diagonal. The number and the sizes of the Jordan blocks are dictated by the geometric and algebraic multiplicities of the eigenvalues. The Jordan decomposition is a fundamental result. One might glean from it immediately that a square matrix is described completely by its eigenvalues, including multiplicity, up to similarity. This shows mathematically the important role played by eigenvalues in the study of matrices;
 
* as an immediate consequence of Jordan decomposition, any matrix ''A'' can be written ''uniquely'' as ''A'' = ''S'' + ''N'' where ''S'' is diagonalizable, ''N'' is [[nilpotent matrix|nilpotent]] (i.e., such that ''N<sup>q</sup>''=0 for some ''q''), and ''S'' commutes with ''N'' (''SN=NS'').
 
 
====Some other properties of eigenvalues====
 
The spectrum is [[invariant (mathematics)|invariant]] under [[similar matrix|similarity transformations]]: the matrices ''A'' and ''P''<sup>-1</sup>''AP'' have the same eigenvalues for any matrix ''A'' and any [[invertible matrix]] ''P''. The spectrum is also invariant under [[transpose|transposition]]: the matrices ''A'' and ''A''<sup>T</sup> have the same eigenvalues.
 
 
Since a linear transformation on finite dimensional spaces is [[bijective]] if and only if it is [[injective]], a matrix is invertible if and only if zero is not an eigenvalue of the matrix.
 
 
Some more consequences of the Jordan decomposition are as follows:
 
 
*a matrix is [[diagonalizable matrix|diagonalizable]] if and only if the algebraic and geometric multiplicities coincide for all its eigenvalues. In particular, an ''n''&times;''n'' matrix which has ''n'' different eigenvalues is always diagonalizable; Under the same reasoning a matrix '''A''' with eigenvectors stored in matrix '''P''' will result in '''P'''<sup>-1</sup>&sdot;'''A'''&sdot;'''P'''='''&Sigma;''' where '''&Sigma;''' is a diagonal matrix with the eigenvalues of '''A''' along the diagonal.
 
*the vector space on which the matrix acts can be viewed as a [[direct sum]] of its invariant subspaces span by its generalized eigenvectors. Each block on the diagonal corresponds to a subspace in the direct sum. When a block is diagonal, its invariant subspace is an eigenspace. Otherwise it is a generalized eigenspace, defined above;
 
*Since the [[trace (matrix)|trace]], or the sum of the elements on the main diagonal of a matrix, is preserved by unitary equivalence, the Jordan normal form tells us that it is equal to the sum of the eigenvalues;
 
*Similarly, because the eigenvalues of a [[triangular matrix]] are the entries on the [[main diagonal]], the [[determinant]] equals the product of the eigenvalues (counted according to algebraic multiplicity).
 
 
The location of the spectrum for a few subclasses of normal matrices are:
 
* All eigenvalues of a [[Hermitian matrix]] (''A'' = ''A''<sup>*</sup>) are real. Furthermore, all eigenvalues of a [[positive-definite matrix]] (''v''<sup>*</sup>''Av'' > 0 for all non-zero vectors ''v'') are positive;
 
* All eigenvalues of a [[skew-Hermitian matrix]] (''A'' = &minus;''A''<sup>*</sup>) are purely imaginary;
 
* All eigenvalues of a [[unitary matrix]] (''A''<sup>-1</sup> = ''A''<sup>*</sup>) have [[absolute value]] one;
 
 
Suppose that ''A'' is an ''m''&times;''n'' matrix, with ''m'' &le; ''n'', and that ''B'' is an ''n''&times;''m'' matrix. Then ''BA'' has the same eigenvalues as ''AB'' plus ''n'' &minus; ''m'' eigenvalues equal to zero.
 
 
Each matrix can be assigned an [[operator norm]], which depends on the norm of its domain. The operator norm of a square matrix is an upper bound for the moduli of its eigenvalues, and thus also for its [[spectral radius]]. This norm is directly related to the [[power method]] for calculating the eigenvalue of largest modulus given above. For normal matrices, the operator norm induced by the Euclidean norm is the largest moduli among its eigenvalues.
 
 
===Conjugate eigenvector===
 
A '''conjugate eigenvector''' or '''coneigenvector''' is a vector sent after transformation to a scalar multiple of its conjugate, where the scalar is called the '''conjugate eigenvalue''' or '''coneigenvalue''' of the linear transformation. The coneigenvectors and coneigenvalues represent essentially the same information and meaning as the regular eigenvectors and eigenvalues, but arise when an alternative coordinate system is used. The corresponding equation is
 
 
: <math>Av = \lambda v^*.\,</math>
 
 
For example, in coherent electromagnetic scattering theory, the linear transformation ''A'' represents the action performed by the scattering object, and the eigenvectors represent polarization states of the electromagnetic wave. In [[optics]], the coordinate system is defined from the wave's viewpoint, known as the [[Forward Scattering Alignment]] (FSA), and gives rise to a regular eigenvalue equation, whereas in [[radar]], the coordinate system is defined from the radar's viewpoint, known as the [[Back Scattering Alignment]] (BSA), and gives rise to a coneigenvalue equation.
 
 
===Generalized eigenvalue problem===
 
A '''generalized eigenvalue problem''' (2nd sense) is of the form
 
: <math> Av = \lambda B v \quad \quad</math>
 
where ''A'' and ''B'' are matrices. The '''generalized eigenvalues''' (2nd sense) &lambda;
 
can be obtained by solving the equation
 
:<math>\det(A - \lambda B)=0.\, </math>
 
The set of matrices of the form <math>A - \lambda B</math>, where <math> \lambda </math> is a complex number, is called a ''pencil''.
 
If ''B'' is invertible, then the original problem can be written in the form
 
: <math> B^{-1}Av = \lambda v \quad \quad </math>
 
which is a standard eigenvalue problem. However, in most situations it is preferable not to perform the inversion, but rather to solve the generalized eigenvalue problem as stated originally.
 
 
An example is provided by the molecular orbital application [[Eigenvalue, eigenvector, and eigenspace#Molecular orbitals|below]].
 
 
===Entries from a ring===
 
In the case of a square matrix ''A'' with entries in a [[ring (mathematics)|ring]], &lambda; is called a '''right eigenvalue''' if there exists a nonzero [[column vector]] ''x'' such that ''Ax''=&lambda;''x'', or a '''left eigenvalue''' if there exists a nonzero [[row vector]] ''y'' such that ''yA''=''y''&lambda;.
 
 
If the ring is [[commutative]], the left eigenvalues are equal to the right eigenvalues and are just called eigenvalues. If not, for instance if the ring is the set of [[quaternion]]s, they may be different.
 
   
 
==Infinite-dimensional spaces==
 
==Infinite-dimensional spaces==
[[Image:Discrete-continuum.png|thumb|250px|Fig. 3.[[absorption spectroscopy|Absorption]] [[spectroscopy|spectrum]] ([[cross section (physics)|cross section]]) of atomic [[Chlorine]]. The sharp lines obtained in theory correspond to the [[discrete spectrum]] ([[Rydberg atom|Rydberg series]]) of the [[Hamiltonian (quantum mechanics)|Hamiltonian]]; the broad structure on the right is associated with the [[continuous spectrum]] ([[ionization]]). The corresponding [[experiment]]al results have been obtained by measuring the intensity of [[X-ray]]s absorbed by a gas of atoms as a function of the incident [[photon]] [[energy]] in [[eV]].<ref>T. W Gorczyca, Auger Decay of the Photoexcited Inner Shell Rydberg Series in Neon, Chlorine, and Argon, Abstracts of the 18th International Conference on X-ray and Inner-Shell Processes, Chicago, August 23-27 (1999).</ref>]]
+
If the vector space is an infinite dimensional [[Banach space]], the notion of eigenvalues can be generalized to the concept of [[Spectrum (functional analysis)|spectrum]]. The spectrum is the set of scalars λ for which <math>\left(T-\lambda\right)^{-1}</math> is not defined; that is, such that <math>T-\lambda</math> has no [[bounded operator|bounded]] inverse.
If the vector space is an infinite dimensional [[Banach space]], the notion of eigenvalues can be generalized to the concept of [[spectrum]]. The spectrum is the set of scalars &lambda; for which <math>\left(T-\lambda\right)^{-1}</math> is not defined; that is, such that <math>T-\lambda</math> has no [[bounded operator|bounded]] inverse.
 
   
Clearly if ''&lambda;'' is an eigenvalue of ''T'', ''&lambda;'' is in the spectrum of ''T''. In general, the converse is not true. There are operators on [[Hilbert space|Hilbert]] or [[Banach space]]s which have no eigenvectors at all. This can be seen in the following example. The [[bilateral shift]] on the Hilbert space <math>\ell^2(\mathbf{Z})</math> (the space of all sequences of scalars <math>\dots a_{-1}, a_0, a_1,a_2,\dots</math> such that <math>\cdots + |a_{-1}|^2 + |a_0|^2 + |a_1|^2 + |a_2|^2 + \cdots</math> converges) has no eigenvalue but has spectral values.
+
Clearly if ''λ'' is an eigenvalue of ''T'', ''λ'' is in the spectrum of ''T''. In general, the converse is not true. There are operators on [[Hilbert space|Hilbert]] or [[Banach space]]s which have no eigenvectors at all. This can be seen in the following example. The [[bilateral shift]] on the Hilbert space <math>\ell^2(\mathbf{Z})</math> (the space of all sequences of scalars <math>\dots a_{-1}, a_0, a_1,a_2,\dots</math> such that <math>\cdots + |a_{-1}|^2 + |a_0|^2 + |a_1|^2 + |a_2|^2 + \cdots</math> converges) has no eigenvalue but has spectral values.
   
In infinite-dimensional spaces, the spectrum of a [[bounded operator]] is always nonempty. This is also true for an unbounded [[self adjoint operator]]. Via its [[spectral measure]]s, the spectrum of any self adjoint operator, bounded or otherwise, can be decomposed into absolutely continuous, pure point, and singular parts. (See [[Decomposition of spectrum (functional analysis)|Decomposition of spectrum]].)
+
In infinite-dimensional spaces, the spectrum of a [[bounded operator]] is always nonempty. This is also true for an unbounded [[self adjoint operator]]. Via its [[spectral measure]]s, the spectrum of any self adjoint operator, bounded or otherwise, can be decomposed into absolutely continuous, pure point, and singular parts. (See [[Decomposition of spectrum (functional analysis)|Decomposition of spectrum]].)
   
The exponential growth or decay <!---of WHAT??--> provides an example of a [[continuous spectrum]], as does the vibrating string example illustrated above. The [[hydrogen atom]] is an example where both types of spectra appear. The [[bound state]]s of the hydrogen atom correspond to the discrete part of the spectrum while the [[ionization]] processes are described by the continuous part. Fig. 3 exemplifies this concept in the case of the [[Chlorine]] atom.
+
Exponential functions are eigenfunctions of the derivative operator (the derivative of exponential functions are proportional to themself). [[Exponential growth]] and [[exponential decay |decay]] therefore provide examples of [[continuous spectrum| continuous spectra]], as does the vibrating string example illustrated above. The [[hydrogen atom]] is an example where both types of spectra appear. The eigenfunctions of the [[molecular Hamiltonian |hydrogen atom Hamiltonian]] are called eigenstates and are grouped into two categories. The [[bound state]]s of the hydrogen atom correspond to the discrete part of the spectrum (they have a discrete set of eigenvalues which can be computed by [[Rydberg formula]]) while the [[ionization]] processes are described by the continuous part (the energy of the collision/ionization is not quantified).
   
 
==Applications==
 
==Applications==
===Schrödinger equation===
+
===Schrödinger equation===<!-- This section is linked from [[Eigenstate]] -->
 
[[Image:HAtomOrbitals.png|thumb|271px|Fig. 4. The [[wavefunction]]s associated with the [[bound state]]s of an [[electron]] in a [[hydrogen atom]] can be seen as the eigenvectors of the [[hydrogen atom|hydrogen atom Hamiltonian]] as well as of the [[angular momentum|angular momentum operator]]. They are associated with eigenvalues interpreted as their energies (increasing downward: ''n''=1,2,3,...) and [[angular momentum]] (increasing across: ''s'', ''p'', ''d'',...). The illustration shows the square of the absolute value of the wavefunctions. Brighter areas correspond to higher [[probability density]] for a position [[measurement in quantum mechanics|measurement]]. The center of each figure is the [[atomic nucleus]], a [[proton]].]]
 
[[Image:HAtomOrbitals.png|thumb|271px|Fig. 4. The [[wavefunction]]s associated with the [[bound state]]s of an [[electron]] in a [[hydrogen atom]] can be seen as the eigenvectors of the [[hydrogen atom|hydrogen atom Hamiltonian]] as well as of the [[angular momentum|angular momentum operator]]. They are associated with eigenvalues interpreted as their energies (increasing downward: ''n''=1,2,3,...) and [[angular momentum]] (increasing across: ''s'', ''p'', ''d'',...). The illustration shows the square of the absolute value of the wavefunctions. Brighter areas correspond to higher [[probability density]] for a position [[measurement in quantum mechanics|measurement]]. The center of each figure is the [[atomic nucleus]], a [[proton]].]]
 
An example of an eigenvalue equation where the transformation <math>\mathcal{T}</math> is represented in terms of a differential operator is the time-independent [[Schrödinger equation]] in [[quantum mechanics]]:
 
An example of an eigenvalue equation where the transformation <math>\mathcal{T}</math> is represented in terms of a differential operator is the time-independent [[Schrödinger equation]] in [[quantum mechanics]]:
:<math>H\Psi_E = E\Psi_E \,</math>
+
:<math>H\psi_E = E\psi_E \,</math>
where ''H'', the [[Hamiltonian (quantum mechanics)|Hamiltonian]], is a second-order [[differential operator]] and <math>\Psi_E</math>, the [[wavefunction]], is one of its eigenfunctions corresponding to the eigenvalue ''E'', interpreted as its [[energy]].
+
where ''H'', the [[Hamiltonian (quantum mechanics)|Hamiltonian]], is a second-order [[differential operator]] and <math>\psi_E</math>, the [[wavefunction]], is one of its eigenfunctions corresponding to the eigenvalue ''E'', interpreted as its [[energy]].
   
However, in the case where one is interested only in the [[bound state]] solutions of the Schrödinger equation, one looks for <math>\Psi_E</math> within the space of [[square integrable]] functions. Since this space is a [[Hilbert space]] with a well-defined [[scalar product]], one can introduce a [[basis set]] in which <math>\Psi_E</math> and ''H'' can be represented as a one-dimensional array and a matrix respectively. This allows one to represent the Schrödinger equation in a matrix form. (Fig. 4 presents the lowest eigenfunctions of the [[Hydrogen atom]] Hamiltonian.)
+
However, in the case where one is interested only in the [[bound state]] solutions of the Schrödinger equation, one looks for <math>\psi_E</math> within the space of [[square integrable]] functions. Since this space is a [[Hilbert space]] with a well-defined [[scalar product]], one can introduce a [[Basis (linear algebra)|basis set]] in which <math>\psi_E</math> and ''H'' can be represented as a one-dimensional array and a matrix respectively. This allows one to represent the Schrödinger equation in a matrix form. (Fig. 4 presents the lowest eigenfunctions of the [[Hydrogen atom]] Hamiltonian.)
   
 
The [[Dirac notation]] is often used in this context. A vector, which represents a state of the system, in the Hilbert space of square integrable functions is represented by <math>|\Psi_E\rangle</math>. In this notation, the Schrödinger equation is:
 
The [[Dirac notation]] is often used in this context. A vector, which represents a state of the system, in the Hilbert space of square integrable functions is represented by <math>|\Psi_E\rangle</math>. In this notation, the Schrödinger equation is:
Line 129: Line 148:
   
 
===Molecular orbitals===
 
===Molecular orbitals===
In [[quantum mechanics]], and in particular in [[atomic physics|atomic]] and [[molecular physics]], within the [[Hartree-Fock]] theory, the [[atomic orbital|atomic]] and [[molecular orbital]]s can be defined by the eigenvectors of the [[Fock operator]]. The corresponding eigenvalues are interpreted as [[ionization potential]]s via [[Koopmans' theorem]]. In this case, the term eigenvector is used in a somewhat more general meaning, since the Fock operator is explicitly dependent on the orbitals and their eigenvalues. If one wants to underline this aspect one speaks of ''implicit eigenvalue equation''. Such equations are usually solved by an [[iteration]] procedure, called in this case [[self-consistent field]] method. In [[quantum chemistry]], one often represents the Hartree-Fock equation in a non-[[orthogonal]] [[basis set (chemistry)|basis set]]. This particular representation is a [[Eigenvalue, eigenvector, and eigenspace#Generalized eigenvalue problem|generalized eigenvalue problem]] called [[Roothaan equations]].
+
In [[quantum mechanics]], and in particular in [[atomic physics|atomic]] and [[molecular physics]], within the [[Hartree-Fock]] theory, the [[atomic orbital|atomic]] and [[molecular orbital]]s can be defined by the eigenvectors of the [[Fock operator]]. The corresponding eigenvalues are interpreted as [[ionization potential]]s via [[Koopmans' theorem]]. In this case, the term eigenvector is used in a somewhat more general meaning, since the Fock operator is explicitly dependent on the orbitals and their eigenvalues. If one wants to underline this aspect one speaks of ''implicit eigenvalue equation''. Such equations are usually solved by an [[iteration]] procedure, called in this case [[self-consistent field]] method. In [[quantum chemistry]], one often represents the Hartree-Fock equation in a non-[[orthogonal]] [[basis set (chemistry)|basis set]]. This particular representation is a generalized eigenvalue problem called [[Roothaan equations]].
  +
  +
===Geology and Glaciology: (Orientation Tensor)===
  +
In [[geology]], especially in the study of [[glacial till]], eigenvectors and eigenvalues are used as a method by which a mass of information of a clast fabric's constituents' orientation and dip can be summarized in a 3-D space by six numbers. In the field, a geologist may collect such data for hundreds or thousands of [[clasts]] in a soil sample, which can only be compared graphically such as in a Tri-Plot (Sneed and Folk) diagram <ref>Graham, D., and Midgley, N., 2000. Earth Surface Processes and Landforms (25) pp 1473-1477</ref>, <ref>Sneed ED, Folk RL. 1958. Pebbles in the lower Colorado River, Texas, a study of particle morphogenesis. Journal of Geology 66(2): 114–150</ref>, or as a Stereonet on a Wulff Net <ref>[http://dx.doi.org/10.1016/S0098-3004(97)00122-2 GIS-stereoplot: an interactive stereonet plotting module for ArcView 3.0 geographic information system]</ref>. The output for the orientation tensor is in the three orthogonal (perpendicular) axes of space. Eigenvectors output from programs such as Stereo32 <ref>[http://www.ruhr-uni-bochum.de/hardrock/downloads.htm Stereo32]</ref> are in the order E1 > E2 > E3, with E1 being the primary orientation of clast orientation/dip, E2 being the secondary and E3 being the tertiary, in terms of strength. The clast orientation is defined as the Eigenvector, on a compass rose of 360°. Dip is measured as the Eigenvalue, the modulus of the tensor: this is valued from 0° (no dip) to 90° (vertical). Various values of E1, E2 and E3 mean different things, as can be seen in the book 'A Practical Guide to the Study of Glacial Sediments' by Benn & Evans, 2004 <ref>Benn, D., Evans, D., 2004. A Practical Guide to the study of Glacial Sediments. London: Arnold. pp 103-107</ref>.
   
 
===Factor analysis===
 
===Factor analysis===
In [[Factor Analysis]], the eigenvectors of a [[covariance matrix]] correspond to [[factor analysis|factors]], and eigenvalues to [[factor analysis|factor loading]]s. Factor analysis is a [[statistics|statistical]] technique used in the [[social science]]s and in [[marketing]], [[product management]], [[operations research]], and other applied sciences that deal with large quantities of data. The objective is to explain most of the covariability among a number of observable [[random variable]]s in terms of a smaller number of unobservable latent variables called factors. The observable random variables are modeled as [[linear combination]]s of the factors, plus unique variance terms.
+
In [[factor analysis]], the eigenvectors of a [[covariance matrix]] or [[correlation matrix]] correspond to [[factor analysis|factors]], and eigenvalues to the variance explained by these factors. Factor analysis is a [[statistics|statistical]] technique used in the [[social science]]s and in [[marketing]], [[product management]], [[operations research]], and other applied sciences that deal with large quantities of data. The objective is to explain most of the covariability among a number of observable [[random variable]]s in terms of a smaller number of unobservable latent variables called factors. The observable random variables are modeled as [[linear combination]]s of the factors, plus unique variance terms. Eigenvalues are used in analysis used by Q-methodology software; factors with eigenvalues greater than 1.00 are considered significant, explaining an important amount of the variability in the data, while eigenvalues less than 1.00 are considered too weak, not explaining a significant portion of the data variability.
   
 
[[Image:Eigenfaces.png|thumb|200px|Fig. 5. [[Eigenface]]s as examples of eigenvectors]]
 
[[Image:Eigenfaces.png|thumb|200px|Fig. 5. [[Eigenface]]s as examples of eigenvectors]]
Line 138: Line 157:
 
===Eigenfaces===
 
===Eigenfaces===
 
In [[image processing]], processed images of [[face]]s can be seen as vectors whose components are the [[brightness]]es of each [[pixel]]. The dimension of this vector space is the number of pixels. The eigenvectors of the [[covariance matrix]] associated to a large set of normalized pictures of faces are called [[eigenface]]s. They are very useful for expressing any face image as a [[linear combination]] of some of them. In the [[Facial recognition system|facial recognition]] branch of [[biometrics]], eigenfaces provide a means of applying [[data compression]] to faces for [[Recognition of human individuals|identification]] purposes. Research related to eigen vision systems determining hand gestures has also been made. More on determining sign language letters using eigen systems can be found here: http://www.geigel.com/signlanguage/index.php
 
In [[image processing]], processed images of [[face]]s can be seen as vectors whose components are the [[brightness]]es of each [[pixel]]. The dimension of this vector space is the number of pixels. The eigenvectors of the [[covariance matrix]] associated to a large set of normalized pictures of faces are called [[eigenface]]s. They are very useful for expressing any face image as a [[linear combination]] of some of them. In the [[Facial recognition system|facial recognition]] branch of [[biometrics]], eigenfaces provide a means of applying [[data compression]] to faces for [[Recognition of human individuals|identification]] purposes. Research related to eigen vision systems determining hand gestures has also been made. More on determining sign language letters using eigen systems can be found here: http://www.geigel.com/signlanguage/index.php
  +
  +
Similar to this concept, eigenvoices concept is also developed which represents the general direction of variability in human pronunciations of a particular utterance, such as a word in a language. Based on a linear combination of such eigenvoices, a new voice pronunciation of the word can be constructed. These concepts have been found useful in automatic speech recognition systems, for speaker adaptation.
   
 
===Tensor of inertia===
 
===Tensor of inertia===
In [[mechanics]], the eigenvectors of the [[moment of inertia#Inertia tensor|inertia tensor]] define the [[principal axes]] of a [[rigid body]]. The tensor of inertia is a key quantity required in order to determine the rotation of a rigid body around its [[center of mass]].
+
In [[mechanics]], the eigenvectors of the [[moment of inertia#Inertia tensor|inertia tensor]] define the [[principal axes]] of a [[rigid body]]. The [[tensor]] of [[inertia]] is a key quantity required in order to determine the rotation of a rigid body around its [[center of mass]].
   
 
===Stress tensor===
 
===Stress tensor===
In [[solid mechanics]], the [[stress tensor]] is symmetric and so can be decomposed into a [[diagonal]] tensor with the eigenvalues on the diagonal and eigenvectors as a basis. Because it is diagonal, in this orientation, the stress tensor has no [[shear]] components; the components it does have are the principal components.
+
In [[solid mechanics]], the [[stress tensor]] is symmetric and so can be decomposed into a [[diagonal]] tensor with the eigenvalues on the diagonal and eigenvectors as a basis. Because it is diagonal, in this orientation, the stress tensor has no [[Shear (mathematics)|shear]] components; the components it does have are the principal components.
   
 
===Eigenvalues of a graph===
 
===Eigenvalues of a graph===
In [[spectral graph theory]], an eigenvalue of a [[graph theory|graph]] is defined as an eigenvalue of the graph's [[adjacency matrix]] ''A'', or (increasingly) of the graph's Laplacian matrix, which is either ''T''&minus;''A'' or <math>I-T^{-1/2}AT^{-1/2}</math>, where ''T'' is a diagonal matrix holding the degree of each vertex, and in <math>T^{-1/2}</math>, 0 is substituted for <math>0^{-1/2}</math>.
+
In [[spectral graph theory]], an eigenvalue of a [[graph theory|graph]] is defined as an eigenvalue of the graph's [[adjacency matrix]] ''A'', or (increasingly) of the graph's [[Discrete Laplace operator|Laplacian]] matrix, which is either ''T''&minus;''A'' or ''I''&minus;''T''<sup> 1/2</sup>''AT''<sup> &minus;1/2</sup>, where ''T'' is a diagonal matrix holding the degree of each vertex, and in ''T''<sup> &minus;1/2</sup>, 0 is substituted for 0<sup>&minus;1/2</sup>. The ''k''th principal eigenvector of a graph is defined as either the eigenvector corresponding to the ''k''th largest eigenvalue of ''A'', or the eigenvector corresponding to the ''k''th smallest eigenvalue of the Laplacian. The first principal eigenvector of the graph is also referred to merely as the principal eigenvector.
  +
  +
The principal eigenvector is used to measure the [[Eigenvector centrality|centrality]] of its vertices. An example is [[Google]]'s [[PageRank]] algorithm. The principal eigenvector of a modified [[adjacency matrix]] of the World Wide Web graph gives the page ranks as its components. This vector corresponds to the [[stationary distribution]] of the [[Markov chain]] represented by the row-normalized adjacency matrix; however, the adjacency matrix must first be modified to ensure a stationary distribution exists. The second principal eigenvector can be used to partition the graph into clusters, via [[Data clustering#Spectral clustering|spectral clustering]].
  +
  +
== See also ==
   
The principal eigenvector of a graph is used to measure the [[Eigenvector centrality|centrality]] of its vertices. An example is [[Google]]'s [[PageRank]] algorithm. The principal eigenvector of a modified [[adjacency matrix]] of the www-graph gives the [[page rank]]s as its components. The two eigenvectors with largest positive eigenvalues can also be used as ''x''- and ''y''-coordinates of vertices in [[Graph drawing|drawing]] the graph via [[spectral layout]] methods. The second eigenvector can be used to partition the graph into clusters, via [[Data clustering#Spectral clustering|spectral clustering]].
+
* [[Nonlinear eigenproblem]]
  +
* [[Quadratic eigenvalue problem]]
   
 
==Notes==
 
==Notes==
<references />
+
{{Wikibooks|The Book of Mathematical Proofs|Algebra/Linear Transformations}}
  +
{{reflist|2}}
   
 
==References==
 
==References==
*{{cite paper | author=Abdi, H | title = [http://www.utdallas.edu/~herve/Abdi-EVD2007-pretty.pdf] ((2007). Eigen-decomposition: eigenvalues and eigenvecteurs.In N.J. Salkind (Ed.): Encyclopedia of Measurement and Statistics. Thousand Oaks (CA): Sage.| year = 2007 |}}
+
<div class="references">
* John Aldrich, Eigenvalue, eigenfunction, eigenvector, and related terms. In Jeff Miller (Editor), ''[http://members.aol.com/jeff570/e.html Earliest Known Uses of Some of the Words of Mathematics]'', last updated [[7 August]] [[2006]], accessed [[22 August]] [[2006]].
+
*{{Citation
* [[Claude Cohen-Tannoudji]], ''Quantum Mechanics'', Wiley (1977). ISBN 0-471-16432-1. (Chapter II. The mathematical tools of quantum mechanics.)
+
| last=Korn
* John B. Fraleigh and Raymond A. Beauregard, ''Linear Algebra'' (3<sup>rd</sup> edition), Addison-Wesley Publishing Company (1995). ISBN 0-201-83999-7 (international edition).
+
| first=Granino A.
* Gene H. Golub and Charles F. van Loan, ''Matrix Computations'' (3<sup>rd</sup> edition), Johns Hopkins University Press, Baltimore, 1996. ISBN 978-0-8018-5414-9.
+
| first2=Theresa M.
* T. Hawkins, Cauchy and the spectral theory of matrices, ''Historia Mathematica'', vol. 2, pp. 1–29, 1975.
+
| last2=Korn
* Roger A. Horn and Charles R. Johnson, ''Matrix Analysis'', Cambridge University Press, 1985. ISBN 0-521-30586-1 (hardback), ISBN 0-521-38632-2 (paperback).
+
| title=Mathematical Handbook for Scientists and Engineers: Definitions, Theorems, and Formulas for Reference and Review
* Morris Kline, ''Mathematical thought from ancient to modern times'', Oxford University Press, 1972. ISBN 0-19-501496-0.
+
| publisher=1152 p., Dover Publications, 2 Revised edition
* Carl D. Meyer, ''Matrix Analysis and Applied Linear Algebra'', Society for Industrial and Applied Mathematics (SIAM), Philadelphia, 2000. ISBN 978-0-89871-454-8.
+
| year=2000
*{{cite paper | author=Valentin, D.,Abdi, H, Edelman, B., O'Toole A. | title = [http://www.utdallas.edu/~herve/abdi.vaeo97.pdf] (1997). Principal Component and Neural Network Analyses of Face Images: What Can Be Generalized in Gender Classification? Journal of Mathematical Psychology, 41, 398-412.|}}
+
| isbn=0-486-41147-8
  +
}}.
  +
*John Aldrich, Eigenvalue, eigenfunction, eigenvector, and related terms. In Jeff Miller (Editor), ''[http://members.aol.com/jeff570/e.html Earliest Known Uses of Some of the Words of Mathematics]'', last updated [[7 August]] [[2006]], accessed [[22 August]] [[2006]].
  +
*{{Citation
  +
| last=Strang
  +
| first=Gilbert
  +
| title=Introduction to linear algebra
  +
| publisher=Wellesley-Cambridge Press, Wellesley, MA
  +
| year=1993
  +
| isbn=0-961-40885-5
  +
}}.
  +
*{{Citation
  +
| last=Strang
  +
| first=Gilbert
  +
| title=Linear algebra and its applications
  +
| publisher=Thomson, Brooks/Cole, Belmont, CA
  +
| year=2006
  +
| isbn=0-030-10567-6
  +
}}.
  +
*{{Citation
  +
| last=Bowen
  +
| first=Ray M.
  +
| first2=Chao-Cheng
  +
| last2=Wang
  +
| title=Linear and multilinear algebra
  +
| publisher=Plenum Press, New York, NY
  +
| year=1980
  +
| isbn=0-306-37508-7
  +
}}.
  +
*[[Claude Cohen-Tannoudji]], ''Quantum Mechanics'', Wiley (1977). ISBN 0-471-16432-1. (Chapter II. The mathematical tools of quantum mechanics.)
  +
*John B. Fraleigh and Raymond A. Beauregard, ''Linear Algebra'' (3<sup>rd</sup> edition), Addison-Wesley Publishing Company (1995). ISBN 0-201-83999-7 (international edition).
  +
*{{Citation
  +
| last=Golub
  +
| first=Gene H.
  +
| first2=Charles F.
  +
| last2=van Loan
  +
| title=Matrix computations (3rd Edition)
  +
| publisher=Johns Hopkins University Press, Baltimore, MD
  +
| year=1996
  +
| isbn=978-0-8018-5414-9
  +
}}.
  +
*T. Hawkins, Cauchy and the spectral theory of matrices, ''Historia Mathematica'', vol. 2, pp. 1–29, 1975.
  +
*Roger A. Horn and Charles R. Johnson, ''Matrix Analysis'', Cambridge University Press, 1985. ISBN 0-521-30586-1 (hardback), ISBN 0-521-38632-2 (paperback).
  +
*{{Citation
  +
| last=Kline
  +
| first=Morris
  +
| title=Mathematical thought from ancient to modern times
  +
| publisher=Oxford University Press
  +
| year=1972
  +
| isbn=0-195-01496-0
  +
}}.
  +
*{{Citation
  +
| last=Meyer
  +
| first=Carl D.
  +
| title=Matrix analysis and applied linear algebra
  +
| publisher=Society for Industrial and Applied Mathematics (SIAM), Philadelphia
  +
| year=2000
  +
| isbn=978-0-89871-454-8
  +
}}.
  +
*Brown, Maureen, "Illuminating Patterns of Perception: An Overview of Q Methodology" October 2004
  +
*Gene H. Golub and Henk A. van der Vorst, "Eigenvalue computation in the 20th century," Journal of Computational and Applied Mathematics 123, 35-65 (2000).
  +
*Max A. Akivis and Vladislav V. Goldberg, ''Tensor calculus'' (in Russian), Science Publishers, Moscow, 1969.
  +
*{{Citation
  +
| last=Gelfand
  +
| first=I. M.
  +
| title=Lecture notes in linear algebra
  +
| location=Russian
  +
| publisher=Science Publishers, Moscow
  +
| year=1971
  +
| isbn=
  +
}}
  +
*Pavel S. Alexandrov, ''Lecture notes in analytical geometry'' (in Russian), Science Publishers, Moscow, 1968.
  +
*Carter, Tamara A., Richard A. Tapia, and Anne Papaconstantinou, [http://ceee.rice.edu/Books/LA/index.html| Linear Algebra]: An Introduction to Linear Algebra for Pre-Calculus Students, Rice University, Online Edition, Retrieved on 2008-02-19.
  +
*Steven Roman, ''Advanced linear algebra'' 3rd Edition, Springer Science + Business Media, LLC, New York, NY, 2008. ISBN 978-0-387-72828-5
  +
*{{Citation
  +
| last=Shilov
  +
| first=G. E.
  +
| title=Finite-dimensional (linear) vector spaces
  +
| location=Russian
  +
| publisher=State Technical Publishing House, 3rd Edition, Moscow
  +
| year=1969
  +
| isbn=
  +
}}.
  +
*{{Citation
  +
| last=Hefferon
  +
| first=Jim
  +
| title=Linear Algebra
  +
| publisher=Online book, St Michael's College, Colchester, Vermont, USA
  +
| url=http://joshua.smcvt.edu/linearalgebra/
  +
| year=2001
  +
| isbn=
  +
}}.
  +
*{{Citation
  +
| last=Kuttler
  +
| first=Kenneth
  +
| title=An introduction to linear algebra
  +
| publisher=Online e-book in PDF format, Brigham Young University
  +
| url=http://www.math.byu.edu/~klkuttle/Linearalgebra.pdf
  +
| year=2007
  +
| isbn=
  +
}}.
  +
*James W. Demmel, ''Applied Numerical Linear Algebra'', SIAM, 1997, ISBN 0-89871-389-7.
  +
*Robert A. Beezer, [http://linear.ups.edu/ ''A First Course In Linear Algebra''], Free online book under GNU licence, University of Puget Sound, 2006
  +
*Lancaster, P. ''Matrix Theory'' (in Russian), Science Publishers, Moscow, 1973, 280 p.
  +
*Paul R. Halmos, ''Finite-Dimensional Vector Spaces'', 8th Edition, Springer-Verlag, New York, 1987, 212 p., ISBN 0387900934
  +
*Pigolkina, T. S. and Shulman, V. S., ''Eigenvalue'' (in Russian), In:Vinogradov, I. M. (Ed.), ''Mathematical Encyclopedia'', Vol. 5, Soviet Encyclopedia, Moscow, 1977.
  +
*Pigolkina, T. S. and Shulman, V. S., ''Eigenvector'' (in Russian), In:Vinogradov, I. M. (Ed.), ''Mathematical Encyclopedia'', Vol. 5, Soviet Encyclopedia, Moscow, 1977.
  +
*{{Citation
  +
| last=Greub
  +
| first=Werner H.
  +
| title=Linear Algebra (4th Edition)
  +
| publisher=Springer-Verlag, New York, NY
  +
| year=1975
  +
| isbn=0-387-90110-8
  +
}}.
  +
*Larson, Ron and Bruce H. Edwards, ''Elementary Linear Algebra'', 5th Edition, Houghton Mifflin Company, 2003, ISBN 0618335676.
  +
*Curtis, Charles W., ''Linear Algebra: An Introductory Approach'', 347 p., Springer; 4th ed. 1984. Corr. 7th printing edition (August 19, 1999), ISBN 0387909923.
  +
*{{Citation
  +
| last=Shores
  +
| first=Thomas S.
  +
| title=Applied linear algebra and matrix analysis
  +
| publisher=Springer Science+Business Media, LLC
  +
| year=2007
  +
| isbn=0-387-33194-8
  +
}}.
  +
*{{Citation
  +
| last=Sharipov
  +
| first=Ruslan A.
  +
| title=Course of Linear Algebra and Multidimensional Geometry: the textbook
  +
| publisher=Online e-book in PDF format, Bashkir State University, Ufa
  +
| url=http://www.geocities.com/r-sharipov
  +
| year=1996
  +
| isbn=5-7477-0099-5
  +
| id={{arxiv|math|0405323v1}}
  +
}}.
  +
</div>
   
 
==External links==
 
==External links==
{{Wikibooks|Algebra/Eigenvalues and eigenvectors}}
+
{{Wikibooks|Algebra|Eigenvalues and eigenvectors}}
*[http://www.caam.rice.edu/software/ARPACK/ ARPACK] is a collection of FORTRAN subroutines for solving large scale (sparse) eigenproblems
+
{{Wikibooks|Linear Algebra|Eigenvalues and eigenvectors}}
  +
*[http://video.google.com/videoplay?docid=-8791056722738431468&hl=en MIT Video Lecture on Eigenvalues and Eigenvectors] at Google Video, from MIT OpenCourseWare
  +
*[http://www.caam.rice.edu/software/ARPACK/ ARPACK] is a collection of FORTRAN subroutines for solving large scale (sparse) eigenproblems.
  +
*[http://www.math.uri.edu/~jbaglama/ IRBLEIGS], has [[MATLAB]] code with similar capabilities to ARPACK. (See [http://www.math.uri.edu/~jbaglama/papers/paper10.pdf this paper] for a comparison between IRBLEIGS and ARPACK.)
 
*[http://netlib.org/lapack/ LAPACK] is a collection of FORTRAN subroutines for solving dense linear algebra problems
 
*[http://netlib.org/lapack/ LAPACK] is a collection of FORTRAN subroutines for solving dense linear algebra problems
 
*[http://www.alglib.net/eigen/ ALGLIB] includes a partial port of the LAPACK to C++, C#, Delphi, etc.
 
*[http://www.alglib.net/eigen/ ALGLIB] includes a partial port of the LAPACK to C++, C#, Delphi, etc.
Line 175: Line 196:
 
*[http://www.bluebit.gr/matrix-calculator/ Online Matrix Calculator] Calculates eigenvalues, eigenvectors and other decompositions of matrices online
 
*[http://www.bluebit.gr/matrix-calculator/ Online Matrix Calculator] Calculates eigenvalues, eigenvectors and other decompositions of matrices online
 
* [http://www.vrand.com Vanderplaats Research and Development] - Provides the [http://www.vrand.com SMS] eigenvalue solver for Structural Finite Element. The solver is in the [http://www.vrand.com/Genesis.html ''GENESIS''] program as well as other commercial programs. SMS can be easily use with MSC.Nastran or NX/Nastran via DMAPs.
 
* [http://www.vrand.com Vanderplaats Research and Development] - Provides the [http://www.vrand.com SMS] eigenvalue solver for Structural Finite Element. The solver is in the [http://www.vrand.com/Genesis.html ''GENESIS''] program as well as other commercial programs. SMS can be easily use with MSC.Nastran or NX/Nastran via DMAPs.
*[http://ocw.mit.edu/OcwWeb/Mathematics/18-06Spring-2005/VideoLectures/index.htm Videos of MIT Linear Algebra Course, spring 2005] - See Lecture ''Eigenvalues and Eigenvectors''
 
 
*[http://www.physlink.com/education/AskExperts/ae520.cfm What are Eigen Values?] from PhysLink.com's "Ask the Experts"
 
*[http://www.physlink.com/education/AskExperts/ae520.cfm What are Eigen Values?] from PhysLink.com's "Ask the Experts"
*[http://www.cs.utk.edu/~dongarra/etemplates/index.html Templates for the Solution of Algebraic Eigenvalue Problems] Edited by Zhaojun Bai, James Demmel, Jack Dongarra, Axel Ruhe, and Henk van der Vorst (a guide to the numerical solution of eigenvalue problems)
+
*[http://www.cs.utk.edu/~dongarra/etemplates/index.html Templates for the Solution of Algebraic Eigenvalue Problems] Edited by Zhaojun Bai, James Demmel, Jack Dongarra, Axel Ruhe, and [[Henk van der Vorst]] (a guide to the numerical solution of eigenvalue problems)
   
  +
[[Category:Fundamental physics concepts]]
 
[[Category:Abstract algebra]]
 
[[Category:Abstract algebra]]
 
[[Category:Linear algebra]]
 
[[Category:Linear algebra]]
  +
[[Category:Matrices]]
 
[[Category:Singular value decomposition]]
 
[[Category:Singular value decomposition]]
 
[[Category:German loanwords]]
 
[[Category:German loanwords]]
   
  +
<!--
 
{{Link FA|es}}
 
{{Link FA|es}}
 
{{Link FA|it}}
 
{{Link FA|it}}
 
{{Link FA|zh}}
 
{{Link FA|zh}}
 
 
[[ar:قيمة ذاتية]]
 
[[ar:قيمة ذاتية]]
[[be:Уласныя лікі, вэктары й прасторы]]
+
[[be-x-old:Уласныя лікі, вэктары й прасторы]]
 
[[cs:Vlastní číslo]]
 
[[cs:Vlastní číslo]]
 
[[da:Egenværdi, egenvektor og egenrum]]
 
[[da:Egenværdi, egenvektor og egenrum]]
Line 198: Line 220:
 
[[it:Autovettore e autovalore]]
 
[[it:Autovettore e autovalore]]
 
[[he:ערך עצמי]]
 
[[he:ערך עצמי]]
  +
[[lt:Tikrinių verčių lygtis]]
  +
[[hu:Sajátvektor és sajátérték]]
 
[[nl:Eigenwaarde (wiskunde)]]
 
[[nl:Eigenwaarde (wiskunde)]]
 
[[ja:固有値]]
 
[[ja:固有値]]
Line 203: Line 227:
 
[[pl:Wartość własna]]
 
[[pl:Wartość własna]]
 
[[pt:Valor próprio]]
 
[[pt:Valor próprio]]
[[ro:Valoare proprie]]
+
[[ro:Vector propriu]]
 
[[ru:Собственные векторы, значения и пространства]]
 
[[ru:Собственные векторы, значения и пространства]]
 
[[sl:Lastna vrednost]]
 
[[sl:Lastna vrednost]]
Line 209: Line 233:
 
[[sv:Egenvärde, egenvektor]]
 
[[sv:Egenvärde, egenvektor]]
 
[[vi:Vectơ riêng]]
 
[[vi:Vectơ riêng]]
[[ur:ویژہ قیمت]]
+
[[uk:Власний вектор]]
  +
[[ur:ویژہ قدر]]
 
[[zh:特征向量]]
 
[[zh:特征向量]]
 
-->
 
-->

Revision as of 18:51, March 12, 2008

Assessment | Biopsychology | Comparative | Cognitive | Developmental | Language | Individual differences | Personality | Philosophy | Social |
Methods | Statistics | Clinical | Educational | Industrial | Professional items | World psychology |

Statistics: Scientific method · Research methods · Experimental design · Undergraduate statistics courses · Statistical tests · Game theory · Decision theory


Mona Lisa with eigenvector
Fig. 1. In this shear mapping of the Mona Lisa, the picture was deformed in such a way that its central vertical axis (red vector) was not modified, but the diagonal vector (blue) has changed direction. Hence the red vector is an eigenvector of the transformation and the blue vector is not. Since the red vector was neither stretched nor compressed, its eigenvalue is 1. All vectors with the same vertical direction - i.e., parallel to this vector - are also eigenvectors, with the same eigenvalue. Together with the zero-vector, they form the eigenspace for this eigenvalue.
PhloxBotAdded by PhloxBot

In mathematics, a vector may be thought of as an arrow. It has a length, called its magnitude, and it points in some particular direction. A linear transformation may be considered to operate on a vector to change it, usually changing both its magnitude and its direction. An eigenvector of a given linear transformation is a vector which is multiplied by a constant called the eigenvalue during that transformation. The direction of the eigenvector is either unchanged by that transformation (for positive eigenvalues) or reversed (for negative eigenvalues).

For example, an eigenvalue of +2 means that the eigenvector is doubled in length and points in the same direction. An eigenvalue of +1 means that the eigenvector is unchanged, while an eigenvalue of −1 means that the eigenvector is reversed in direction. An eigenspace of a given transformation is the span of the eigenvectors of that transformation with the same eigenvalue, together with the zero vector (which has no direction). An eigenspace is an example of a subspace of a vector space.

In linear algebra, every linear transformation between finite-dimensional vector spaces can be given by a matrix, which is a rectangular array of numbers arranged in rows and columns. Standard methods for finding eigenvalues, eigenvectors, and eigenspaces of a given matrix are discussed below.

These concepts play a major role in several branches of both pure and applied mathematics — appearing prominently in linear algebra, functional analysis, and to a lesser extent in nonlinear mathematics.

Many kinds of mathematical objects can be treated as vectors: functions, harmonic modes, quantum states, and frequencies, for example. In these cases, the concept of direction loses its ordinary meaning, and is given an abstract definition. Even so, if this abstract direction is unchanged by a given linear transformation, the prefix "eigen" is used, as in eigenfunction, eigenmode, eigenstate, and eigenfrequency.

History

Eigenvalues are often introduced in the context of linear algebra or matrix theory. Historically, however, they arose in the study of quadratic forms and differential equations.

Euler had also studied the rotational motion of a rigid body and discovered the importance of the principal axes. As Lagrange realized, the principal axes are the eigenvectors of the inertia matrix.[1] In the early 19th century, Cauchy saw how their work could be used to classify the quadric surfaces, and generalized it to arbitrary dimensions.[2] Cauchy also coined the term racine caractéristique (characteristic root) for what is now called eigenvalue; his term survives in characteristic equation.[3]

Fourier used the work of Laplace and Lagrange to solve the heat equation by separation of variables in his famous 1822 book Théorie analytique de la chaleur.[4] Sturm developed Fourier's ideas further and he brought them to the attention of Cauchy, who combined them with his own ideas and arrived at the fact that symmetric matrices have real eigenvalues.[2] This was extended by Hermite in 1855 to what are now called Hermitian matrices.[3] Around the same time, Brioschi proved that the eigenvalues of orthogonal matrices lie on the unit circle,[2] and Clebsch found the corresponding result for skew-symmetric matrices.[3] Finally, Weierstrass clarified an important aspect in the stability theory started by Laplace by realizing that defective matrices can cause instability.[2]

In the meantime, Liouville studied eigenvalue problems similar to those of Sturm; the discipline that grew out of their work is now called Sturm-Liouville theory.[5] Schwarz studied the first eigenvalue of Laplace's equation on general domains towards the end of the 19th century, while Poincaré studied Poisson's equation a few years later.[6]

At the start of the 20th century, Hilbert studied the eigenvalues of integral operators by viewing the operators as infinite matrices.[7] He was the first to use the German word eigen to denote eigenvalues and eigenvectors in 1904, though he may have been following a related usage by Helmholtz. "Eigen" can be translated as "own", "peculiar to", "characteristic" or "individual"—emphasizing how important eigenvalues are to defining the unique nature of a specific transformation. For some time, the standard term in English was "proper value", but the more distinctive term "eigenvalue" is standard today.[8]

The first numerical algorithm for computing eigenvalues and eigenvectors appeared in 1929, when Von Mises published the power method. One of the most popular methods today, the QR algorithm, was proposed independently by Francis and Kublanovskaya in 1961.[9]

Definitions: the eigenvalue equation

See also: Eigenplane

Linear transformations of a vector space, such as rotation, reflection, stretching, compression, shear or any combination of these, may be visualized by the effect they produce on vectors. In other words, they are vector functions. More formally, in a vector space L a vector function A is defined if for each vector x of L there corresponds a unique vector y = A(x) of L. For the sake of brevity, the parentheses around the vector on which the transformation is acting are often omitted. A vector function A is linear if it has the following two properties:

additivity \ A(\mathbf{x}+\mathbf{y})=A(\mathbf{x})+A(\mathbf{y})
homogeneity \ A(\alpha \mathbf{x})=\alpha A(\mathbf{x})

where x and y are any two vectors of the vector space L and α is any real number. Such a function is variously called a linear transformation, linear operator, or linear endomorphism on the space L.

Given a linear transformation A, a non-zero vector x is defined to be an eigenvector of the transformation if it satisfies the eigenvalue equation A \mathbf{x} = \lambda \mathbf{x} for some scalar λ. In this situation, the scalar λ is called an eigenvalue of A corresponding to the eigenvector x.

The key equation in this definition is the eigenvalue equation, Ax = λx. Most vectors x will not satisfy such an equation. A typical vector x changes direction when acted on by A, so that Ax is not a multiple of x. This means that only certain special vectors x are eigenvectors, and only certain special numbers λ are eigenvalues. Of course, if A is a multiple of the identity matrix, then no vector changes direction, and all non-zero vectors are eigenvectors. But in the usual case, eigenvectors are few and far between. They are the "normal modes" of the system, and they act independently.[10]

The requirement that the eigenvector be non-zero is imposed because the equation A0 = λ0 holds for every A and every λ. Since the equation is always trivially true, it is not an interesting case. In contrast, an eigenvalue can be zero in a nontrivial way. An eigenvalue can be, and usually is, also a complex number. In the definition given above, eigenvectors and eigenvalues do not occur independently. Instead, each eigenvector is associated with a specific eigenvalue. For this reason, an eigenvector x and a corresponding eigenvalue λ are often referred to as an eigenpair. One eigenvalue can be associated with several or even with infinite number of eigenvectors. But conversely, if an eigenvector is given, the associated eigenvalue for this eigenvector is unique. Indeed, from the equality Ax = λx = λ'x and from x0 it follows that λ = λ'.[11]

File:Eigenvalue equation.svg
Fig. 2. The eigenvalue equation as a homothety (similarity transformation) on the vector x.

Geometrically (Fig. 2), the eigenvalue equation means that under the transformation A eigenvectors experience only changes in magnitude and sign — the direction of Ax is the same as that of x. This type of linear transformation is defined as homothety (dilatation[12], similarity transformation). The eigenvalue λ is simply the amount of "stretch" or "shrink" to which a vector is subjected when transformed by A. If λ = 1, the vector remains unchanged (unaffected by the transformation). A transformation I under which a vector x remains unchanged, Ix = x, is defined as identity transformation. If λ = –1, the vector flips to the opposite direction (rotates to 180°); this is defined as reflection.

If x is an eigenvector of the linear transformation A with eigenvalue λ, then any vector y = αx is also an eigenvector of A with the same eigenvalue. From the homogeneity of the transformation A it follows that Ay = α(Ax) = α(λx) = λ(αx) = λy. Similarly, using the additivity property of the linear transformation, it can be shown that any linear combination of eigenvectors with eigenvalue λ has the same eigenvalue λ.[13] Therefore, any non-zero vector in the line through x and the zero vector is an eigenvector with the same eigenvalue as x. Together with the zero vector, those eigenvectors form a subspace of the vector space called an eigenspace. The eigenvectors corresponding to different eigenvalues are linearly independent[14] meaning, in particular, that in an n-dimensional space the linear transformation A cannot have more than n eigenvectors with different eigenvalues.[15] The vectors of the eigenspace generate a linear subspace of A which is invariant (unchanged) under this transformation.[16]

If a basis is defined in vector space Ln, all vectors can be expressed in terms of components. Polar vectors can be represented as one-column matrices with n rows where n is the space dimensionality. Linear transformations can be represented with square matrices; to each linear transformation A of Ln corresponds a square matrix of rank n. Conversely, to each square matrix of rank n corresponds a linear transformation of Ln at a given basis. Because of the additivity and homogeneity of the linear trasformation and the eigenvalue equation (which is also a linear transformation — homothety), those vector functions can be expressed in matrix form. Thus, in a the two-dimensional vector space L2 fitted with standard basis, the eigenvector equation for a linear transformation A can be written in the following matrix representation:

 \begin{bmatrix} a_{11} & a_{12} \\ a_{21} & a_{22} \end{bmatrix} \begin{bmatrix} x \\ y \end{bmatrix} = \lambda \begin{bmatrix} x \\ y \end{bmatrix},

where the juxtaposition of matrices means matrix multiplication. This is equivalent to a set of n linear equations, where n is the number of basis vectors in the basis set. In these equations both the eigenvalue λ and the components of x are unknown variables.

The eigenvectors of A as defined above are also called right eigenvectors because they are column vectors that stand on the right side of the matrix A in the eigenvalue equation. If there exists a transposed matrix AT that satifies the eigenvalue equation, that is, if ATx = λx, then λxT = (λx)T = (ATx)T = xTA, or xTA = λxT. The last equation is similar to the eigenvalue equation but instead of the column vector x it contains its transposed vector, the row vector xT, which stands on the left side of the matrix A. The eigenvectors that satisfy the eigenvalue equation xTA = λxT are called left eigenvectors. They are row vectors.[17] In many common applications, only right eigenvectors need to be considered. Hence the unqualified term "eigenvector" can be understood to refer to a right eigenvector. Eigenvalue equations, written in terms of right or left eigenvectors (Ax = λx and xTA = λxT) have the same eigenvalue λ.[18]

An eigenvector is defined to be a principal or dominant eigenvector if it corresponds to the eigenvalue of largest magnitude (for real numbers, largest absolute value). Repeated application of a linear transformation to an arbitrary vector results in a vector proportional (collinear) to the principal eigenvector.[18]

The applicability the eigenvalue equation to general matrix theory extends the use of eigenvectors and eigenvalues to all matrices, and thus greatly extends the scope of use of these mathematical constructs not only to transformations in linear vector spaces but to all fields of science that use matrices: linear equations systems, optimization, vector and tensor calculus, all fields of physics that use matrix quantities, particularly quantum physics, relativity, and electrodynamics, as well as many engineering applications.

Characteristic equation

Main article: Characteristic equation
Main article: Characteristic polynomial

The determination of the eigenvalues and eigenvectors is important in virtually all areas of physics and many engineering problems, such as stress calculations, stability analysis, oscillations of vibrating systems, etc. It is equivalent to matrix diagonalization, and is the first step of orthogonalization, finding of invariants, optimization (minimization or maximization), analysis of linear systems, and many other common applications.

The usual method of finding all eigenvectors and eigenvalues of a system is first to get rid of the unknown components of the eigenvectors, then find the eigenvalues, plug those back one by one in the eigenvalue equation in matrix form and solve that as a system of linear equations to find the components of the eigenvectors. From the identity transformation Ix = x, where I is the identity matrix, x in the eigenvalue equation can be replaced by Ix to give:

A \mathbf{x} = \lambda I \mathbf{x}

The identity matrix is needed to keep matrices, vectors, and scalars straight; the equation (A − λ) x = 0 is shorter, but mixed up since it does not differentiate between matrix, scalar, and vector.[19] The expression in the right hand side is transferred to left hand side with a negative sign, leaving 0 on the right hand side:

A \mathbf{x} - \lambda I \mathbf{x} = 0

The eigenvector x is pulled out behind parentheses:

(A - \lambda I) \mathbf{x} = 0

This can be viewed as a linear system of equations in which the coefficient matrix is the expression in the parentheses, the matrix of the unknowns is x, and the right hand side matrix is zero. According to Cramer's rule, this system of equations has non-trivial solutions (not all zeros, or not any number) if and only if its determinant vanishes, so the solutions of the equation are given by:

\det(A - \lambda I) = 0 \,

This equation is defined as the characteristic equation (less often, secular equation) of A, and the left-hand side is defined as the characteristic polynomial. The eigenvector x or its components are not present in the characteristic equation, so at this stage they are dispensed with, and the only unknowns that remain to be calculated are the eigenvalues (the components of matrix A are given, i. e, known beforehand). For a vector space L2, the transformation A is a 2 × 2 square matrix, and the characteristic equation can be written in the following form:

\begin{vmatrix} a_{11} - \lambda & a_{12}\\a_{21} & a_{22} - \lambda\end{vmatrix} = 0

Expansion of the determinant in the left hand side results in a characteristic polynomial which is a monic (its leading coefficient is 1) polynomial of the second degree, and the characteristic equation is the quadratic equation

\lambda^2 - \lambda (a_{11} + a_{22}) + (a_{11} a_{22} - a_{12} a_{21}) = 0, \,

which has the following solutions (roots):

\lambda_{1,2} = \frac{1}{2} \left [(a_{11} + a_{22}) \pm \sqrt{4a_{12} a_{21} + (a_{11} - a_{22})^2} \right ].

For real matrices, the coefficients of the characteristic polynomial are all real. The number and type of roots depends on the value of the discriminant, Δ. For cases Δ = 0, Δ > 0, or Δ < 0, respectively, the roots are one real, two real, or two complex. If the roots are complex, they are also complex conjugates of each other. When the number of roots is less than the degree of the characteristic polynomial (the latter is also the rank of the matrix, and the number of dimensions of the vector space) the equation has a multiple root. In the case of a quadratic equation with one root, this root is a double root, or a root with multiplicity 2. A root with a multiplicity of 1 is a simple root. A quadratic equation with two real or complex roots has only simple roots. In general, the algebraic multiplicity of an eigenvalue is defined as the multiplicity of the corresponding root of the characteristic polynomial. The spectrum of a transformation on a finite dimensional vector space is defined as the set of all its eigenvalues. In the infinite-dimensional case, the concept of spectrum is more subtle and depends on the topology of the vector space.

The general formula for the characteristic polynomial of an n-square matrix is

p(\lambda) = \sum_{k=0}^n (-1)^k S_k \lambda^{n-k},

where S0 = 1, S1 = tr(A), the trace of the transformation matrix A, and Sk with k > 1 are the sums of the principal minors of order k.[20] The fact that eigenvalues are roots of an n-order equation shows that a linear transformation of an n-dimensional linear space has at most n different eigenvalues.[21] According to the fundamental theorem of algebra, in a complex linear space, the characteristic polynomial has at least one zero. Consequently, every linear transformation of a complex linear space has at least one eigenvalue. [22][23] For real linear spaces, if the dimension is an odd number, the linear transformation has at least one eigenvalue; if the dimension is an even number, the number of eigenvalues depends on the determinant of the transformation matrix: if the determinant is negative, there exists at least one positive and one negative eigenvalue, if the determinant is positive nothing can be said about existence of eigenvalues.[24] The complexity of the problem for finding roots/eigenvalues of the characteristic polynomial increases rapidly with increasing the degree of the polynomial (the dimension of the vector space), n. Thus, for n = 3, eigenvalues are roots of the cubic equation, for n = 4 — roots of the quartic equation. For n > 4 there are no exact solutions and one has to resort to root-finding algorithms, such as Newton's method (Horner's method) to find numerical approximations of eigenvalues. For large symmetric sparse matrices, Lanczos algorithm is used to compute eigenvalues and eigenvectors.

In order to find the eigenvectors, the eigenvalues thus found as roots of the characteristic equations are plugged back, one at a time, in the eigenvalue equation written in a matrix form (illustrated for the simplest case of a two-dimensional vector space L2):

\left (\begin{bmatrix} a_{11} & a_{12}\\a_{21} & a_{22}\end{bmatrix} - \lambda \begin{bmatrix} 1 & 0\\0 & 1\end{bmatrix} \right ) \begin{bmatrix} x \\ y \end{bmatrix} = \begin{bmatrix} a_{11} - \lambda & a_{12}\\a_{21} & a_{22} - \lambda \end{bmatrix} \begin{bmatrix} x \\ y \end{bmatrix} = \begin{bmatrix} 0 \\ 0 \end{bmatrix},

where λ is one of the eigenvalues found as a root of the characteristic equation. This matrix equation is equivalent to a system of two linear equations:

\begin{cases}
    \left ( a_{11} - \lambda \right ) x + a_{12} y = 0 \\
    a_{21} x + \left ( a_{22} - \lambda \right ) y = 0
\end{cases}

The equations are solved for x and y by the usual algebraic or matrix methods. Often, it is possible to divide both sides of the equations to one or more of the coefficients which makes some of the coefficients in front of the unknowns equal to 1. This is called normalization of the vectors, and corresponds to choosing one of the eigenvectors (the normalized eigenvector) as a representative of all vectors in the eigenspace corresponding to the respective eigenvalue. The x and y thus found are the components of the eigenvector in the coordinate system used (most often Cartesian, or polar).

Using the Cayley-Hamilton theorem which states that every square matrix satisfies its own characteristic equation, it can be shown that (most generally, in the complex space) there exists at least one non-zero vector that satisfies the eigenvalue equation for that matrix.[25] As it was said in the Definitions section, to each eigenvalue correspond an infinite number of colinear (linearly dependent) eigenvectors that form the eigenspace for this eigenvalue. On the other hand, the dimension of the eigenspace is equal to the number of the linearly independent eigenvectors that it contains. The geometric multiplicity of an eigenvalue is defined as the dimension of the associated eigenspace. A multiple eigenvalue may give rise to a single eigenvector so that its algebraic multiplicity may be different than the geometric multiplicity.[26] However, as already stated, different eigenvalues are paired with linearly independent eigenvectors.[14] From the aforementioned, it follows that the geometric multiplicity cannot be greater than the algebraic multiplicity.[27]

For instance, an eigenvector of a rotation in three dimensions is a vector located within the axis about which the rotation is performed. The corresponding eigenvalue is 1 and the corresponding eigenspace contains all the vectors along the axis. As this is a one-dimensional space, its geometric multiplicity is one. This is the only eigenvalue of the spectrum (of this rotation) that is a real number.

Examples

The examples that follow are for the simplest case of two-dimensional vector space L2 but they can easily be applied in the same manner to spaces of higher dimensions.

Homothety, identity, point reflection, and null transformation

File:Homothety in two dim.svg
Fig. 3. When a surface is stretching equally in homothety, any one of the radial vectors can be the eigenvector.

As a one-dimensional vector space L1, consider a rubber string tied to unmoving support in one end, such as that on a child's sling. Pulling the string away from the point of attachment stretches it and elongates it by some scaling factor λ which is a real number. Each vector on the string is stretched equally, with the same scaling factor λ, and although elongated it preserves its original direction. This type of transformation is called homothety (similarity transformation). For a two-dimensional vector space L2, consider a rubber sheet stretched equally in all directions such as a small area of the surface of an inflating balloon (Fig. 3). All vectors originating at a fixed point on the balloon surface are stretched equally with the same scaling factor λ. The homothety transformation in two-dimensions is described by a 2 × 2 square matrix, acting on an arbitrary vector in the plane of the stretching/shrinking surface. After doing the matrix multiplication, one obtains:

A \mathbf{x} = \begin{bmatrix}\lambda & 0\\0 & \lambda\end{bmatrix} \begin{bmatrix} x \\ y \end{bmatrix} = \begin{bmatrix}\lambda . x + 0 . y \\0 . x + \lambda . y\end{bmatrix} = \lambda \begin{bmatrix} x \\ y \end{bmatrix} = \lambda \mathbf{x},

which, expressed in words, means that the transformation is equivalent to multiplying the length of the vector by λ while preserving its original direction. The equation thus obtained is exactly the eigenvalue equation. Since the vector taken was arbitrary, in homothety any vector in the vector space undergoes the eigenvalue equation, i. e. any vector lying on the balloon surface can be an eigenvector. Whether the transformation is stretching (elongation, extension, inflation), or shrinking (compression, deflation) depends on the scaling factor: if λ > 1, it is stretching, if λ < 1, it is shrinking.

Several other transformations can be considered special types of homothety with some fixed, constant value of λ: in identity which leaves vectors unchanged, λ = 1; in reflection about a point which preserves length and direction of vectors but changes their orientation to the opposite one, λ = −1; and in null transformation which transforms each vector to the zero vector, λ = 0. The null transformation does not give rise to an eigenvector since the zero vector cannot be an eigenvector but it has eigenspace since eigenspace contains also the zero vector by definition.

Unequal scaling

For a slightly more complicated example, consider a sheet that is stretched uneqally in two perpendicular directions along the coordinate axes, or, similarly, stretched in one direction, and shrunk in the other direction. In this case, there are two different scaling factors: k1 for the scaling in direction x, and k2 for the scaling in direction y. The transformation matrix is \begin{bmatrix}k_1 & 0\\0 & k_2\end{bmatrix}, and the characteristic equation is λ2 − λ (k1 + k2) + k1k2 = 0. The eigenvalues, obtained as roots of this equation are λ1 = k1, and λ2 = k2 which means, as expected, that the two eigenvalues are the scaling factors in the two directions. Plugging k1 back in the eigenvalue equation gives one of the eigenvectors:

\begin{bmatrix}k_1 - k_1 & 0\\0 & k_2 - k_1\end{bmatrix} \begin{bmatrix} x \\ y\end{bmatrix} = \begin{cases}
    \left ( k_1 - k_1 \right ) x + 0 . y \\
    0 . x + \left ( k_2 - k_1 \right ) y
\end{cases} = \left ( k_2 - k_1 \right ) y = 0.

Dividing the last equation by k2k1, one obtains y = 0 which represents the x axis. A vector with lenght 1 taken along this axis represents the normalized eigenvector corresponding to the eigenvalue λ1. The eigenvector corresponding to λ2 which is a unit vector along the y axis is found in a similar way. In this case, both eigenvalues are simple (with algebraic and geometric multiplicities equal to 1). Depending on the values of λ1 and λ2, there are several notable special cases. In particular, if λ1 > 1, and λ2 = 1, the transformation is a stretch in the direction of axis x. If λ2 = 0, and λ1 = 1, the transformation is a projection of the surface L2 on the axis x because all vectors in the direction of y become zero vectors.

Let the rubber sheet is stretched along the x axis (k1 > 1) and simultaneously shrunk along the y axis (k2 < 1). Then λ1 = k1 will be the principal eigenvalue. Repeatedly applying this transformation of stretching/shrinking many times to the rubber sheet will turn the latter more and more similar to a rubber string. Any vector on the surface of the rubber sheet will be oriented closer and closer to the direction of the x axis (the direction of stretching), that is, it will become collinear with the principal eigenvector.

Mona Lisa

Mona Lisa with eigenvector

For the example shown on the right, the matrix that would produce a shear transformation similar to this would be

A=\begin{bmatrix}1 & 0\\ -\frac{1}{2} & 1\end{bmatrix}.

The set of eigenvectors \mathbf{x} for A is defined as those vectors which, when multiplied by A, result in a simple scaling \lambda of \mathbf{x}. Thus,

A\mathbf{x} = \lambda\mathbf{x}.

If we restrict ourselves to real eigenvalues, the only effect of the matrix on the eigenvectors will be to change their length, and possibly reverse their direction. So multiplying the right hand side by the Identity matrix I, we have

A\mathbf{x} = (\lambda I)\mathbf{x},

and therefore

(A-\lambda I)\mathbf{x}=0.

In order for this equation to have non-trivial solutions, we require the determinant \det(A - \lambda I), which is called the characteristic polynomial of the matrix A, to be zero. In our example we can calculate the determinant as

\det\!\left(\begin{bmatrix}1 & 0\\ -\frac{1}{2} & 1\end{bmatrix} - \lambda\begin{bmatrix}1 & 0\\ 0 & 1\end{bmatrix} \right)=(1-\lambda)^2,

and now we have obtained the characteristic polynomial (1-\lambda)^2 of the matrix A. There is in this case only one distinct solution of the equation (1-\lambda)^2 = 0, \lambda=1. This is the eigenvalue of the matrix A. As in the study of roots of polynomials, it is convenient to say that this eigenvalue has multiplicity 2.

Having found an eigenvalue \lambda=1, we can solve for the space of eigenvectors by finding the nullspace of A-(1)I. In other words by solving for vectors \mathbf{x} which are solutions of

\begin{bmatrix}1-\lambda & 0\\ -\frac{1}{2} & 1-\lambda \end{bmatrix}\begin{bmatrix}x_1\\ x_2\end{bmatrix}=0

Substituting our obtained eigenvalue \lambda=1,

\begin{bmatrix}0 & 0\\ -\frac{1}{2} & 0 \end{bmatrix}\begin{bmatrix}x_1\\ x_2\end{bmatrix}=0

Solving this new matrix equation, we find that vectors in the nullspace have the form

\mathbf{x} = \begin{bmatrix}0\\ c\end{bmatrix}

where c is an arbitrary constant. All vectors of this form, i.e. pointing straight up or down, are eigenvectors of the matrix A. The effect of applying the matrix A to these vectors is equivalent to multiplying them by their corresponding eigenvalue, in this case 1.

In general, 2-by-2 matrices will have two distinct eigenvalues, and thus two distinct eigenvectors. Whereas most vectors will have both their lengths and directions changed by the matrix, eigenvectors will only have their lengths changed, and will not change their direction, except perhaps to flip through the origin in the case when the eigenvalue is a negative number. Also, it is usually the case that the eigenvalue will be something other than 1, and so eigenvectors will be stretched, squashed and/or flipped through the origin by the matrix.

Other examples

As the Earth rotates, every arrow pointing outward from the center of the Earth also rotates, except those arrows which are parallel to the axis of rotation. Consider the transformation of the Earth after one hour of rotation: An arrow from the center of the Earth to the Geographic South Pole would be an eigenvector of this transformation, but an arrow from the center of the Earth to anywhere on the equator would not be an eigenvector. Since the arrow pointing at the pole is not stretched by the rotation of the Earth, its eigenvalue is 1.

Standing wave
Fig. 2. A standing wave in a rope fixed at its boundaries is an example of an eigenvector, or more precisely, an eigenfunction of the transformation giving the acceleration. As time passes, the standing wave is scaled by a sinusoidal oscillation whose frequency is determined by the eigenvalue, but its overall shape is not modified.
PhloxBotAdded by PhloxBot

However, three-dimensional geometric space is not the only vector space. For example, consider a stressed rope fixed at both ends, like the vibrating strings of a string instrument (Fig. 2). The distances of atoms of the vibrating rope from their positions when the rope is at rest can be seen as the components of a vector in a space with as many dimensions as there are atoms in the rope.

Assume the rope is a continuous medium. If one considers the equation for the acceleration at every point of the rope, its eigenvectors, or eigenfunctions, are the standing waves. The standing waves correspond to particular oscillations of the rope such that the acceleration of the rope is simply its shape scaled by a factor—this factor, the eigenvalue, turns out to be -\omega^2 where \omega is the angular frequency of the oscillation. Each component of the vector associated with the rope is multiplied by a time-dependent factor \sin(\omega t). If damping is considered, the amplitude of this oscillation decreases until the rope stops oscillating, corresponding to a complex ω. One can then associate a lifetime with the imaginary part of ω, and relate the concept of an eigenvector to the concept of resonance. Without damping, the fact that the acceleration operator (assuming a uniform density) is Hermitian leads to several important properties, such as that the standing wave patterns are orthogonal functions.

Eigenfunctions

However, it is sometimes unnatural or even impossible to write down the eigenvalue equation in a matrix form. This occurs for instance when the vector space is infinite dimensional, for example, in the case of the rope above. Depending on the nature of the transformation T and the space to which it applies, it can be advantageous to represent the eigenvalue equation as a set of differential equations. If T is a differential operator, the eigenvectors are commonly called eigenfunctions of the differential operator representing T. For example, differentiation itself is a linear transformation since

 \displaystyle\frac{d}{dt}(af+bg) = a \frac{df}{dt} + b \frac{dg}{dt}

(f(t) and g(t) are differentiable functions, and a and b are constants).

Consider differentiation with respect to t. Its eigenfunctions h(t) obey the eigenvalue equation:

\displaystyle\frac{dh}{dt} = \lambda h,

where λ is the eigenvalue associated with the function. Such a function of time is constant if \lambda = 0, grows proportionally to itself if \lambda is positive, and decays proportionally to itself if \lambda is negative. For example, an idealized population of rabbits breeds faster the more rabbits there are, and thus satisfies the equation with a positive lambda.

The solution to the eigenvalue equation is g(t)= \exp (\lambda t), the exponential function; thus that function is an eigenfunction of the differential operator d/dt with the eigenvalue λ. If λ is negative, we call the evolution of g an exponential decay; if it is positive, an exponential growth. The value of λ can be any complex number. The spectrum of d/dt is therefore the whole complex plane. In this example the vector space in which the operator d/dt acts is the space of the differentiable functions of one variable. This space has an infinite dimension (because it is not possible to express every differentiable function as a linear combination of a finite number of basis functions). However, the eigenspace associated with any given eigenvalue λ is one dimensional. It is the set of all functions g(t)= A \exp (\lambda t), where A is an arbitrary constant, the initial population at t=0.

Spectral theorem

For more details on this topic, see spectral theorem.

In its simplest version, the spectral theorem states that, under certain conditions, a linear transformation of a vector \mathbf{v} can be expressed as a linear combination of the eigenvectors, in which the coefficient of each eigenvector is equal to the corresponding eigenvalue times the scalar product (or dot product) of the eigenvector with the vector \mathbf{v}. Mathematically, it can be written as:

\mathcal{T}(\mathbf{v})= \lambda_1 (\mathbf{v}_1 \cdot \mathbf{v}) \mathbf{v}_1 + \lambda_2 (\mathbf{v}_2 \cdot \mathbf{v}) \mathbf{v}_2 + \cdots

where \mathbf{v}_1, \mathbf{v}_2, \dots and \lambda_1, \lambda_2, \dots stand for the eigenvectors and eigenvalues of \mathcal{T}. The theorem is valid for all self-adjoint linear transformations (linear transformations given by real symmetric matrices and Hermitian matrices), and for the more general class of (complex) normal matrices.

If one defines the nth power of a transformation as the result of applying it n times in succession, one can also define polynomials of transformations. A more general version of the theorem is that any polynomial P of \mathcal{T} is given by

P(\mathcal{T})(\mathbf{v}) = P(\lambda_1) (\mathbf{v}_1 \cdot \mathbf{v}) \mathbf{v}_1 + P(\lambda_2) (\mathbf{v}_2 \cdot \mathbf{v}) \mathbf{v}_2 + \cdots

The theorem can be extended to other functions of transformations, such as analytic functions, the most general case being Borel functions.

Eigendecomposition

Main article: Eigendecomposition (matrix)

The spectral theorem for matrices can be stated as follows. Let \mathbf{A} be a square (n\times n) matrix. Let \mathbf{q}_1 ... \mathbf{q}_k be an eigenvector basis, i.e. an indexed set of k linearly independent eigenvectors, where k is the dimension of the space spanned by the eigenvectors of \mathbf{A}. If k=n, then \mathbf{A} can be written

\mathbf{A}=\mathbf{Q}\mathbf{\Lambda}\mathbf{Q}^{-1}

where \mathbf{Q} is the square (n\times n) matrix whose ith column is the basis eigenvector \mathbf{q}_i of \mathbf{A} and \mathbf{\Lambda} is the diagonal matrix whose diagonal elements are the corresponding eigenvalues, i.e. \Lambda_{ii}=\lambda_i.

Infinite-dimensional spaces

If the vector space is an infinite dimensional Banach space, the notion of eigenvalues can be generalized to the concept of spectrum. The spectrum is the set of scalars λ for which \left(T-\lambda\right)^{-1} is not defined; that is, such that T-\lambda has no bounded inverse.

Clearly if λ is an eigenvalue of T, λ is in the spectrum of T. In general, the converse is not true. There are operators on Hilbert or Banach spaces which have no eigenvectors at all. This can be seen in the following example. The bilateral shift on the Hilbert space \ell^2(\mathbf{Z}) (the space of all sequences of scalars \dots a_{-1}, a_0, a_1,a_2,\dots such that \cdots + |a_{-1}|^2 + |a_0|^2 + |a_1|^2 + |a_2|^2 + \cdots converges) has no eigenvalue but has spectral values.

In infinite-dimensional spaces, the spectrum of a bounded operator is always nonempty. This is also true for an unbounded self adjoint operator. Via its spectral measures, the spectrum of any self adjoint operator, bounded or otherwise, can be decomposed into absolutely continuous, pure point, and singular parts. (See Decomposition of spectrum.)

Exponential functions are eigenfunctions of the derivative operator (the derivative of exponential functions are proportional to themself). Exponential growth and decay therefore provide examples of continuous spectra, as does the vibrating string example illustrated above. The hydrogen atom is an example where both types of spectra appear. The eigenfunctions of the hydrogen atom Hamiltonian are called eigenstates and are grouped into two categories. The bound states of the hydrogen atom correspond to the discrete part of the spectrum (they have a discrete set of eigenvalues which can be computed by Rydberg formula) while the ionization processes are described by the continuous part (the energy of the collision/ionization is not quantified).

Applications

Schrödinger equation

HAtomOrbitals
Fig. 4. The wavefunctions associated with the bound states of an electron in a hydrogen atom can be seen as the eigenvectors of the hydrogen atom Hamiltonian as well as of the angular momentum operator. They are associated with eigenvalues interpreted as their energies (increasing downward: n=1,2,3,...) and angular momentum (increasing across: s, p, d,...). The illustration shows the square of the absolute value of the wavefunctions. Brighter areas correspond to higher probability density for a position measurement. The center of each figure is the atomic nucleus, a proton.
PhloxBotAdded by PhloxBot

An example of an eigenvalue equation where the transformation \mathcal{T} is represented in terms of a differential operator is the time-independent Schrödinger equation in quantum mechanics:

H\psi_E = E\psi_E \,

where H, the Hamiltonian, is a second-order differential operator and \psi_E, the wavefunction, is one of its eigenfunctions corresponding to the eigenvalue E, interpreted as its energy.

However, in the case where one is interested only in the bound state solutions of the Schrödinger equation, one looks for \psi_E within the space of square integrable functions. Since this space is a Hilbert space with a well-defined scalar product, one can introduce a basis set in which \psi_E and H can be represented as a one-dimensional array and a matrix respectively. This allows one to represent the Schrödinger equation in a matrix form. (Fig. 4 presents the lowest eigenfunctions of the Hydrogen atom Hamiltonian.)

The Dirac notation is often used in this context. A vector, which represents a state of the system, in the Hilbert space of square integrable functions is represented by |\Psi_E\rangle. In this notation, the Schrödinger equation is:

H|\Psi_E\rangle = E|\Psi_E\rangle

where |\Psi_E\rangle is an eigenstate of H. It is a self adjoint operator, the infinite dimensional analog of Hermitian matrices (see Observable). As in the matrix case, in the equation above H|\Psi_E\rangle is understood to be the vector obtained by application of the transformation H to |\Psi_E\rangle.

Molecular orbitals

In quantum mechanics, and in particular in atomic and molecular physics, within the Hartree-Fock theory, the atomic and molecular orbitals can be defined by the eigenvectors of the Fock operator. The corresponding eigenvalues are interpreted as ionization potentials via Koopmans' theorem. In this case, the term eigenvector is used in a somewhat more general meaning, since the Fock operator is explicitly dependent on the orbitals and their eigenvalues. If one wants to underline this aspect one speaks of implicit eigenvalue equation. Such equations are usually solved by an iteration procedure, called in this case self-consistent field method. In quantum chemistry, one often represents the Hartree-Fock equation in a non-orthogonal basis set. This particular representation is a generalized eigenvalue problem called Roothaan equations.

Geology and Glaciology: (Orientation Tensor)

In geology, especially in the study of glacial till, eigenvectors and eigenvalues are used as a method by which a mass of information of a clast fabric's constituents' orientation and dip can be summarized in a 3-D space by six numbers. In the field, a geologist may collect such data for hundreds or thousands of clasts in a soil sample, which can only be compared graphically such as in a Tri-Plot (Sneed and Folk) diagram [28], [29], or as a Stereonet on a Wulff Net [30]. The output for the orientation tensor is in the three orthogonal (perpendicular) axes of space. Eigenvectors output from programs such as Stereo32 [31] are in the order E1 > E2 > E3, with E1 being the primary orientation of clast orientation/dip, E2 being the secondary and E3 being the tertiary, in terms of strength. The clast orientation is defined as the Eigenvector, on a compass rose of 360°. Dip is measured as the Eigenvalue, the modulus of the tensor: this is valued from 0° (no dip) to 90° (vertical). Various values of E1, E2 and E3 mean different things, as can be seen in the book 'A Practical Guide to the Study of Glacial Sediments' by Benn & Evans, 2004 [32].

Factor analysis

In factor analysis, the eigenvectors of a covariance matrix or correlation matrix correspond to factors, and eigenvalues to the variance explained by these factors. Factor analysis is a statistical technique used in the social sciences and in marketing, product management, operations research, and other applied sciences that deal with large quantities of data. The objective is to explain most of the covariability among a number of observable random variables in terms of a smaller number of unobservable latent variables called factors. The observable random variables are modeled as linear combinations of the factors, plus unique variance terms. Eigenvalues are used in analysis used by Q-methodology software; factors with eigenvalues greater than 1.00 are considered significant, explaining an important amount of the variability in the data, while eigenvalues less than 1.00 are considered too weak, not explaining a significant portion of the data variability.

Eigenfaces
Fig. 5. Eigenfaces as examples of eigenvectors
PhloxBotAdded by PhloxBot

Eigenfaces

In image processing, processed images of faces can be seen as vectors whose components are the brightnesses of each pixel. The dimension of this vector space is the number of pixels. The eigenvectors of the covariance matrix associated to a large set of normalized pictures of faces are called eigenfaces. They are very useful for expressing any face image as a linear combination of some of them. In the facial recognition branch of biometrics, eigenfaces provide a means of applying data compression to faces for identification purposes. Research related to eigen vision systems determining hand gestures has also been made. More on determining sign language letters using eigen systems can be found here: http://www.geigel.com/signlanguage/index.php

Similar to this concept, eigenvoices concept is also developed which represents the general direction of variability in human pronunciations of a particular utterance, such as a word in a language. Based on a linear combination of such eigenvoices, a new voice pronunciation of the word can be constructed. These concepts have been found useful in automatic speech recognition systems, for speaker adaptation.

Tensor of inertia

In mechanics, the eigenvectors of the inertia tensor define the principal axes of a rigid body. The tensor of inertia is a key quantity required in order to determine the rotation of a rigid body around its center of mass.

Stress tensor

In solid mechanics, the stress tensor is symmetric and so can be decomposed into a diagonal tensor with the eigenvalues on the diagonal and eigenvectors as a basis. Because it is diagonal, in this orientation, the stress tensor has no shear components; the components it does have are the principal components.

Eigenvalues of a graph

In spectral graph theory, an eigenvalue of a graph is defined as an eigenvalue of the graph's adjacency matrix A, or (increasingly) of the graph's Laplacian matrix, which is either TA or IT 1/2AT −1/2, where T is a diagonal matrix holding the degree of each vertex, and in T −1/2, 0 is substituted for 0−1/2. The kth principal eigenvector of a graph is defined as either the eigenvector corresponding to the kth largest eigenvalue of A, or the eigenvector corresponding to the kth smallest eigenvalue of the Laplacian. The first principal eigenvector of the graph is also referred to merely as the principal eigenvector.

The principal eigenvector is used to measure the centrality of its vertices. An example is Google's PageRank algorithm. The principal eigenvector of a modified adjacency matrix of the World Wide Web graph gives the page ranks as its components. This vector corresponds to the stationary distribution of the Markov chain represented by the row-normalized adjacency matrix; however, the adjacency matrix must first be modified to ensure a stationary distribution exists. The second principal eigenvector can be used to partition the graph into clusters, via spectral clustering.

See also

Notes

Wikibooks-logo-en
The Book of Mathematical Proofs may have more about this subject.


  1. See Hawkins (1975), §2.
  2. 2.0 2.1 2.2 2.3 See Hawkins (1975), §3.
  3. 3.0 3.1 3.2 See Kline 1972, pp. 807-808
  4. See Kline 1972, p. 673
  5. See Kline 1972, pp. 715-716
  6. See Kline 1972, pp. 706-707
  7. See Kline 1972, p. 1063
  8. See Aldrich (2006).
  9. See Golub & van Loan 1996, §7.3; Meyer 2000, §7.3
  10. See Strang 2006, p. 249
  11. See Sharipov 1996, p. 66
  12. See Bowen & Wang 1980, p. 148
  13. For a proof of this lemma, see Shilov 1969, p. 131, and Lemma for the eigenspace
  14. 14.0 14.1 For a proof of this lemma, see Shilov 1969, p. 130, Hefferon 2001, p. 364, and Lemma for linear independence of eigenvectors
  15. See Shilov 1969, p. 131
  16. For proof, see Sharipov 1996, Theorem 4.4 on p. 68
  17. See Shores 2007, p. 252
  18. 18.0 18.1 For a proof of this theorem, see Weisstein, Eric W. Eigenvector From MathWorld − A Wolfram Web Resource
  19. See Strang 2006, footnote to p. 245
  20. For details and proof, see Meyer 2000, p. 494-495
  21. See Greub 1975, p. 118
  22. See Greub 1975, p. 119
  23. For proof, see Gelfand 1971, p. 115
  24. For proof, see Greub 1975, p. 119
  25. For details and proof, see Kuttler 2007, p. 151
  26. See Shilov 1969, p. 134
  27. See Shilov 1969, p. 135 and Problem 11 to Chapter 5
  28. Graham, D., and Midgley, N., 2000. Earth Surface Processes and Landforms (25) pp 1473-1477
  29. Sneed ED, Folk RL. 1958. Pebbles in the lower Colorado River, Texas, a study of particle morphogenesis. Journal of Geology 66(2): 114–150
  30. GIS-stereoplot: an interactive stereonet plotting module for ArcView 3.0 geographic information system
  31. Stereo32
  32. Benn, D., Evans, D., 2004. A Practical Guide to the study of Glacial Sediments. London: Arnold. pp 103-107

References

  • Korn, Granino A.; Korn, Theresa M. (2000), Mathematical Handbook for Scientists and Engineers: Definitions, Theorems, and Formulas for Reference and Review, 1152 p., Dover Publications, 2 Revised edition, ISBN 0-486-41147-8 .
  • John Aldrich, Eigenvalue, eigenfunction, eigenvector, and related terms. In Jeff Miller (Editor), Earliest Known Uses of Some of the Words of Mathematics, last updated 7 August 2006, accessed 22 August 2006.
  • Strang, Gilbert (1993), Introduction to linear algebra, Wellesley-Cambridge Press, Wellesley, MA, ISBN 0-961-40885-5 .
  • Strang, Gilbert (2006), Linear algebra and its applications, Thomson, Brooks/Cole, Belmont, CA, ISBN 0-030-10567-6 .
  • Bowen, Ray M.; Wang, Chao-Cheng (1980), Linear and multilinear algebra, Plenum Press, New York, NY, ISBN 0-306-37508-7 .
  • Claude Cohen-Tannoudji, Quantum Mechanics, Wiley (1977). ISBN 0-471-16432-1. (Chapter II. The mathematical tools of quantum mechanics.)
  • John B. Fraleigh and Raymond A. Beauregard, Linear Algebra (3rd edition), Addison-Wesley Publishing Company (1995). ISBN 0-201-83999-7 (international edition).
  • Golub, Gene H.; van Loan, Charles F. (1996), Matrix computations (3rd Edition), Johns Hopkins University Press, Baltimore, MD, ISBN 978-0-8018-5414-9 .
  • T. Hawkins, Cauchy and the spectral theory of matrices, Historia Mathematica, vol. 2, pp. 1–29, 1975.
  • Roger A. Horn and Charles R. Johnson, Matrix Analysis, Cambridge University Press, 1985. ISBN 0-521-30586-1 (hardback), ISBN 0-521-38632-2 (paperback).
  • Kline, Morris (1972), Mathematical thought from ancient to modern times, Oxford University Press, ISBN 0-195-01496-0 .
  • Meyer, Carl D. (2000), Matrix analysis and applied linear algebra, Society for Industrial and Applied Mathematics (SIAM), Philadelphia, ISBN 978-0-89871-454-8 .
  • Brown, Maureen, "Illuminating Patterns of Perception: An Overview of Q Methodology" October 2004
  • Gene H. Golub and Henk A. van der Vorst, "Eigenvalue computation in the 20th century," Journal of Computational and Applied Mathematics 123, 35-65 (2000).
  • Max A. Akivis and Vladislav V. Goldberg, Tensor calculus (in Russian), Science Publishers, Moscow, 1969.
  • Gelfand, I. M. (1971), Lecture notes in linear algebra, Russian: Science Publishers, Moscow 
  • Pavel S. Alexandrov, Lecture notes in analytical geometry (in Russian), Science Publishers, Moscow, 1968.
  • Carter, Tamara A., Richard A. Tapia, and Anne Papaconstantinou, Linear Algebra: An Introduction to Linear Algebra for Pre-Calculus Students, Rice University, Online Edition, Retrieved on 2008-02-19.
  • Steven Roman, Advanced linear algebra 3rd Edition, Springer Science + Business Media, LLC, New York, NY, 2008. ISBN 978-0-387-72828-5
  • Shilov, G. E. (1969), Finite-dimensional (linear) vector spaces, Russian: State Technical Publishing House, 3rd Edition, Moscow .
  • Hefferon, Jim (2001), Linear Algebra, Online book, St Michael's College, Colchester, Vermont, USA, http://joshua.smcvt.edu/linearalgebra/ .
  • Kuttler, Kenneth (2007), An introduction to linear algebra, Online e-book in PDF format, Brigham Young University, http://www.math.byu.edu/~klkuttle/Linearalgebra.pdf .
  • James W. Demmel, Applied Numerical Linear Algebra, SIAM, 1997, ISBN 0-89871-389-7.
  • Robert A. Beezer, A First Course In Linear Algebra, Free online book under GNU licence, University of Puget Sound, 2006
  • Lancaster, P. Matrix Theory (in Russian), Science Publishers, Moscow, 1973, 280 p.
  • Paul R. Halmos, Finite-Dimensional Vector Spaces, 8th Edition, Springer-Verlag, New York, 1987, 212 p., ISBN 0387900934
  • Pigolkina, T. S. and Shulman, V. S., Eigenvalue (in Russian), In:Vinogradov, I. M. (Ed.), Mathematical Encyclopedia, Vol. 5, Soviet Encyclopedia, Moscow, 1977.
  • Pigolkina, T. S. and Shulman, V. S., Eigenvector (in Russian), In:Vinogradov, I. M. (Ed.), Mathematical Encyclopedia, Vol. 5, Soviet Encyclopedia, Moscow, 1977.
  • Greub, Werner H. (1975), Linear Algebra (4th Edition), Springer-Verlag, New York, NY, ISBN 0-387-90110-8 .
  • Larson, Ron and Bruce H. Edwards, Elementary Linear Algebra, 5th Edition, Houghton Mifflin Company, 2003, ISBN 0618335676.
  • Curtis, Charles W., Linear Algebra: An Introductory Approach, 347 p., Springer; 4th ed. 1984. Corr. 7th printing edition (August 19, 1999), ISBN 0387909923.
  • Shores, Thomas S. (2007), Applied linear algebra and matrix analysis, Springer Science+Business Media, LLC, ISBN 0-387-33194-8 .
  • Sharipov, Ruslan A. (1996), Course of Linear Algebra and Multidimensional Geometry: the textbook, Online e-book in PDF format, Bashkir State University, Ufa, arXiv:math/0405323v1, ISBN 5-7477-0099-5, http://www.geocities.com/r-sharipov .

External links

Wikibooks-logo-en
Algebra may have more about this subject.


Wikibooks-logo-en
Linear Algebra may have more about this subject.


This page uses Creative Commons Licensed content from Wikipedia (view authors).

Around Wikia's network

Random Wiki