Artificial Intelligence and Quantum Computing for Advanced Wireless Networks - Savo G. Glisic

Скачать книгу

concept can be extend and, instead of making a hard class assignment, a fuzzy class assignment can be used by allowing 0 ≤ uχ(x) ≤ 1 and requiring sigma-summation Underscript chi equals 1 Overscript upper I Endscripts u Subscript chi Baseline left-parenthesis normal x right-parenthesis equals 1 for all x. This is another vector quantization algorithm called fuzzy k ‐means. The k‐means algorithm is based on a quadratic objective function, which is known to be strongly affected by outliers. This drawback can be alleviated by taking the l1 norm of the approximation errors and modifying the problem to JK‐medians = double-vertical-bar upper X minus italic upper W upper U double-vertical-bar Subscript 1 Superscript 2 subject to Ut U = I and uij ∈ {0, 1}. A different approach can be used to find data representatives less affected by outliers, which we may call robust vector quantization, upper J Subscript italic upper R upper V upper Q Baseline equals upper E left-brace sigma-summation Underscript chi equals 1 Overscript upper K Endscripts u Subscript chi Baseline left-parenthesis normal x right-parenthesis upper Phi left-parenthesis double-vertical-bar normal x minus normal x overbar Subscript chi Baseline double-vertical-bar squared right-parenthesis right-brace, where Φ(x) is a function less sensitive to outliers than Φ(x) = x, for instance, Φ(x) = xα with α about 0.5.

      Principal component analysis (PCA): Introduced in Section 2.1, is by far one of the most popular algorithms for dimensionality reduction [39–42]. Given a set of observations x, with dimension M (they lie in ℝM), PCA is the standard technique for finding the single best (in the sense of least‐square error) subspace of a given dimension, m. Without loss of generality, we may assume the data is zero‐mean and the subspace to fit is a linear subspace (passing through the origin).

      This algorithm is based on the search for orthogonal directions explaining as much variance of the data as possible. In terms of dimensionality reduction, it can be formulated [43] as the problem of finding the m orthonormal directions wi minimizing the representation error upper J Subscript italic upper P upper C upper A Baseline equals upper E left-brace double-vertical-bar normal x minus sigma-summation Underscript i equals 1 Overscript m Endscripts left pointing angle normal w Subscript i Baseline comma normal x right pointing angle normal w Subscript i Baseline double-vertical-bar squared right-brace. In this objective function, the reduced vectors are the projections χ = (〈w1, x〉, …, 〈wm, x〉)t This can be much more compactly written as χ = Wtx, where W is a M × m matrix whose columns are the orthonormal directions wi {or equivalently Wt W = I). The approximation to the original vectors is given by ModifyingAbove normal x With ampersand c period circ semicolon equals sigma-summation Underscript i equals 1 Overscript m Endscripts left pointing angle normal w Subscript i Baseline comma x〉wi, or equivalently, ModifyingAbove normal x With ampersand c period circ semicolon equals W χ .

upper J Subscript italic upper P upper C upper A Baseline equals left-brace double-vertical-bar normal x minus upper W bold-italic chi double-vertical-bar squared right-brace equals upper E left-brace double-vertical-bar normal x minus upper W upper W Superscript t Baseline normal x double-vertical-bar squared right-brace proportional-to double-vertical-bar upper X minus upper W upper W Superscript t Baseline upper X double-vertical-bar Subscript upper F Superscript 2 Baseline period

      Note that the class membership matrix (U in vector quantization) has been substituted in this case by Wt X, which in general can take any positive or negative value. It, thus, has lost its membership meaning and simply constitutes the weights of the linear combination of the column vectors of W that better approximate each input x. Finally, the PCA objective function can also be written as

      JPCA = Tr{Wt ∑X W} [44], where sigma-summation Underscript upper X Endscripts equals StartFraction 1 Over upper N EndFraction sigma-summation Underscript i Endscripts left-parenthesis normal x Subscript i Baseline minus normal x overbar right-parenthesis left-parenthesis normal x Subscript i Baseline minus normal x overbar right-parenthesis Superscript t is the covariance matrix of the observed data. The PCA formulation has also been extended to complex‐valued input vectors [45]; the method is called non‐circular PCA.

      The matrix projection of the input vectors onto a lower‐dimensional space ( χ = Wtx) is a widespread technique in dimensionality reduction. As an illustration, let us look at the following example [46]:

      Design Example 2.5

      Assume that we are analyzing scientific articles related to a specific domain. Each article will be represented by a vector x of word frequencies; that is, we choose a set of M words representative of our scientific area, and we annotate how many times each word appears in each article. Each vector x is then orthogonally projected onto the new subspace defined by the vectors wi. Each vector wi has dimension M, and it can be understood as a “topic” (i.e. a topic is characterized by the relative frequencies of the M different words; two different topics will differ in the relative frequencies of the M words). The projection of x onto each wi gives an idea of how important topic wi is for representing the article. Important topics have large projection values and, therefore, large values in the corresponding component of χ.

      It can be shown [43, 47], as already indicated in Section 2.1, that when the input vectors, x, are zero‐mean (if they are not, we can transform the input data simply by subtracting the sample average vector), then the solution of the minimization of JPCA is given by the m eigenvectors associated to the largest m eigenvalues of the covariance matrix of x {upper C Subscript normal x Baseline equals StartFraction 1 Over upper N EndFraction upper X upper X Superscript t, note that the covariance matrix of x is a M × M matrix with M eigenvalues). If the eigenvalue decomposition of the input covariance matrix is upper C Subscript normal x Baseline equals upper W Subscript <p style= Скачать книгу