Notes 20110215 CIS 6050 Neural Networks

From SnOwy - Ed's Wiki Notebook

Contents 1 Self Organizing Maps (SOMs) continued 1.1 Feature Map 1.2 Properties of a Feature Map 2 Learning Vector Quantizer (LVQ)

Self Organizing Maps (SOMs) continued

• two phases occur during training
  1. ordering or self-organizing
     • weights are topologically ordered
     • groups of weights are modified simultaneously
  2. convergence -- weights are fine tuned to identify smaller groups
     • neighbourhood consists of a few or one winning node

Feature Map

• a non-linear transformation -- maps continuous input space to discrete outputs
• features to a cluster
• weights attached to a winning node can be viewed as a point in the input space which represents the neuron
• the winning node is called the prototype
• feature vector starts to resemble real input space
• the weights actually start to look like the input vectors

Properties of a Feature Map

hidden layer

1. provides good approximation of input space
   • reduces input space to a smaller number of prototypes
2. topologically ordered
3. input vectors with high probability of occurring ...
   • mapped to larger areas of hidden layer
   • these have better resolution (more nodes) than lower probability patterns
4. given non-linear input vectors, system can select set of features modelling the underlying distribution

Learning Vector Quantizer (LVQ)

another way to use a SOM

• a supervised version of a SOM
• uses class information to improve decision regions
• the SOM approximates this but in an unsupervised manner
  • SOM does not use classes
  • requires input and clusters for input
• LVQ is built on the SOM architecture
• a vector classifier (quantizer) with minimum encoding distortion ...
  • called a Voronoi or nearest neighbour quantizer
  • Voronoi cells around a set of inut points create a partition in input space
  • use nearest neighbours in Euclidean space to define the cell
• LVQ uses class information
  • to move Voronoi vectors to improve classifier regions
• LVQ modifies the SOM clusters
• training algorithm
• Given an input vector X_i and a set of Voronoi vectors w_j, j = (1 ... l)
• the Voronoi vector W_c
  • the weights most similar to the input vector
  • adjusted with ...
    • w_c(n+1) = w_c(n) + α_n[x_i - w_c(n)]
    • where 0 < α_n < 1 is the learning rate -- moves weights closer to x_i
  • ... only if class associated with w_c is the correct class for input x_i.
• means that only classes that belong to the correct category get trained
  • rewarding a LVQ based on correct classification
• we move the weights closer if they are classified correctly
• a supervised operation
• if class for x_i is not the same as the class for w_c
  • w_c(n+1) = w_c(n) - α_n[x_i-w_c(n)]
  • if input vector x_i and classification w_c agree,
    • make it more likely to classify similarly again
  • if they disagree,
    • make it less likely to classify that way again
  • if a node does not activate, then it undergoes little to no training