Notes 20110310 CIS 6050 Neural Networks

From SnOwy - Ed's Wiki Notebook

ART continued

ARTMAP

• these are clustering algorithms
• how do we turn these into classifying systems?
• this is where ARTMAP comes in
• supervised learning system
• classifier
• there are two variations of ARTMAP
  • binary
  • fuzzy-ARTMAP
• consists of two ART or Fuzzy-ART modules
• one categorizes input patterns and the other, output patterns
• during training, ART_a receives input a^l ...
• while ART_b receives input b^l ...
• where b^l is the correct classification for a^l.
• the two ART networks are linked together using an associative learning network.
  • builds an association between ART_a and ART_b
• this creates a minimal number of ART_a categories
• (minimal ≠ minimum -- minimal means as little as possible without being mathematically proven to be the least)
• uses a minimax learning rule to minimize error while maximizing code compression
  • tries to encode using the smallest number of units and keeps error low
• ARTMAP architecture -- ART_a and ART_b linked by mapfield F^ab

       wjab         copy
       +-----[xab]-----+
       |               |
Fa2 [  ya  ]       [  yb  ] Fb2
       |               |
       |waj            |wbk
       |               |
Fa1 [  xa  ]       [  xb  ] Fb1
       |               |
       |copy           |copy
       |               |
Fa0 [  A   ]       [   B  ] Fb0
 
     ART a          ART b

• modifies the vigilance parameter p_a in ART_a to respond to errors
• if a new category is needed, then p_a is increased to force its creation
• low vigilance values allow larger categories to form
• low p_a allows larger categories to form
  • when this is not appropriate (cluster too large and not specific enough)
• then P_a increased (to reduce error)
• if ART_b does not predict the output for the input to ART_a (predictive failure)
• then p_a is increased -- forces a new category in ART_a
• increasing vigilance to force new categories is called match tracking
• initially, w_j^ab are set equal to 1.0
• when either ART becomes active, the map field also activates
• activation in F^ab.
• F_a activation filtered through w_J^ab and F^ab activation is derived from the weights
• if F_b becomes active then the activation is copied to F^ab
• if both F^a and F^B are active during learning, then F^ab reflects ...
  • the correctness of the association between F^a, F^b
• x^ab = {
  • y^b∧w_J^ab if J^th F₂^a active, F^b₂ active
  • w_J^ab if J^th F₂^a active, F^b₂ inactive
  • y^b if J^th F₂^a inactive, F^b₂ active
  • 0 if both inactive
• training F^ab
  • once activation in F^ab has occurred -- inputs to both ARTs are present --
  • the correctness of the association is compared
• match tracking occurs when:
• |x^ab| < p^a
• where p^ab is the vigilance parameter for F^ab
• match tracking -- fast learning allows network to learn rare events
  • in collection of frequent events that lead to different predictions

Input 2 |                     F2a [ O O O ]
        |         xx                x | |   --compressing everything into one category
        |   x x  xx   x x x           !1|   --exception 1 to the category
        | x   x x! x!   x x             !2  --exception 2 to the category
        |       x  x   x
        | x         x
        +------------------- Input 1

• in the diagram above, there are two events marked by "!" in a large number of similar events "x"
  • but these two dots "!" classified differently
• what this means is -- we want these things to stand out
• we want them to produce different clusters in ART_a
• network can force categories to be created for these unique events
  • identified and categorized differently
  • -- so we're actually allowing the F2a to grow as the training continues
• this is an activity that is not compatible with feed forward back propagation networks and self organizing maps