Notes 20110308 CIS 6050 Neural Networks

From SnOwy - Ed's Wiki Notebook

Continuing on ART Processing

ART Processing

we're really discussing ART1 and Fuzzy-ART in this context

1. weight layer values between F1 and F2 are all set equal to {1.0}
   • recall that F1 is field 1 -- the hidden node layer
   • recall that F2 is field 2 -- the output node layer
   • {1.0} means unallocated
   • F1 contains M nodes, F2 contains N nodes
   • F1 is indexed 1 .. M; F2 is indexed 1 .. N
   • set parameters ...
   • vigilance p ∈ [0.0, 1.0]
     • decides the number of clusters we will create
     • high vigilance creates more clusters
     • determines how similar patterns must be within the same cluster
     • a value of 1.0 creates a new cluster for each different input pattern
   • choice parameter α
     • usually small positive constant (0.0001)
   • learning rate β
     • if β = {1.0} -- then fast learning
     • as β approaches {0.0}, learning rate decreases
     • slower learning used if there is a lot of noise
2. input to F0
   • inputs must be in range [0, 1]
   • if input pattern is a1, a2, a3 .. al then the pattern presented to F0 is ...
   • I = a1,a2 .. al,1-a1,1-a2 .. 1-a3 = I1,I2 .. IM -- M = 2L
   • why encode as complement?
   • eliminates need to normalize input (eliminates some preprocessing)
   • complement is cheap to calculate
3. input is copied from F0 to F1 (literally copied)
   • F1 activity is denoted as x = (x1, x2 .. xM)
4. activation in F2 is calculated (categorize input)
   • activity in F2 node j for input I is given by ...
   • T_j(I) = (|I∧w_j|) / (α + |w_j|)
     • where ...
   • |I ∧ w_j| = Σ_i=1^MI_i∧w_ji
   • |w_j| = Σ_i=1^Mw_ji
   • activation T_j for node j is result of input pattern I and weights leading to node j
   • calculate T_j for all nodes in F2
   • where {∧} indicates Fuzzy AND or intersection (using standard ^(u,v) ::= min(u,v))
   • for |I∧w_j| -- compare weights of inputs and select smaller value
     • all of the smaller values are summed in node j
5. select category in F2 which best represents input pattern
   • this will be the F2 node with largest T_j value
   • if there is more than one with highest value (i.e. of there's a tie between two nodes) ...
     • just pick the first one (we're able to change our minds later)
   • T_J = max{Tj : j = 1 .. N}
     • J is the index of the winning node
   • when T_J is selected, set its activation to {1.0}, set all other activations to {0.0}
6. check for resonance or reset ...
   • thanks to Richard for this block
     • Resonance: when the weights match the input patterns (determined by having F2 activate F1)
     • if it's resonant, than the activated node is correct and we want to update the weights
     • resonance means the F2 category correctly represents the input pattern.
     • this is determined by having the F2 node create activation in F1 (top down).
     • this is compared to the input pattern.
   • if the selected category is not similar to the input pattern, then node J is reset {0.0}
   • F2 node with next largest T_j is selected
   • if highest activation T_j isn't correct, try next highest T_J
   • the correct category (resonance) ...
     • |I∧w_J| / |I| ≥ p
       • |I| sum of all inputs
       • |I ∧ w_J| = for all weights on node J, sum either weight w_ji or input x_i
       • whichever is smaller
   • if resonance does not occur, reset node J until a new input is presented
7. learning (adjust weights)
   • once resonance occurs, weights are set using ...
   • w_j^newβ(I∧w_J^old) + (1-β)w_J^old
   • faster learning -- when β = {1.0}
     • weights are set immediately to smaller of I or w_J
   • slow learning -- β < 1.0
     • more of the previous (old) W_J is maintained
8. return to step 2