Notes 20110201 CIS 6050 Neural Networks
From SnOwy - Ed's Wiki Notebook
Contents |
Assignment
- Due Friday
Models of Artificial Intelligence
Connectionst Model
- mostly related to biological neural networks
Machine Intelligence
- moving into computer science and engineering
- describes mostly neural network research
- focused on {pattern recognition, prediction, ...}
- less concerned with biological plausibility
Artificial Intelligence
- describe intelligence with symbols
- serial processing
- logical manipulation of information
- good at {algebraic manipulations, games, ...}
- bad at {pattern identification, vision, extrapolation, ...}
Components of a Machine Intelligence (MI)
or Connectionist System
- continuous change (often present)
- a system can adapt to the environment indefinitely
- decentralized decision making
- knowledge is distributed throughout the weights
- different from symbol model (AI) where all information for each thing is stored in one place
- grandmother neuron -- losing the neuron that stores information about your grandmother
- graceful degradation
- removing connections or neurons affects performance incrementally
- parallel processing
- each node can do its processing separate from all other nodes
- self-organizing
- system develops own representation for problems
- learning & retrieval phase
- weights are adjusted during learning
- the two phases can overlap
- learning can continue through the lifetime of the network or stop after a desired behaviour
ANNs as Memory
- most of what ANNs do can be described as a memory
- types of patterns ...
- spatial -- single input or an input-output pair
- no context maintained between presentations of patterns
- activation in nodes are zeroed between inputs
- all activations are cleared between presentations
- spatial-temporal-context is maintained from previous inputs
- sequence of patterns important
- node activation maintained between presentations
- temporal processing
- spatial -- single input or an input-output pair
- types of memory ...
- random access memory (RAM)
- requires an address to retrieve data (probably not useful to ANNs)
- content addressable memory (CAM)
- maps data to an address
- retrieve a pattern from a sufficiently representative partial sample
- Hopfield Network (example) -- stores data as a stable state
- autoassociative (input only needed to create activations and train weights)
- clustering
- stores A1, A2, A3, ... An
- auto-associative memory (maps data onto other data)
- provides output pattern given an input pattern
- heteroassociative stores pattern pairs
- classification
- stores (A1,B1), (A2,B2), (A3,B3), ... (An,Bn)
- random access memory (RAM)