Notes 20110201 CIS 6050 Neural Networks

From SnOwy - Ed's Wiki Notebook

Contents 1 Assignment 2 Models of Artificial Intelligence 2.1 Connectionst Model 2.2 Machine Intelligence 2.3 Artificial Intelligence 3 Components of a Machine Intelligence (MI) 4 ANNs as Memory

Assignment

• Due Friday

Models of Artificial Intelligence

Connectionst Model

• mostly related to biological neural networks

Machine Intelligence

• moving into computer science and engineering
• describes mostly neural network research
• focused on {pattern recognition, prediction, ...}
• less concerned with biological plausibility

Artificial Intelligence

• describe intelligence with symbols
• serial processing
• logical manipulation of information
• good at {algebraic manipulations, games, ...}
• bad at {pattern identification, vision, extrapolation, ...}

Components of a Machine Intelligence (MI)

or Connectionist System

• continuous change (often present)
  • a system can adapt to the environment indefinitely
• decentralized decision making
  • knowledge is distributed throughout the weights
• different from symbol model (AI) where all information for each thing is stored in one place
  • grandmother neuron -- losing the neuron that stores information about your grandmother
• graceful degradation
  • removing connections or neurons affects performance incrementally
• parallel processing
  • each node can do its processing separate from all other nodes
• self-organizing
  • system develops own representation for problems
• learning & retrieval phase
  • weights are adjusted during learning
  • the two phases can overlap
  • learning can continue through the lifetime of the network or stop after a desired behaviour

ANNs as Memory

• most of what ANNs do can be described as a memory
• types of patterns ...
  • spatial -- single input or an input-output pair
    • no context maintained between presentations of patterns
    • activation in nodes are zeroed between inputs
    • all activations are cleared between presentations
  • spatial-temporal-context is maintained from previous inputs
    • sequence of patterns important
    • node activation maintained between presentations
    • temporal processing
• types of memory ...
  • random access memory (RAM)
    • requires an address to retrieve data (probably not useful to ANNs)
  • content addressable memory (CAM)
    • maps data to an address
    • retrieve a pattern from a sufficiently representative partial sample
    • Hopfield Network (example) -- stores data as a stable state
    • autoassociative (input only needed to create activations and train weights)
    • clustering
    • stores A1, A2, A3, ... An
  • auto-associative memory (maps data onto other data)
    • provides output pattern given an input pattern
    • heteroassociative stores pattern pairs
    • classification
    • stores (A1,B1), (A2,B2), (A3,B3), ... (An,Bn)