Notes 20110301 CIS 6050 Neural Networks

From SnOwy - Ed's Wiki Notebook

First class back from the break

• note -- a phase change in a trendline is called an intervention to statisticians
• example of neural network usages

Title: Detection of Disease Outbreaks in Pharmaceutical Sales

... Neural Networks and Threshold Algorithms

Glenn Guthrie, Deborah Stacey, David Calvert

• motivation: Walkerton outbreak -- E. coli 0157 (also Cryptosporidium, North Battleford, Saskatchewan)
• syndromic surveillance
• monitor data to figure out how healthy a population is
• count emergency room visits
• these two diseases have two very different progressions
  • E. coli: very fast
  • Cryptosporidium: slow onset -- sudden phase change
• performance measures -- false positives and false negatives or true positives and true negatives
• prefer false positives over false negatives
  • that is, better not to miss outbreaks
  • maximize of course true positives, true negatives as first priority
• we assume that the number of affected individuals (in an outbreak) is highly correlated with sales of medicines
• the data used was simulated data derived from the real data
  • this is beneficial for any kind of medical study -- the data is now free to distribute without privacy restrictions
• used a sliding window in time to feed into the artificial neural network
• seven days -- today, today-1, today-2 .. today-6
• outbreak detection must be earlier not later
• threshold needed to detect outbreak
• vs industry standard: the moving average (an average calculated over the last n-days)
• this study was continued with telehealth data and with emergency room data
• thought that the emergency room data would be better (doctor looks over a patient)
• telehealth data was very accurate
• over the counter sales were really too variable to do anything useful
• emergency room data had too many confounding variables
• all of these worked with varying degrees of success

Adaptive Resonance Theory (ART)

• started as an attempt to do biological modeling
• how things in the brain work
• was interesting to start but not practical
• ended up being pretty practical
• similar to a kind of clustering called first-in clustering
• contrast: traditional clustering -- all data is evaluated simultaneously, cluster boundaries drawn together
• first-in clustering: the very first thing that is seen becomes the center of a cluster;
• a threshold determines when a new point is just too far and should spawn its own cluster
• -- ART ...
• active regulation of self-organizing learning
• -- actively tries to create new clusters instead of smoothing clusters together
• recognition by attention and expectation
  • system recognizes patterns by looking for previously presented patterns
  • patterns stored in weights
• design problem -- goal
  • how do you design a network that can continue to learn without forgetting old patterns
  • contrast: back propagation network pushes weights away if trained with highly contrasting exemplars
  • how do we retain this plasticity
  • how do intelligent systems remain adaptive in response to significant events
    • and still remain stable in response to irrelevant events
  • compare: back propagation does this with slow weight change
    • a few irrelevant events will be smoothed out
  • ART network must work in real time
    • online training
  • known as the stability-plasticity dilemma
• key computational ideas
  • top-down learned expectations
  • focus attention on bottom-up information such that previously learned patterns are stable
  • new patterns can be added
• inputs (bottum-up) care compared with learned patterns (in weights)
  • either clustered by system or create a new set of weights to store a new pattern