tech
Researchers isolate memorization from problem-solving in AI neural networks
Basic arithmetic ability lives in the memorization pathways, not logic circuits.
TL;DR
- New research from Goodfire.ai indicates AI language models may use separate neural pathways for memorization and reasoning.
- Removing memorization pathways resulted in a 97% loss of verbatim recall but retained nearly all logical reasoning ability.
- Arithmetic operations appear to share neural pathways with memorization, explaining AI's difficulty with math.
- The technique used ranking weight components by 'curvature' to identify and selectively remove memorization circuits.
- This discovery could allow for the removal of copyrighted or private information from AI models without harming their functional abilities.
- Current methods do not guarantee complete elimination of sensitive information and may only suppress it.
- The researchers used K-FAC to analyze loss landscapes and distinguish between memorization and reasoning pathways.