This page contains some of the academic work from my road to a masters degree in computer science. I hope others may find it an interesting read, and perhaps even useful for them. Do not hesitate to contact me if you have any questions related to the work that is published here.
Catastrophic Forgetting in Neural Networks
This is a report from the work I and Ole-Marius Moe-Helgesen did on the problem of catastrophic forgetting in neural networks during the autumn of 2005.
Catastrophic forgetting occurs when a neural network cannot recall a pattern it previously has been taught to acceptance. The manifestation of catastrophic forgetting is correlated to the orthogonality of input vectors fed through the network. Simply stated, the less orthogonal the input vectors are, the more likely is the manifestation of catastrophic forgetting in the network. The problem also hinders sequential learning in some neural networks, which makes those networks biologically implausible, and unfit as a model or simulation of human memory.
The report gives a taxonomy of methods to overcome the problem, and an attempt to reproduce the result presented in some of the original papers on methods for overcoming catastrophic forgetting.
Read it here.