A method of implicit regularization based on the phenomena of retrieval-induced forgetting (RIF)

Abstract

Deep learning models have been successfully applied to a variety of real-world problems due to its ability to recognize a complex structure in large datasets through revealing non-trivial relationships among multiple levels of data representations. However, widely used in deep learning gradient-based algorithms may cause numerous difficulties on account of limited memory. While recent studies addressed the problem of the lack of an external memory, and, thus, improved the generalization ability, the proposed solutions introduced a kind of implicit regularization which seems poorly controlled and, as a consequence, decrease the interpretability of learning models. In an attempt to deepen understanding the nature of generalization ability, the present study is aimed at looking at implicit regularization from a psychological perspective. This research puts forward a method of implicit regularization based on the phenomena of retrieval-induced forgetting (RIF). The findings of this study may greatly assist in solving the major problems of proper understanding the deep learning procedure, improving the generalization ability, and the capacity control.