We study the issue of catastrophic forgetting in the context of neural multimodal approaches to Visual Question Answering (VQA). Motivated by evidence from psycholinguistics, we devise a set of linguistically-informed VQA tasks, which differ by the types of questions involved (Wh-questions and polar questions). We test what impact task difficulty has on continual learning, and whether the order in which a child acquires question types facilitates computational models. Our results show that dramatic forgetting is at play and that task difficulty and order matter. Two well-known current continual learning methods mitigate the problem only to a limiting degree.
CITATION STYLE
Greco, C., Plank, B., Fernández, R., & Bernardi, R. (2020). Psycholinguistics meets continual learning: Measuring catastrophic forgetting in visual question answering. In ACL 2019 - 57th Annual Meeting of the Association for Computational Linguistics, Proceedings of the Conference (pp. 3601–3605). Association for Computational Linguistics (ACL). https://doi.org/10.18653/v1/p19-1350
Mendeley helps you to discover research relevant for your work.