Building Updated Research Agenda by Investigating Papers Indexed on Google Scholar: A Natural Language Processing Approach

Abstract

Under many circumstances, scholars need to identify new research directions by going through many different databases to identify the research gap and identify areas which have not yet been studied thus far. Checking all the electronic databases is tiresome, and one often misses the important pieces. In this paper, we propose to shorten the time required for identifying the research gap by using web scraping and natural language processing. We tested this approach by reviewing three distinct areas: (i) safety awareness, (ii) housing price, (iii) sentiment and artificial intelligence from 1988 to 2019. Tokenisation was used to parse the titles of the publications indexed on Google Scholar. We then ranked the collocations from the highest to the lowest frequency. Thus, we determined the sets of keywords that had not been stated in the title and identified the initial idea as a research void.