Assessing broader impacts of funded research: The US National Science Foundation v. Lamar Smith

Abstract

The US National Science Foundation (NSF) must demonstrate the value of the research that it funds. To that end, NSF has developed two criteria for its funding decisions: intellectual merit (IM), the potential to advance knowledge, and broader impacts (BI), the potential to benefit society. Critics have argued that these criteria are insufficient, including former Rep. Lamar Smith (R-TX), who offered his own criteria delineating specific desired impacts, such as economic competitiveness. We empirically assess this criticism by comparing public ratings of NSF-funded projects, based on their Project Outcomes Report (POR), using NSF's and Smith's criteria. Participants rated NSF-funded research as satisfying both NSF's and Smith's criteria, which were moderately correlated. Adding explicit references to societal BI improved ratings slightly. Noting NSF support did not. Our results suggest that having PORs explicitly address additional criteria could increase perceived BI without compromising IM.