Mapping Excellence in Science and Technology across Europe Life Sciences

Abstract

This report describes the potential of bibliometric and patent studies to identify centers of excellence in research in Europe. For this study, four Life Sciences fields of are taken as case studies (immunology, neuroscience, bioinformatics and genetics and heredity). The contractors describe the methodological and practical problems of the used data and analyses as well as the potential and feasibility of such an exercise. To summarize, the major questions of the present report are whether it is possible: • to handle the large data sets linked to the considered areas in a satisfying way, • to executed these studies in a cost effective way, • to identify relevant institutions in the areas considered, • to present in a way that is adequate to the needs of different user groups. Based on the experiences of the present exercise, various measures to improve the methods of analysis and presentation are suggested. The entire project with the developed tool is available at www.cwts.nl/ec-coe. We succeeded to create a tool that enables various types of users to identify on the level of ‘main organization’ (university, company) centers of excellence. I In the course of this project we encountered some problems that need to be resolved if the identification of centers is to be carried out in a cost-effective way and with reliable results on a larger scale that just one field. There is also a noticeable difference in the quality of the underlying data between fields. In some fields the quality is good (neuroscience), whilst in others it is poor (bioinformatics). Difficulties with the veracity of the address data in all fields must also bring into question the accuracy of the geographical maps. Expert input is crucial for collecting the proper publication and patent data to be used as the basis for the analyses and particularly the delineation of fields. At present it is not possible to collect these data without experts who are able to compile an effective search strategy to retrieve the relevant data. These experts should not only ‘know’ the field but also have knowledge about search strategies and their use. These experts can be supported by a search interface to see the effect of specific search strings. The field delineation on the basis of patents differs from publications, because patent databases contain a well-developed classification system (the International Patent Classification, IPC). This classification creates an additional and powerful facility to collect the proper data. The publication databases essentially lack such an overall generic scheme. The above observations lead to the conclusion that implementation of our approach as described in this report on a larger scale (i.e., applying it to hundreds of ‘fields’) is not feasible, simply because we expect that it is impossible to get experts involved on such a large scale in a reasonable way, without losing control over the results. Moreover, we know that the science landscape is changing, and that particularly new and developing fields will attract interest to identify centers of excellence. But precisely in these developing fields delineation of the field on the basis of expert input is problematic, as discussed above. However, there are good prospects to deal with this delineation issue, but this has to be investigated in more detail. In principle it should be possible to start with a limited set of publications and to enlarge this set on the basis of co-citation relations, similar keyword patterns and other bibliometric characteristics. With respect to the use of address data in publication and patent data, we conclude that they may be used at the level of ‘main organization’ (university, company, research institute) in most member states of the EU and associated states. At that level, cleaning of data by national experts is certainly feasible. It seems however, that problems with cleaning are not the same in every country. Apart from the size of the country, it is well known that the science system in countries like France (particularly, the ‘interwoveness’ of the CNRS) differs considerably from the system in the Netherlands. The complexity of the system in France makes it almost impossible, also for national experts, to clean the data, even on the level of organization. Cleaning of these address data would be easier in a ‘bottom-up approach’. This means that beforehand a limited list of organizations has to be compiled within each country. Then the address data could be cleaned using this basic list of organizations. With respect to linking patent and publication indicators, we have made in this project a huge step forward as we were able to identify inventors as authors in the same field. Hence, we were able to identify the ‘research address’ of inventors and thus to build indicators for institutions having both patent and publication data. This enables us to find ‘bridges’ between scientific and technological performance within an R&D field. In this project, we created a tool for different users to enter the fields chosen for this study. The design of this tool had to be flexible enough to be used by different types of users. Because of the variety of users (from scientific experts to policy makers), we are not yet completely able to determine whether the requirements of all users are satisfied. Still in view of the purposes of this study we are convinced that we indeed have. The tool enables users to determine their own criteria and thresholds to identify research entities of a certain productivity or impact. In particular, the possibility to combine different indicators enhances the utility of the tool for the different user groups considerably. On a large scale we were able to combine patent and publication indicators, which can be considered as a major step forward to explore the multiple aspects of excellence. With respect to the size of research entities, we were within the scope of this project not able to go a step below the level of ‘main organization’, e.g., from university to department. It appeared that the quality of address data in publications on the level of departments and (if available) faculty, was so low, that we do not provide results on department level systematically. Moreover, the cleaning efforts for experts in the different national science systems would be huge. Especially in larger countries like Germany, France and the UK, we could not ask to clean the address data at any lower level than the main organization. It should be noted that the activity and performance of these organizations are only measured within the field. The name of the organization as mentioned in the tables and rankings do not refer to the entire organization but only for the part active in a particular field. Apart from these data problems, we mention the debate on the validity of performance indicators on the level of departments. For some purposes and within particular contexts, the entity to focus on should be even below the departments. In these cases the ‘group’ seems more appropriate. In this study we were not able explore this, but we have ideas as to how to deal with this. We suggest that combination of author names and organization name could be used effectively to define groups. A combination of groups may be used to define a department or even a faculty. Still, as mentioned above, we were able to provide information on research in a specific field in an efficient interactive tool, enabling the user to use his/her own criteria and thresholds to identify research entities at the level of organization, with a particular performance. Moreover, we provide the tool at different levels of aggregation (world, EU, and national level). The geographical interface can be used to localize the identified organizations. This enables a specific user to search for entities together with the information of its geographical position.