Protocol analysis for exploring the role of application domain in conceptual schema understanding

Abstract

In keeping with prior research [1] that suggests that the objective of empirical research in conceptual modeling is to understand the cognitive model created by viewing conceptual schemas (e.g., ER diagrams), this research contributes to the foundation for developing a cognitive model of conceptual modelers. The aspect on which we focus in our research is the role of the application domain in conceptual schema understanding. Although Information Systems (IS) development can be viewed as application domain problem solving using a software solution, research in the field of IS development has investigated the role of the 75 domain almost to the total exclusion of the application domain. Additionally, most of the empirical research in conceptual modeling has focused primarily on observing the effects of certain stimuli and has, thereby, treated IS problem solving as a "black box." Studies that address how problem solving occurs focus on "opening up the black box" that lies between problem-solving inputs and outputs; that is, they investigate what happens during individual problem solving (isomorphic approach) rather than simply observing the effects of certain stimuli averaged over a number of cases, as in traditional studies (paramorphic approach). The most common approach to opening up the black box is to examine the characteristics of the problem-solving process using protocol analysis. Because both the theory and prior exploratory findings [2] suggest that application domain knowledge is important on just those problem-solving tasks that require transformation of knowledge in the conceptual schema, so-called schema-based problem-solving tasks, we explored how problem solvers address such tasks by examining participants' problem-solving processes using protocol analysis. We found that knowledge of the application and IS domains result in similar search behavior: both familiarity with the application domain and high IS knowledge result in more focused, that is, deeper search, while unfamiliarity with the application domain and low IS knowledge result in broader, and therefore shallower search. © Springer-Verlag Berlin Heidelberg 2006.