Crawling AJAX-based web applications through dynamic analysis of user interface state changes

  • Mesbah A
  • Van Deursen A
  • Lenselink S
  • 5


    Mendeley users who have this article in their library.
  • N/A


    Citations of this article.


Using JAVASCRIPT and dynamic DOM manipulation on the client side of Web applications is becoming a widespread approach for achieving rich interactivity and responsiveness in modern Web applications. At the same time, such techniques-collectively known as AJAX-shatter the concept of webpages with unique URLs, on which traditional Web crawlers are based. This article describes a novel technique for crawling AJAX-based applications through automatic dynamic analysis of user-interface-state changes in Web browsers. Our algorithm scans the DOM tree, spots candidate elements that are capable of changing the state, fires events on those candidate elements, and incrementally infers a state machine that models the various navigational paths and states within an AJAX application. This inferred model can be used in program comprehension and in analysis and testing of dynamic Web states, for instance, or for generating a static version of the application. In this article, we discuss our sequential and concurrent AJAX crawling algorithms. We present our open source tool called CRAWLJAX, which implements the concepts and algorithms discussed in this article. Additionally, we report a number of empirical studies in which we apply our approach to a number of open-source and industrialWeb applications and elaborate on the obtained results. © 2012 ACM 1559-1131/2012/03-ART3 $10.00.

Author-supplied keywords

  • Algorithms; Design; User interfaces; World Wide W
  • Analysis and testing; Automatic dynamic analysis;
  • Java programming language

Get free article suggestions today

Mendeley saves you time finding and organizing research

Sign up here
Already have an account ?Sign in

Find this document


  • A Mesbah

  • A Van Deursen

  • S Lenselink

Cite this document

Choose a citation style from the tabs below

Save time finding and organizing research with Mendeley

Sign up for free