An approach to decentralizing search, using stigmergic hyperlinks

Abstract

A stigmergic hyperlink or "stigh" is an object that looks and behaves like a regular HTML hyperlink, but runs at the server side. A system of stighs displays interesting emergent behaviors, of some complexity, but a stigh alone is very simple: it has a life attribute only reinforced when users click it, and methods to provide meta-information about its destination. We reason that stigmergic hyperlinks could support a more decentralized approach to the Web search problem, particularly for addressing the "Deep Web", which we consider all the WWW that is uncharted by search engines. We discuss vertical and horizontal solutions for the "Deep Web" and present a specialized system that makes searchable the publications hiding at the biggest Portuguese digital magazines site. The index that the system builds, feeds the search related methods of a stigmergic hyperlink linking to that destination. Our contributions are, to make the case for a broader "Deep Web" concept, that goes beyond databases hiding behind HTML forms; describe an approach that could decentralize Web search, based on stigmergic hyperlinks and a supporting business model; and to exemplify one specialized system that enables searching the biggest Portuguese digital magazines website. © 2010 Springer-Verlag.