Abstract
Understanding the availability of site metadata on the Web is a foundation for any system or application that wants to work with the pages published by Web sites, and also wants to understand a Web site's structure. There is little information available about how much information Web sites make available about themselves, and this paper presents data addressing this question. Based on this analysis of available Web site metadata, it is easier for Web-oriented applications to be based on statistical analysis rather than assumptions when relying on Web site metadata. Our study of robots.txt files and sitemaps can be used as a starting point for Web-oriented applications wishing to work with Web site metadata. © 2009 Springer Berlin Heidelberg.
Cite
CITATION STYLE
Wilde, E., & Roy, A. (2009). Web site metadata. In Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) (Vol. 5648 LNCS, pp. 300–314). https://doi.org/10.1007/978-3-642-02818-2_25
Register to see more suggestions
Mendeley helps you to discover research relevant for your work.
