Automatic intelligence gathering from the web: A case study in container traffic

Abstract

The Internet provides one of the largest public repositories of data available to all with a networked computer. The amount of data available is so large, and so easily available that it opens up a wonderful possibility to extract the data, normalise it for the problem at hand, and package it for output to users who may not have the time or technical capacity to directly make full use of the information sources on the web. This paper describes a project where large amounts of data for the movement of containers were turned into a consolidated database that could assist members of the anti-fraud community investigating customs or other fraud relating to the transport of goods using containers. The main emphasis was placed on how these vast amounts of data could be harnessed systematically and disseminated over the Internet to those interested.