Posted by kdawson on Wednesday November 15, 2006 @02:13AM from the five-eyes dept. IdaAshley writes, "Web spiders are software agents that traverse the Internet gathering, filtering, and potentially aggregating information for a user. This article shows you how to build spiders and scrapers for Linux to crawl a Web site and gather information, stock data, in this case. Using common scripting langu
