Methanol is a modular, customizable Web crawling system with crawlers optimized for speed. It is designed to allow the administrator to set up any kind of filetype handling, parsing, and indexing rules.
Information regarding Project Releases and Project Resources. Note that the information here is a quote from Freecode.com page, and the downloads themselves may not be hosted on OSDN.
This releases features new server-side programs as well as lots of improvements to the Web crawler client. There is now full support for HTML entities, conversion between character encodings, and many new features.
