搜索源代码（JAVA实现)

Bot Package V1.0 Released with "Programming Spiders, Bots and Aggregators in Java" Bot Package V1.1 Not released externally Bot Package V1.2 * Ant build scripts * Changed all of the "_variable" name's to "variable" name's. * Fixed several issues with the log entry "workload lost" being reported. * Fixed NullPointerException that would sometimes happen when certain types of cookies were encountered * Fixed cookie processing by removing the keyword static from the "cookie store". This was causing cookies to be improperly shared between HTTP classes. * Enhanced the HTTP class with a rootURL property that allows you to determine the first page that was requested(before any redirects are done) * Enhanced HTTP class to allow access to the document as bytes, thus saving the time to convert to a string, and other errors that might occur when translating binary data into a string. * Added new utility function to automatically create the directory structure for a website as the spider downloads it. * Added the ability for the HTTP class to process GZIP'd pages * Enhanced the HTTP class with a "encoding" property. This allows the HTTP class to properly handle multi-byte character sets. * Modified spider to process images using thread pool as well. Bot Package V1.3 * HTTP.getRootURL is now public * Added HTTPException class for HTTP specific exceptions * Added proxy support * Fixed null-pointer exception that would occur if the final HTML tag on the page did not have an ending > Bot Package V1.4 * Removed some un-needed routines * Fixed error that would occur when a POST was redirected, the POST data was being dropped(but introduced in V1.1) * Improved log display in some areas