|$mode||bool||If TRUE, the crawler will follow redirects until content was finally found.|
Defaults to TRUE.
Sometimes, when requesting an URL, the first thing the webserver does is sending a redirect to
another location, and sometimes the server of this new location is sending a redirect again
(and so on).
So at least its possible that you find the expected content on a totally different host
If you set this option to TRUE, the crawler will follow all these redirects until it finds some content.
If content finally was found, the root-url of the crawling-process will be set to this url and all
defined options (folllow-mode, filter-rules etc.) will relate to it from now on.