These documents give you an overview of the core Scrapy Cluster Crawler and how to work within both Scrapy and Scrapy Cluster to fit your needs.
- Learn about the design considerations for the Scrapy Cluster Crawler
- Quick Start
- How to use and run the distributed crawlers
- Learning how to control your Scrapy Cluster will enable you to get the most out of it
- How to use both Scrapy and Scrapy Cluster to enhance your crawling capabilites
- Explains all of the settings used by the Crawler