RestΒΆ

The Rest service allows for restful interactions with Scrapy Cluster, and utilitizes Kafka to allow you to submit crawl requests or get information about its current state.

Design
Learn about the design considerations for the Rest service
Quick Start
How to use and run the Rest service
API
The API the comes with the endpoint
Settings
Explains all of the settings used by the Rest component