Kafka MonitorΒΆ

The Kafka Monitor serves as the entry point into the crawler architecture. It validates API requests so that any point afterwards can ensure the data is in the correct format.

Learn about the design considerations for the Kafka Monitor
Quick Start
How to use and run the Kafka Monitor
The default Kafka API the comes with Scrapy Cluster
Gives an overview of the different plugin components within the Kafka Monitor, and how to make your own.
Explains all of the settings used by the Kafka Monitor