The Kafka Monitor serves as the entry point into the crawler architecture. It validates API requests so that any point afterwards can ensure the data is in the correct format.
- Learn about the design considerations for the Kafka Monitor
- Quick Start
- How to use and run the Kafka Monitor
- The default Kafka API the comes with Scrapy Cluster
- Gives an overview of the different plugin components within the Kafka Monitor, and how to make your own.
- Explains all of the settings used by the Kafka Monitor