firecrawl-toolkit

Python

Web crawling, scraping, and mapping toolkit.

The easiest way for developers to interact with web content through crawling, scraping, and mapping. Supports custom actions, multiple output formats, batch processing, device emulation, and geolocation.

python scraping crawling

View on GitHub Get Started

[features]

Web Crawling

Customizable depth and path controls

Content Extraction

Markdown, HTML, raw HTML output

Custom Actions

Click, scroll, form fill automation

Device Emulation

Mobile/desktop views, custom headers

[install]

git clone https://github.com/RMNCLDYO/firecrawl-toolkit.git
cd firecrawl-toolkit
pip install -r requirements.txt

[usage]

# CLI - Scrape
python cli.py --scrape --url "https://example.com"

# CLI - Crawl
python cli.py --crawl --url "https://example.com" --limit 10

# CLI - Map
python cli.py --map --url "https://example.com"

[requirements]

Python 3.x
Firecrawl API key

[license]

This project is licensed under the MIT license.

[links]

GitHub Repository