Mingye-Lu/AgenticCrawler

Overview

acrawl is a Rust-based web crawler that takes plain English descriptions of what to extract and returns structured data. It bundles 25 provider integrations and includes a built-in MCP server, all in a single binary.

Best for

Best for
Developers who need a quick, LLM-driven web scraper with multi-provider support and MCP compatibility.

Use cases

• Scrape structured data from websites using natural language queries
• Automate data extraction pipelines with MCP server integration
• Quickly prototype crawls across multiple LLM providers without configuration overhead

How to use

Tools exposed

• go_back
• switch_tab
• fit_markdown
• click_at
• fill_form
• select_option
• press_key
• set_device
• execute_js
• page_map
• read_content
• list_resources
• save_file
• list_network_activity
• inspect_request
• list_page_logs
• inspect_log
• list_websocket_activity
• inspect_websocket
• get_page_performance

Tested with

Claude Code, Cursor, VS Code, Zed

Notes

acrawl is a Rust-based web crawler that takes plain English descriptions of what to extract and returns structured data. It bundles 25 provider integrations and includes a built-in MCP server, all in a single binary.

5 stars on GitHub. Last updated 2026-06-11. Licensed MIT.

Use cases

• Scrape structured data from websites using natural language queries
• Automate data extraction pipelines with MCP server integration
• Quickly prototype crawls across multiple LLM providers without configuration overhead

Pros

• Single Rust binary simplifies deployment and dependency management
• Supports 25 providers out of the box for flexible LLM backend choice
• Built-in MCP server enables easy integration with existing tools

Cons

• Rust binary may not suit teams without Rust toolchain experience
• Natural language parsing can produce inconsistent results on complex pages
• Limited to LLM-driven extraction, not a general purpose web scraper

Indexed from awesome-mcp-servers-punkpeye and enriched against its public facts.