C
crawlee-python
apify/crawlee-python
Crawlee—A web scraping and browser automation library for Python to build reliable crawlers. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Parsel, BeautifulSoup, Playwright, and raw HTTP. Both headful and headless mode. With proxy rotation.
★9.4kstars
Python
Apache-2.0
Updated: Today
📋 Project at a Glance
Tap to expand
What's this?A , built with Python open-source project in the Agents category, core strengths: apify/automation
Who made it?Maintained by apify team, 9.4K⭐ on GitHub, #203 out of 2166 in Agents
Why does it exist?With the growing demand for apify and automation in Agents, apify created crawlee-python as a streamlined solution.
What can it do?Key use cases: beautifulsoup, crawler, crawling
How to install with AI?Use an AI coding assistant (Claude Code, Cursor, Copilot) to automatically set up pip dependencies and virtual env. Follow the README — the AI handles the rest.
🔗 github.com/apify/crawlee-python | 官网 https://crawlee.dev/python/
🔗 github.com/apify/crawlee-python | 官网 https://crawlee.dev/python/
Topics
apifyautomationbeautifulsoupcrawlercrawlingheadlessheadless-chromeparselpipplaywrightpythonscraperscrapingseleniumweb-crawlerweb-crawlingweb-scraping