You can use ChatGPT's web search and deep research features to scrape basic information from websites. Just ask it to find and extract what you need - like competitor prices or industry news. Perfect for one-off tasks and exploratory research, though you'll have less granular control over exactly what gets scraped. But here's the catch: ChatGPT only works when you're actively using it. If you want scraping to happen automatically while you sleep, or need it to run on a schedule, you'll need dedicated tools... Here's some examples we use Jina.ai - The Zero-Setup Scraper This is the easiest way to start playing around with scraping - no API key needed! Just prepend 'https://r.jina.ai/' to any URL you want to scrape. Jina AI For example:
The Reader API extracts the core content from a URL and converts it into clean, LLM-friendly text, perfect for feeding into AI systems. It's free with 20 requests per minute, or 200 with a free API key. Jina AI It even handles PDFs and can caption images automatically. Others include Tavily - It's built for finding answers across multiple sources quickly and a handy 500 credits per month free plan for exploring. Firecrawl - This is your industrial-strength scraper for when you need to extract data regularly and at scale. Handles JavaScript-heavy sites that other tools struggle with. and our current favourite... Apify - The Swiss Army Knife Apify is like having access to hundreds of pre-built scrapers for specific websites. No coding needed - just pick the scraper you want and run it. It' actually a treasure trove for potential products. Three ways we use Apify right now:
Why Now? (AI Changed Everything) Web scraping used to be a developer-only skill because it was incredibly brittle. You'd write code to extract data from specific parts of a webpage, but the moment that site updated its design, your scraper would break. It was like building a house of cards - one small change and everything collapsed. AI made scraping accessible to everyone because: 🎯 AI understands messy, unstructured text. Feed it raw HTML or markdown from any webpage, and it figures out what's what. A price is a price whether it's in a table, a div, or buried in a paragraph. 🎯 No more breaking when sites update. In the past, if a website moved the price from the top-right to bottom-left, your scraper died. Now AI just reads the whole page and finds prices wherever they are. 🎯 Natural language extraction. Instead of writing code to find specific HTML elements, you just tell AI: "extract all pricing information for competitor analysis" or "find customer complaints about shipping." It understands what you want and delivers it. 🚀 Next steps
PS... If you want to learn more about automations (including scraping) we've partnered with 100 School's Agent Bootcamp. All Create With members get 20% off all bootcamps.
|
|
Learn about AI and NoCode. Curated news, tutorials and interviews every week. Deep dives on how to build with AI and NoCode tools.
Create something today with AI & Visual Development ✨ Hello and happy Tuesday! 👋 As part of our mission to educate more people about AI, we’ve got a deep dive and a standout story on going from a corporate job to profitable SaaS. Let’s jump in… 🔎 Preview: In today's issue 🤔 Why talking to AI just works better 📺 Amsterdam meetup story of corporate job to solo SaaS 🚀 Replit drops 300 models into its builder ✅ Thanks for being an active Create With Member Access your member perks, exclusive...
Create something today with AI & Visual Development ✨ Hello and Happy Tuesday to the regulars and the 102 new subscribers 👋 As part of our mission to educate as many people as possible about AI, we're closing out our series on building AI Agents but looking at the feature "human in the loop"...plus more.Let's jump in... 🔎 Preview: In today's issue 🤔 How Zapier does Human in the Loop (Episode 6 of AI Agents) 📺 Must watch video on Claude Skills. Very Entertaining 🚀 Vibe code with...
Create something today with AI & Visual Development ✨ Welcome back, Creators! 👋 We've got +214 new builders joining our newsletter community this week Pour yourselves a cuppa and let's jump in... 🔎 Preview: In today's issue 🤔 Episode 2 of our 6 part series on AI Agents 📺 Podcast: Kieran and James try to wrap their heads around the future (or lack of) for software companies as we know it 🚀 Huge releases from OpenAI, Softr + more 👩💻 AI Agents: Part II We’re back and it's week 2 of our 6-part...