← 10.711.2 →
951 words Slides

11.1 WebFetch: Fetching Web Content

Course: Claude Code - Power User Section: Accessing the Web Video Length: 2-5 minutes Presenter: Daniel Treasure

Opening Hook

"You're coding and need to check API documentation, read an article, or pull data from a web page—but you don't want to leave Claude Code and manually hunt down the info. WebFetch brings the web to you. No browser launcher needed. Just ask for what you need."

Key Talking Points

1. What WebFetch Does

  • Built-in tool available in Claude Code—no setup required
  • Fetches any URL's content via HTTP request
  • Converts HTML to clean markdown automatically
  • Returns content directly into your conversation context
  • Perfect for documentation, articles, API references, release notes

What to say: "WebFetch is like asking Claude to go grab something from the web and bring it back to you. It's not a browser—it's a direct fetch. Fast, simple, and perfect when you just need the content without the UI."

What to show on screen: Open Claude Code. Show a simple WebFetch request in the chat (don't execute yet, just show the structure).

2. When to Use WebFetch

  • Pulling library/framework docs into your chat
  • Reading API reference pages
  • Fetching changelogs and release notes
  • Grabbing blog posts or technical articles
  • Checking error pages or status updates
  • No need for visual interaction—just pure content

What to say: "If you just need the text content and don't need to interact with the page—click buttons, fill forms, scroll dynamically—WebFetch is your tool."

What to show on screen: Show examples of what kinds of URLs work well: docs.example.com, a GitHub raw markdown file, an API reference page.

3. How WebFetch Works Internally

  • Makes an HTTP GET request to the URL
  • Parses HTML and converts to markdown
  • Handles redirects automatically
  • Returns formatted, readable text
  • Respects robots.txt and other web standards
  • Works on both HTTP and HTTPS

What to say: "Claude handles the heavy lifting. You give it a URL, it fetches, converts, and returns clean markdown you can actually use in your code workflow."

What to show on screen: Diagram or text showing the flow: User request → HTTP fetch → HTML to markdown conversion → Content returned to Claude.

4. Permission Rules and Domain Control

  • By default, WebFetch works on most public domains
  • Can set domain-specific allow/deny policies: WebFetch(domain:example.com)
  • Useful for security: restrict fetching from untrusted sources
  • Can disable WebFetch entirely: WebFetch deny
  • Organizations can enforce policies at the account level

What to say: "You control where Claude can fetch from. Want to lock it down to just your internal docs? Set policies. Want to block a sketchy domain? Done."

What to show on screen: Show the permission configuration syntax in Claude Code settings or documentation.

Demo Plan

  1. Demo 1: Fetch a Public Documentation Page
  2. Open Claude Code
  3. Ask Claude to "Fetch the Python requests library documentation from requests.readthedocs.io and summarize the main use cases"
  4. Show the fetch happening in real-time
  5. Display the markdown output

  6. Show how it's clean and usable

  7. Demo 2: Fetch a GitHub Raw File

  8. Ask Claude to fetch a raw markdown file from GitHub
  9. Show that it parses cleanly

  10. Demonstrate using that content directly in a coding task

  11. Demo 3: Error Handling

  12. Attempt to fetch a URL that redirects or requires authentication
  13. Show how Claude handles it (redirect info, or what fails gracefully)
  14. Explain limitations

Code Examples & Commands

Example 1: Basic WebFetch in Chat

User: "Fetch the docs from https://docs.example.com/api/overview and tell me what endpoints are available"

Claude automatically uses WebFetch tool and returns the content.

Example 2: Using Fetched Content in Code

User: "Fetch https://raw.githubusercontent.com/user/repo/main/README.md and create a Python script that implements the first example"

Claude fetches the README, parses the example code, and writes a working script.

Example 3: Domain Permission in Configuration

[In .claude/config or via Claude Code settings]
WebFetch(domain:internal-docs.company.com) = allow
WebFetch(domain:random-sketchy-site.com) = deny

Gotchas & Tips

  • Large Pages: Some pages are very large. WebFetch still works but may take a moment. Claude will let you know.
  • Dynamic Content: If a page requires JavaScript to load content, WebFetch gets the initial HTML only. For dynamic sites, use Playwright (video 11.3).
  • Authentication: Pages behind logins won't work with WebFetch. You can't pass cookies or credentials.
  • Rate Limiting: Respect robots.txt and don't spam fetches. Some sites may rate-limit or block rapid requests.
  • Markup-Heavy Pages: Some sites have tons of ads, tracking code, and junk. The markdown conversion still works but may be noisy—Claude filters well, but review if it matters.

Pro tip: Use WebFetch for static content (docs, articles, config files). Use Playwright for interactive experiences.

Lead-out

"WebFetch is your shortcut to keeping docs and references in your chat without tab-switching. Next up, we'll look at WebSearch—when you don't know the URL but need to find the answer."

Reference URLs

  • https://docs.anthropic.com/en/docs/build-a-claude-chatbot-with-a-web-crawler
  • https://github.com/anthropics/claude-code

Prep Reading

  • Review WebFetch limitations: dynamic content, authentication, rate limiting
  • Test a few WebFetch calls in Claude Code before recording
  • Prepare 2-3 real documentation URLs to demo with
  • Have a redirect/error scenario ready for the third demo

Notes for Daniel: Keep the energy upbeat—this is a tool that saves time. Show the contrast between manually finding docs vs. WebFetch doing it instantly. The demo should feel effortless. If a demo fetch takes longer than 10 seconds, cut it and show a prerecorded result to keep pacing tight.