Getting data off the web, or getting an agent to act on it, comes down to picking the lightest tool that does the job and moving up only when you hit a wall. This is the map I use, one surface at a time.
You need to get data off a website, or have an agent do something on one for you. There are many tools for that, from a plain fetch to a full stealth browser, and it's rarely obvious which one your situation calls for.
The answer is almost always the lightest one that works. Most public pages give up their data to a plain fetch. A real browser earns its place when the page hides behind JavaScript or a login, and stealth tooling only when a site is actively fighting you off. Start simple, escalate only when the current tool fails.
This series walks that ladder rung by rung: the decision framework, then driving a real browser you're already logged into, letting an agent explore a site and then baking the flow into a script, acting inside your everyday browser while you watch, throwaway browsers for tests and screenshots, and what to do when a site is built to block you.
Read in order, or jump to any part.
The decision framework for the whole series: the escalation ladder, the two questions that pick a browser surface, and a safety rule for unattended jobs.
The real-session pattern: run a dedicated browser with your logins, point automation at it, and reuse that warmed-up session instead of fighting a login every run.
The two ways an agent drives a browser: live and snapshot-driven for figuring a site out, or a saved script for a flow you'll run again. How each works, and how they hand off.
The extension surface: an agent works inside the browser you actually use, with your real logins, in new tabs, while you watch. When that's exactly right, and when it isn't.
A disposable browser with no login and no memory, launched fresh and discarded. Right for rendering a public page, a screenshot, or a test, and wrong the moment a login appears.
Some sites block automation on purpose. The escalation order for getting through, why the first check is the network and not the browser, and the tiers that aren't worth it for a solo builder.
A sanitized version of the skill I use to pick and drive each of these surfaces, free to download.
browser-automationOne front door for every surface in this series. It holds the decision guide, the attended-vs-unattended safety rule, and the connect-and-drive patterns, with the machine-specific parts stripped out so you can wire in your own.