The Python script that monitors a page for changes — and what it costs you six months in

You needed to know when one page changed. So you wrote the loop. Fetch, hash, compare, notify. Twenty minutes, two dependencies, done before lunch.

Six months later that script is either lying to you or silent, and you can't tell which from the outside. Those two states produce the same inbox.

Here's the script, and here's the bill.

The Python script to monitor website changes that everyone writes

This is the honest version. It strips the tags that change on every request, diffs the text, persists state to disk so a restart doesn't lose the baseline, and exits non-zero when the fetch fails so cron actually tells you.

import json, hashlib, difflib, pathlib, sys
import requests
from bs4 import BeautifulSoup

URL = "https://example.com/pricing"
STATE = pathlib.Path("state.json")
UA = {"User-Agent": "Mozilla/5.0 (compatible; my-monitor/1.0)"}

def fetch(url):
    r = requests.get(url, headers=UA, timeout=20)
    r.raise_for_status()
    soup = BeautifulSoup(r.text, "html.parser")
    for tag in soup(["script", "style", "meta", "noscript"]):
        tag.decompose()
    lines = [l.strip() for l in soup.get_text("\n").splitlines() if l.strip()]
    return "\n".join(lines)

def notify(subject, body):
    # swap for SMTP, a Slack incoming webhook, whatever you already run
    print(subject)
    print(body)

def main():
    try:
        text = fetch(URL)
    except requests.RequestException as e:
        notify("monitor: fetch failed", f"{URL}: {e}")
        sys.exit(1)

    digest = hashlib.sha256(text.encode()).hexdigest()
    state = json.loads(STATE.read_text()) if STATE.exists() else {}
    old = state.get(URL)

    if old is None or old["hash"] != digest:
        if old is not None:
            diff = "\n".join(difflib.unified_diff(
                old["text"].splitlines(), text.splitlines(),
                fromfile="before", tofile="after", lineterm="", n=2))
            notify(f"monitor: {URL} changed", diff)
        state[URL] = {"hash": digest, "text": text}
        STATE.write_text(json.dumps(state))

main()

Run it from cron. It works.

Credit where it's due: the GeeksforGeeks version is the canonical answer to this question and it's correct. It keeps the baseline in a variable inside a while True loop, so a restart forgets everything, and its notification is print. That's the shape of every tutorial on this search results page.

And difflib.unified_diff is not a toy. It's the exact same comparison and the exact same evidence format our own exact diff tier produces. If your page is quiet and every change matters, the script above is a legitimate answer. Keep reading anyway, because most pages aren't quiet.

Month one: the hash was never the hard part

A reader on Pi My Life Up put it better than any of us could. He downloaded the same page three times, opened the files in WinMerge, and found that "the HTML is different, the website looks the same though."

CSRF tokens. Cache-busting query strings on assets. A "trusted by" carousel that rotates by index. A rendered timestamp in a footer. Your hash sees all of it and none of it means anything.

The tag-stripping arms race

So you strip more. script, style, meta — that's the standard advice, and the standard advice is careful about what it promises. Pi My Life Up says removing them "should reduce the chances of getting a false positive." A practitioner post from July 2024 reaches the same conclusion independently and uses the same verb: strip more HTML "to reduce false positives."

Reduce. That word is load-bearing in both.

Next you strip the nav. Then the footer. Then you find the container div for the section you care about and pin a selector to it, and now you own a selector that breaks the next time someone renames a class.

The threshold you can't set

The other common fix is fuzzy matching: run difflib.SequenceMatcher(...).ratio() and only alert below 98%. The most recent guide on this query recommends exactly that, and says out loud that a change at 99.5% similarity "is probably just a timestamp."

Probably. You are now shipping a coin flip inside your alerting path.

The threshold is a guess about the size of a change when what you needed was a judgment about its meaning. Two characters can turn a 14-day trial into a 7-day one. That's 99.9% similar and it's the whole reason you built the monitor. This is why the useful unit of configuration is a sentence describing the change you care about, not a percentage.

Month two: the page stops being HTML

Here's the test, and it takes fifteen seconds. Open the page, view the raw source (View Source, not the inspector — the inspector shows you the DOM after JavaScript ran), and search for the text you're watching.

If it isn't there, requests has been diffing an empty shell this whole time. Your monitor has never once been wrong, because it has never once been looking at anything.

Source: Web Almanac by HTTP Archive, 2020–2025 SEO chapters.

So you port to Playwright. Reasonable move. Now you own a browser: a pinned Chromium version in your container image, a few hundred megabytes of RAM per instance, a cookie banner that covers the content you're screenshotting, and the eternal question of when a page has finished settling. networkidle is a heuristic, not a promise.

That's the moment where the choice between exact diff, semantic and browser agent tiers stops being academic. Our browser agent tier launches real Chromium, waits for the page to settle, dismisses cookie banners, scrolls, and attaches a full-page "page as checked" screenshot. We built it because the same three problems show up on every client-rendered page, not because it sounded impressive.

Month three: the 403

Cloudflare doesn't wait for your parser. It fingerprints the TLS handshake and the header ordering, which means a raw HTTP client can be identified before it ever sees content. Headless Playwright doesn't solve it either — it executes JavaScript, "but they also expose automation signals that Cloudflare detects immediately," like navigator.webdriver being true and HeadlessChrome sitting in the user agent string.

So you buy IPs. Residential proxies run roughly $3–$15 per GB, with the pay-as-you-go floor around $1/GB on a $5 minimum. Datacenter IPs start near $0.50 each, and they're cheap for a reason: reported success rates on protected targets sit around 90–99% for residential against 40–60% for datacenter. (Proxy pricing verified against public pricing pages in July 2026 — check their sites for current numbers.)

Then there's the half-life. selenium-stealth is no longer maintained and no longer effective against current Cloudflare; the maintained cloudscraper fork still fails. Challenge scripts get updated, encryption keys rotate, new detection logic ships continuously. Every one of those is a potential breakage in a script you wrote to watch a pricing page.

You did not set out to maintain a bot-detection bypass.

Month four: cron lies, quietly

This is the expensive one, and it's the one nobody writes about.

Say you moved the script to GitHub Actions to stop paying for a box. From GitHub's own documentation, verified July 2026:

• "The shortest interval you can run scheduled workflows is once every 5 minutes."
• Scheduled events "can be delayed during periods of high loads," and "if the load is sufficiently high enough, some queued jobs may be dropped."
• "In a public repository, scheduled workflows are automatically disabled when no repository activity has occurred in 60 days."

Read that last one twice. Your monitor turns itself off after two quiet months. There is no email. There is no red X. There is nothing, which is precisely what a working monitor on an unchanged page also produces.

The field reports match the docs. One community discussion describes a daily workflow firing 8 to 14 hours late with one day dropped entirely, and the delay persisting after changing the cron minute. Another reports drift over four hours and climbing. A third notes that a consistent 20–40 minute delay "isn't unheard of", with the folk remedy being to pick a random minute instead of 0 * * * *.

Now the structural problem. A monitor that stopped running and a page that didn't change produce identical inboxes. Silence is the output in both cases, and you have no way to distinguish them without going and looking — which is the exact task you automated away.

The DIY fix is a heartbeat: a second job that alerts when the first one hasn't checked in. Ours ships as run_completed events on signed webhooks, so a dead monitor is something you get told about rather than something you eventually notice. If you're staying with the script, the honest advice is to add a dead man's switch and, on a public repo, a keepalive commit — there's a Marketplace action whose entire job is committing to your repo so GitHub doesn't suspend your cron, which tells you how common the failure is.

Month five: the email nobody can act on

Your alert lands. Subject: "monitor: example.com/pricing changed." Body: 2,000 characters of unified diff, including forty lines of whitespace churn from a template change.

Somebody has to reopen the page. Somebody always reopens the page. That's the whole cost of the alert, paid by a person, every time it fires.

The bar we hold ourselves to is that you should be able to forward an alert or paste it into a ticket without anyone reopening anything. Concretely, that means one sentence describing what changed, the before/after excerpts (or the unified diff on the exact diff tier), the model's own confidence score surfaced rather than hidden, a full-page screenshot on the agent tier, and a link to the monitor timeline that holds the full history.

The timeline is the source of truth. Everything else is a copy of it.

Month six: the invoice

Assume a loaded rate of $75/hour and one moderately protected, moderately dynamic page. Substitute your own rate — the point is the shape, not my number.

Pro at $25/mo is $150 over the same six months. That's 25 monitors, checks every 15 minutes, 5 team members, all three tiers, checks never metered.

$150 buys two hours at that rate. The script spends two hours before it sends its first alert you'd forward to anyone.

When the script is still the right call

Say it plainly, because it's often true.

One static page. A prompt that amounts to "tell me about anything." A diff you're genuinely happy to read yourself. A box you already pay for and already monitor. Keep the script. You're doing byte-for-byte comparison on a quiet page, which is the correct tier for that job, and you don't need us to run difflib for you.

If you'd rather self-host something more complete than 40 lines, changedetection.io is the right answer for a lot of people — Docker-deployable, free if you run it yourself, hosted from around $8.99/month as of July 2026 (check their site for current numbers). We keep a comparison of the main change-monitoring tools that names where each of them beats us.

Keep the Python, drop the pager

You don't have to choose between your code and a hosted checker. Our REST API turns the script into a caller instead of a crawler.

Create an API key under Settings → API (owners and admins, scoped to one team, shown once, stored hashed). You get 120 requests/min, with a 429 and a Retry-After if you push past it. Then:

• POST /monitors — one URL, one prompt, one tier.
• POST /monitors/:id/run — force a check now, returns 202.
• GET /monitors/:id/events?since=<ISO> — pull change events with summary, evidence.before, evidence.after, confidence and seen_at.

tier is a field: diff | ai | agent. The page you've been diffing for six months turned into a React app? That's a dropdown, not a migration, and not a rewrite of your fetch layer.

If you'd rather be pushed to than poll, point a signed webhook at your service. Every delivery carries X-Modsignal-Signature: t=<unix>,v1=HMAC-SHA256 computed over t.body, so you can verify it's us in about six lines. Same evidence as the email. Plus run_completed heartbeats, so your side knows the checks are still happening.

The twenty lines worth keeping

Fetch, hash, diff. That part was never expensive. It's about forty lines and it works on day one, which is exactly why every tutorial ends there.

Scheduling it honestly. Keeping it un-blocked. Quieting it down without deafening it. Making its output something a colleague can act on without opening a browser. That's the part that bills you, month after month, in hours you meant to spend elsewhere.

The free plan covers three monitors on daily checks with two team members, exact diff and semantic tiers, no credit card. Point one at the page your script has been watching and compare what lands in your inbox for a week. If the script wins, you've lost nothing and you still have the script.