Find broken links

A page can render correctly while some of its requests fail. You'll learn what the deploy tree actually serves, why development never shows you the difference and how to catch a failed request that leaves no mark on the screen.

cargo install snapfire_compiler
cargo install snapfire_fsr_cli
fsr --version

A page that renders with failed requests#

On one live site, eleven module requests per page load answered 404 for a week. The browser had already loaded those modules under a different URL, so every page rendered correctly and the only sign was errors in the console. Each of those 404s was still a defect.

What the tree holds#

fsr bundle app --out dist

serve/static/js/fsr              serves /static/js/fsr
serve/static/js/app              serves /static/js/app
serve/static/js/vendor           serves /static/js/vendor
app/generated/plan.sexp          read by the host
config                           read by the host

117 files, 4013390 bytes under dist
place beside it: the binary, the logging configuration

The command reads the host's own static roots from the configuration the same way the host does at boot, so a web server answers off disk the same URLs the host answered under fsr dev. Adding a [[static]] entry changes the deployment.

The binary and the logging configuration are left out deliberately, because they differ per deployment. The command says so on the last line.

Development and production#

Under fsr dev the host serves every static root itself, out of the source tree. In production a web server serves serve/ off disk and everything else proxies through. Those are different code paths, so development only exercises one of them.

A file the host would have served from memory in development can be missing from the tree. nginx then answers =404 and the host never sees the request at all. Nothing appears in your logs or your traces and the page still renders, because whatever needed that file had already got it some other way.

Run the production shape locally#

Point a static-first stand-in at serve/ with everything else proxied to the process. This reproduces the deployment on your laptop. Save it as stand_in.py beside the bundle:

import http.server
import mimetypes
import os
import sys
import urllib.error
import urllib.request

SERVE = os.path.abspath(sys.argv[1] if len(sys.argv) > 1 else "dist/serve")
UPSTREAM = "http://127.0.0.1:3000"

mimetypes.add_type("text/javascript", ".mjs")


class NoRedirect(urllib.request.HTTPRedirectHandler):
    def redirect_request(self, *args):
        return None


opener = urllib.request.build_opener(NoRedirect)


class StandIn(http.server.BaseHTTPRequestHandler):
    def do_GET(self):
        self.answer()

    def do_HEAD(self):
        self.answer()

    def do_POST(self):
        self.answer()

    def answer(self):
        path = self.path.split("?", 1)[0]
        if path.startswith("/static"):
            disk = os.path.normpath(os.path.join(SERVE, path.lstrip("/")))
            if not disk.startswith(SERVE) or not os.path.isfile(disk):
                print(f"404 {path}", flush=True)
                return self.send_error(404)
            with open(disk, "rb") as f:
                body = f.read()
            self.reply(200, {"Content-Type": mimetypes.guess_type(disk)[0] or "application/octet-stream"}, body)
            return
        length = int(self.headers.get("Content-Length") or 0)
        data = self.rfile.read(length) if length else None
        headers = {k: v for k, v in self.headers.items() if k.lower() not in ("host", "content-length")}
        request = urllib.request.Request(UPSTREAM + self.path, data=data, headers=headers, method=self.command)
        try:
            response = opener.open(request)
            status = response.status
        except urllib.error.HTTPError as e:
            response = e
            status = e.code
        kept = {k: v for k, v in response.headers.items() if k.lower() not in ("transfer-encoding", "connection", "content-length")}
        self.reply(status, kept, response.read())

    def reply(self, status, headers, body):
        self.send_response(status)
        for k, v in headers.items():
            self.send_header(k, v)
        self.send_header("Content-Length", str(len(body)))
        self.end_headers()
        if self.command != "HEAD":
            self.wfile.write(body)


http.server.ThreadingHTTPServer(("127.0.0.1", 8080), StandIn).serve_forever()

Start the host from the bundle on port 3000, run python3 stand_in.py dist/serve and browse http://127.0.0.1:8080.

The print puts every path the tree does not hold in one log you can read at the end of a sweep, rather than in a console you have to be watching.

Resolve the path to an absolute one before the containment check, otherwise every request 404s. Put .mjs in your MIME table, otherwise the browser refuses modules for a reason that reads like a bundler problem.

Check every route#

Read the console and the network log, since a screenshot looks the same whether requests failed or not.

browserc check http://localhost:11991/some/route --json

check navigates, waits for load, lets async errors settle and reports console errors, uncaught exceptions and failed network requests, plus a screenshot. Run it over every route rather than the one you changed, because the failure above was on every page and looking at one page never showed it.

A route passes when it reports pass: true with an empty errors array. Treat a 4xx in that array as a defect even when the screenshot looks right.

Sign off a deploy#

Run check on every route in the production shape before you call a deploy working. A green build, a page that renders and a passing test suite don't look at requests, so none of them shows a 404 like the ones above.

Next up: 450. Set a content security policy.