Skip to content

feat: debugging in lerd, from the browser through the request to a debug bar on the page - #2113

Closed
ExeQue wants to merge 42 commits into
lerd-env:mainfrom
ExeQue:feat/debugging
Closed

ExeQue wants to merge 42 commits into
lerd-env:mainfrom
ExeQue:feat/debugging

Conversation

@ExeQue

@ExeQue ExeQue commented Oct 5, 2026 •

Copy link
Copy Markdown
Contributor

This is the whole of the debugging work, built in one branch so the parts could shape each other. It will land as a series of smaller pull requests, each covering one part of #2088, and this one stays as the place they are carved from.

A script lerd injects through nginx captures the browser's errors, console messages, failed requests and page views, and the package definitions of libraries such as Inertia, Turbo, htmx and Vite add the DOM events they report failures through. One request id then ties a page view to the PHP request that served it, the requests it sends and the jobs they queue, across sites too, and a Requests lens shows each of them with a timeline from the browser to the framework's own phases, queries, cache and HTTP calls, stack traces with the code beside them, GraphQL operations with their input, response and schema, and JSON payloads, masked by rules a project sets in .lerd.yaml. An assistant reads the same request over MCP with a request tool, a summary first and then any tab a page at a time.

A debug bar, switched on per site, carries the same view onto the page. It sums up the request behind the page and the requests the page sends, and opens the dashboard's request view in place without leaving the site. From the bar a developer pins notes to elements of the page, and an assistant picks them up over MCP with the request behind the page, fixes them and resolves them, which clears the pin.

The store side lives in lerd-env/frameworks#97 for the framework phases and seams, and lerd-env/frameworks#96 for the browser capture events. lerd/debug, the package apps use to add entries of their own, is not on Packagist yet.

Refs #2088

ExeQue added 30 commits October 4, 2026 16:37
A new browser capture, switched on its own and off by default, injects a script into the pages of PHP-FPM, host-proxy and custom-container sites through nginx and reports uncaught errors, unhandled rejections and page views to lerd-ui, along with console messages, failed requests, resources that failed to load and configured DOM events where the site asks for them. nginx names the site in headers it sets itself and lerd-ui only accepts reports nginx handed over.

Each site's settings live under browser_capture in .lerd.yaml, or in the site registry when the project has none, and the script is served with them baked in so a change needs no nginx reload. Presets from the framework store add a frontend library's own events, detected from the project's composer and npm packages.

The dashboard gets a Browser lens grouped per page view with a settings modal, a toggle in the Sites header and System Health, and a Debug tab on sites without PHP. MCP gains browser_events, browser_toggle and browser_presets, and the CLI gains lerd browser-capture.
The dashboard opens a file at its line in an editor picked from a curated list, VS Code, Cursor, VSCodium, Windsurf, Sublime Text, Zed and the JetBrains IDEs, with the ones installed on this machine listed first. The choice is made globally on the System page and per site in the site's controls, where it is kept in lerd's own registry since it is personal, and an editor not on the list is set as a command or URL template. An editor whose binary is not on PATH is reached through its URL scheme, which the browser hands to the desktop.

Once an editor is chosen, the site header gets a button that opens the site's folder, or the selected worktree's, as a project in it. The site's picker names the global editor while it follows it.
…store seams

The devtools collector now understands more kinds of store-declared seams than queued jobs. A span times a framework phase under the label the definition gives it, so bootstrap, routing, middleware, the controller and each view land on the request's timeline for whichever framework declares them. Other kinds read the matched route's name, the session as it is saved, who the request runs as, each Redis command and storage operation, and a UI component's lifecycle, and a lerd kind carries what an app writes through the lerd/debug package. A name expression may walk a dotted property and offer alternatives, and a method callable is named Class@method.

Every major of a framework contributes its seams rather than only one, the extension holds up to 256 of them, and a store seam on a method the extension already captures still times it. Console commands that only loop are listed under exclude_commands, by the name typed, which may run over its first words, or by class, and their own events are left out while the jobs they run keep reporting. The request event carries the time nginx and the FPM queue took, and Laravel jobs each get an id of their own.

Binary query bindings are sent as a placeholder naming their size, since JSON cannot carry them, and session keys that read as credentials are masked.
…rved them

nginx stamps the request id on the browser capture script it injects and passes on how long it spent before PHP was reached, so the first page view takes the id of the request that served the page. Requests the page sends afterwards are tied to theirs from the response, and each carries the browser's own timing phases, DNS, connect, wait, download and the DOM events, which nginx allows across origins with Timing-Allow-Origin. Linked requests get a budget of their own, so a chatty page cannot crowd out its errors.
The Debug window and each site's Debug tab open with a Requests lens that lists every request, job and CLI run lerd saw, newest first, with the calls a page sent later indented under it and what went wrong on each row. Opening one shows it in a dialog filling the window. Performance leads with the headline numbers above a timeline that puts the browser's phases, nginx, the FPM queue, the framework's phases, queries, components, views, logs and the app's own events on one chronological list, filterable by layer. Request holds the route, controller, middleware, headers, query string, body, cookies and session with credentials masked, and every other kind of work gets its own tab, including the tabs an app writes through lerd/debug with its tables, figures, code and charts.

The same picture is available to assistants through an MCP requests action, and the request a dump, log or query belongs to can be followed from the other lenses.
…d the lerd/debug package

The Debug window page now covers the Requests lens with screenshots of the list, a request's timeline and its Request tab, how a page and the requests it sends are linked to the PHP request that served them, which phases every supported framework times, and the editor the Debug window opens files in. A new page describes the lerd/debug package, its timeline, logging and tab API and the Laravel, Symfony and Yii integrations, and the framework definition reference gains the new capture kinds, the auth name rule and multi-word command exclusion.
The debug package page now starts with what lerd records without the package, on every PHP site and per framework, from Livewire components on Laravel to the controller filters on Yii, before it shows what the package adds and how to set it up on each framework.
The Debug window page now leads with the Requests lens and how requests are linked, sums up what each lens captures and where it works in one table, keeps each lens to a few short points, and moves how the capture works to the end.
…ll button

A framework or package definition can now list composer packages under suggest_packages, with a reason and a docs link. A site whose project does not have one installed is offered it above the Requests lens, with a button that runs composer require for it in the run modal and one that stops offering it for that site. Only a package the site is being offered can be installed, and a name composer would not accept is never offered. A docs link to lerd.sh opens in the built-in docs.
macOS finds an installed editor by its app bundle rather than a desktop entry claiming its URL scheme, so the test for that Linux mechanism has nothing to check there.
macOS knows an installed editor by its app bundle where Linux uses a desktop entry claiming its URL scheme, so the test now sets up whichever the platform uses instead of skipping on macOS.
Condense now folds the rows of one kind that follow each other into a single row spanning them, queries, components, views, cache calls, Redis, storage, logs, dumps and the browser's phases, whatever the gap between them. The framework phases, errors and the categories an app writes itself always keep a row each, and the folded row's popover lists each item with its duration.
Clicking a folded row on the request timeline now opens it in place into its rows, each one clickable for its own details like any other row, and clicking the folded row again closes it.
Each outgoing request a page or job makes now carries both ends' headers with credentials and cookies masked, the status and its reason, the body sizes and curl's phase times, DNS, connect, TLS, sent and first byte. Laravel's HTTP client reports them through its events. Symfony's HttpClient takes the request headers as the call starts and reads the response through a new http_response seam as it is destroyed, so a lazy response is complete and the client's own status check still runs where the app expects it; a request whose response never ends is still reported when the process does.

The HTTP tab opens each call into its phases, status, sizes and headers, with a faint line under the row showing when in the request it ran. On the timeline outgoing HTTP is a layer of its own in purple, a failed call keeps the colour with a warning so it folds in with its siblings, a folded row names its layer, and rows that ran at once stack in lanes instead of hiding behind one another. The request detail's tab row snaps to a whole tab.
The project's Debug window settings move from debug to devtools in .lerd.yaml, the key a framework definition uses for its capture. Under devtools.redact a project chooses how masked values look, redacted or masked with a number of visible characters, cropped or keeping the length, and the mask character, and names more headers to mask on outgoing requests per direction, request, response or both, each with its own settings and globs allowed. The values lerd always masks stay masked whatever the project says, in the style it picked.
…window

devtools.redact.inbound_http masks headers on the site's own requests and responses per direction, and input names across the query string, body, cookies, session and route parameters, each with the same settings as outgoing headers. Query values and route parameters are masked in the request's URL as well, so a secret in a path or a query string never shows in the list or a request's header; on Laravel the route parameters are read before binding, by their name in the route.
The Request tab's middleware list links each middleware the app wrote, a class outside vendor or a closure, to its source, drawn as the surrounding text with a dotted underline in the accent colour and without a line number, which would read as a middleware parameter. The framework's and its packages' middleware stay plain names.
A route_params seam reads the matched route's raw parameters as its method returns, so a framework definition can hand them over for masking in the request's path the way the Laravel adapter does. Seam name expressions now chain steps, a method to call, a property to read and bare keys reading on, so a value behind a method and an array key, a request's attributes say, can be reached; every expression written before reads the same.
devtools.redact.inbound_http.route_params names route parameters to mask in the request's path by their name in the route, each with the same settings as any other rule. A route parameter without a rule of its own still follows an input rule of the same name and the values lerd always masks.
The Request tab lists the matched route's parameters, each raw value as the request asked for it and masked the way the path is, beside the model it resolved to with its class and key, where a key that is the masked value itself is masked too. The app's own models open in the editor there and on the Models tab, by the same rule as its middleware.

Server-Timing now carries every timed event that ended before the headers went out, each query, cache call, Redis command, storage operation, outgoing request, component phase and row of the app's own as an entry of its own after the queue and each phase's total, kept within what nginx buffers for headers. The Request tab leaves it out of the response headers, since the timeline shows it in full.
A Laravel route like /users/{user:id} binds its parameter by a column other than the key it is named after, and the Request tab only showed the parameter name. The adapter now reports the binding field next to the model, and the route parameters section shows it as user:id.
A route parameter's raw value stays masked by its redaction rule, but the primary key of the model it resolved to is no longer masked along with it, so the Request tab always says which record the route bound.
The Database, Cache, Redis and Filesystem tabs and the timeline's row details name the line that triggered each event, and every event already carried the stack it was called through. A stack button beside that line now opens the trace in a popover of its own, starting at the app's first line when there is one. The Cache tab gains the caller column it was missing.

Frames that only report the event are left out: lerd's Laravel adapter, the framework calling into the adapter's listeners, and the lerd/debug package. The adapter now builds its trace through the collector so both follow the same rules. The copy and stack buttons beside a path are always shown.
The stack button opens the trace as a path from the line that triggered the event back to where the request began. The app's own lines are stops named by the function they sit in, runs of vendor frames fold into one that opens on a click, and the code around the picked line shows beside the path, coloured by the dashboard's Monaco. A Code switch hides it and is remembered, the arrow keys step through the stops, line numbers open the editor, and Copy writes the trace the way PHP prints one.

lerd-ui reads those lines through a new route that only reads a file inside a linked site or one of its worktrees, with symlinks resolved. The collector names each frame's Composer package from the install paths Composer recorded, so a package installed outside vendor/ is tagged too.

Links to the editor now read Open in and the editor the file's site opens in, and a popover opened over another one no longer closes the one below it.
The Components tab names each component as a link to its file. A class component already carried its file. A single-file component is compiled into a cached class, so a new source seam kind records the files a method loads while it handles a source path, and a class defined in one of them is shown at that source instead. An older lerd ignores the kind.
With a stack trace on every query, cache call and log line an event averages about 4 KB, so the 3000 events lerd kept held fewer than fifty requests and a busy page pushed the one being looked at out within seconds. The ring now keeps 20,000 events, a few hundred requests in about 80 MB, while a dashboard tab still opens on the newest 3000 and reads the rest through the Requests lens.
Stack traces were most of what the buffer held, and they repeat the same framework frames for every query, cache call and log line: a few hundred distinct frames across tens of thousands. The buffer now keeps each frame once and an event only the numbers of its frames, building the trace again for the request being shown, so an event takes about 500 bytes instead of 4 KB. It keeps 60,000 events, around a thousand requests, in about 30 MB, and listing requests no longer touches the traces at all.
Queries in the Queries lens and the timeline are coloured by a small highlighter for SQL, PHP and GraphQL that needs no editor bundle, and the timeline shows a query laid out with each clause on its own line, its columns one per line and subqueries indented.

A class reads by its own name wherever the Debug window shows one, a controller, a job, a model, a caller or a link to the editor, with the full name in its tooltip.
The collector reads a JSON request body, which PHP leaves unparsed, and masks credentials at any depth of it, and keeps the JSON a request with a JSON body answered, through an output buffer that passes every write straight on. Both show on the Request tab.

A GraphQL request is recognised by its body, whatever server answers it. A new GraphQL tab right after Request shows each operation with the queries and mutations it called, the input they were given with variables filled in, and what came back, with the full query coloured below. A request is named by its operation in the list and the header, with GraphQL's mark beside it, a batch by its first operation and how many more.

A new graphql_schema seam kind hands the collector the server's schema, so each field is named by the type it returns, links to the app class that defines it and its type, and the part of the schema the request used can be browsed eight levels down, a type the schema circles back to marked rather than opened again. The Database tab can lay its queries out formatted too.
A site can carry a bar on its pages that sums up the request behind the page and the requests the page sends afterwards, and opens the dashboard's own request view in place. It is switched on per site with lerd debugbar, a toggle on the site's Requests lens or devtools.debugbar in .lerd.yaml, and nginx injects it next to browser capture through the same route, with lerd-ui answering only for the site that asked.

The bar lives in a shadow root on a layer of its own, so the page's styles cannot reach it and nothing the page stacks high covers it, and it follows the dashboard's theme, imported and desktop themes included. It comes as a floating dock or a compact strip along an edge, set under System, minimises to lerd's mark in a corner that can be dragged to another, shows when its request is still loading, and on a site lerd proxies follows the page view browser capture names. From this machine its paths open the editor and its stack traces show the code, which lerd only allows while the site can only be reached locally.

lerd now serves its own fonts, the dashboard included, and the stack trace view colours code with the same highlighter as SQL instead of an editor bundle.
ExeQue added 12 commits October 5, 2026 15:31
A page that runs thousands of queries or renders thousands of views now opens as quickly as a small one, and nothing it recorded is dropped. The collector keeps one connection to lerd open and sends each distinct stack trace once per request, lerd keeps it once and hands it to the request view as a table, and the debug bar polls only the counts its chips show, loading the full request when a chip opens it. Long lists in the request view and opened timeline groups show 200 rows at a time with a button for the next, and views get a timeline layer of their own.

Queries and views are numbered in the order they ran, and an N+1 finding can narrow the query list to the queries it repeats.

Values a view, job or event was given are now masked like the request body, including the fields of a JSON string, so an error page that renders the body no longer shows a password. A secret written straight into a GraphQL query is masked in the query text and the field's arguments when its name is one of the always masked ones.

The debug bar's request chip centres its labels whatever fonts the system uses, and the docs describe everything the request view captures and opens in the editor, with new screenshots.
A password passed straight into a GraphQL query, rather than as a variable, still showed in the request's body. The body's query text is now masked the same way the operations are, before it is shown anywhere, and the docs tell an assistant where a request's GraphQL, N+1 queries and stack traces are in what it reads by request id.
An assistant holding a request's id from the X-Lerd-Rid header now reads it through a request tool of its own instead of one call that returned everything at once, which ran to hundreds of kilobytes on an ordinary page. The summary gives the timing, the phases, the N+1 and slow findings with runnable examples, exceptions, GraphQL operations and the requests it sent; tabs lists the tabs the request has, the app's own included; tab reads one of them a page at a time, the Request tab by section; and trace reads the stack trace a row points at. The request action moves out of diag into the new tool.

A query the framework runs on its own, with no app line on the stack, is now placed at the framework class that asked for it, such as the session handler or the cache store, rather than at the event dispatcher that carried it, so its N+1 finding names something worth looking at.
A nested value in a request body now shows as a tree in the Request tab, masked inside, instead of as a line of JSON. A component phase names each argument by the method's own parameter, so a mount reads as its name, key and params rather than values run together. Each rendered view on the Views tab takes the time of its own render instead of the first render of a view with the same name.
# Conflicts:
#	internal/editor/editor.go
#	internal/nginx/manager.go
…to resolve

The debug bar gets a notes chip that lists the open notes on the page and starts a picker. The element under the pointer is outlined and a click picks it without the page seeing the click, then a box beside it takes what is wrong with it. lerd keeps the note with a selector checked to find only that element, its tag and text, where it sat, the page's address and the request that served the page, and a numbered pin marks the element whenever the page is open. A pin or a row in the list brings the element into view and opens the note to edit, resolve or delete.

An assistant reads the notes through a new annotation tool over MCP: list gives a site's open notes, get one of them with the request behind its page for the request tool to read, and resolve marks it done with what was changed, which takes the pin off the page. Notes are kept per site under lerd's data folder and go through lerd-ui, so one process writes them.

The bar's local check now also works on sites served over plain http, where the browser sends no Sec-Fetch headers, by comparing the page's origin with the site nginx names. That also fixes reading code and opening the editor from the bar on such sites.

The docs describe annotations, and their screenshots are taken again in lerd's default theme.
…ship them

Presets now come from the store's package layer instead of a directory of their own. A package definition can say it is an npm package and declare the DOM events its library fires in a browser block, and the packages that share a preset name are offered as one, detected by any of them. Every npm package's preset is offered, since there are few and a site may add one it does not use yet, and a composer package's where the project has it installed, so a listing never fetches every package file.

npm packages are read from the index's npm_packages, apart from packages, which every released lerd fetches whole on update and would refuse an npm name from. An npm package's file is cached under an npm- prefix so it can never share a name with a composer package's, and a package whose file declares a type lerd does not know is reported by name and never read as either.
The dashboard and the debug bar no longer bundle the Inter font. Both name Inter in their font stack as before and fall back to the system's own sans-serif, so lerd ships no third-party font files.
…s pills

A query run from a closure written in a class, the cache store's for one, is named by that class like the rest instead of showing its whole vendor path, while a closure in a plain file, a route's, still is not given the class it ran bound to. An N+1 finding is named the same way as its queries, with their stack trace a click away. The debug bar's N+1 and failure pills sit on the same line as the text beside them.
An assistant can now switch the debug bar on or off for a site with diag's debugbar_toggle, the way it already does browser capture, and asks whether it is on by leaving enable out. The MCP docs and the reference an assistant reads count the fourteen grouped tools, request and annotation among them.
The mise lookup falls back to fixed system folders, so its test for no mise at all found the one a machine has installed there; the folders are now a seam the test points at an empty one. The custom FPM mount test set HOME and the config folder but not the data folder, so on a machine with XDG_DATA_HOME set it wrote a sites backup into the real lerd state.
@ExeQue

ExeQue commented Oct 5, 2026

Copy link
Copy Markdown
Contributor Author

Split into smaller pull requests as we talked about, to be merged in order: #2121, #2122, #2123, #2124, #2125, #2126, #2127, #2128, #2129, #2130, #2131, #2132 and #2133. Each one builds on the one before and says which commits are its own, and the last one closes #2088. Closing this one in favour of them.

@ExeQue ExeQue closed this Oct 5, 2026
@ExeQue
ExeQue deleted the feat/debugging branch October 6, 2026 09:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant