Skip to content

fix(drupal): hook the theme engine service and guarantee render hook removal - #4145

Open
Leiyks wants to merge 2 commits into
masterfrom
leiyks/fix-apms-20395
Open

fix(drupal): hook the theme engine service and guarantee render hook removal#4145
Leiyks wants to merge 2 commits into
masterfrom
leiyks/fix-apms-20395

Conversation

@Leiyks

@Leiyks Leiyks commented Aug 27, 2026

Copy link
Copy Markdown
Contributor

Description

Fixes APMS-20395.

On Drupal >= 11.3, drupal.template.file is never set on any drupal.theme.render span, and hooks accumulate on twig_render_template until DD_TRACE_HOOK_LIMIT is reached. A customer measured the tag present on 0 of 61,160 render spans over 2 days across 7 services — including services that never reached the hook limit.

Root cause

DrupalIntegration installs a hook on {$engine}_render_template from inside the prehook of trace_method('Drupal\Core\Theme\ThemeManager', 'render', ['recurse' => true]) — once per render, nested renders included — and the only remove_hook lives inside that hook's own callback. The now-deleted comment stated the invariant it relied on:

The render function will always be called during the ThemeManager::render call

That invariant broke in Drupal 11.3.0. ThemeManager::render() now prefers a theme_engine-tagged service (ThemeManager.php:377-380 @ 11.4.4):

if ($theme_engine_service = $this->getThemeEngine($theme_engine)) {
  $render_function = [$theme_engine_service, 'renderTemplate'];
  $extension = '';
}

while ThemeInitialization.php:157-158 still unconditionally include_onces twig.engine, which still defines the deprecated twig_render_template(). So function_exists() returns true, the hook installs on every render, the function is never called, the hook is never removed, and the tag is never set. function_exists() is useless as a version discriminator here — that is the bug.

Verified boundary (md5 / HTTP probes of upstream tags): ThemeManager.php is byte-identical 11.3.0 ↔ 11.4.4; TwigThemeEngine.php 404s through 11.2.14 and appears at 11.3.0. Deprecated in drupal:11.3.0, removed from drupal:12.0.0.

Drupal twig_render_template() defined called by core engine service
10.x, 11.0–11.2.x yes yes no
11.3.0 – 11.4.5 yes (still included) no yes
12.x (main) no (twig.engine deleted) n/a yes

A second defect, affecting every Drupal version

ThemeManager::render() has an early return FALSE at 11.4.4:178 (theme hook not found), where the render function is never reached and today's hook is never removed. This is not merely a leak — the leaked hooks fire on later renders and retroactively mis-tag unrelated spans. From the new regression test on master:

drupal.template.file[<missing>]           = 11   (expected 30)
drupal.template.file[page.html.twig]      = 49   (expected 30)

19 stale hooks re-tagged 19 unrelated spans. This affects Drupal <= 10.1 too, i.e. versions CI already covers.

Changes

  1. Install one hook at init() on Drupal\Core\Theme\ThemeEngineInterface::renderTemplate. Interface-targeted, so it covers Twig, contrib engines, and the $info['type'] == 'module' case (where core forces Twig) in a single install. Installed once, it cannot accumulate. On Drupal <= 11.2 the interface does not exist and the hook simply never resolves.

    Core passes the template path without its extension on the service path ($extension = ''; TwigThemeEngine::renderTemplate appends self::EXTENSION itself), so .html.twig is re-appended for Twig to keep the tag value byte-identical to what Drupal <= 10 emitted. On Drupal 12 the $extension variable is gone upstream entirely, so this stays correct.

  2. Take the legacy branch only when core itself would — when no engine service resolves — instead of probing function_exists().

  3. Remove the legacy hook unconditionally in the posthook. Self-removal inside the callback is kept, because it is load-bearing for correct outer/inner render pairing, but it can no longer be the only removal path.

Ids are held in a map keyed by spl_object_hash($span) rather than a positional stack: the posthook is skipped for a dropped span (tracer/hook/uhook_legacy.c:226) while the prehook still runs, which would permanently desync a stack. The prehook resets the entry before use, so a recycled object handle cannot resurface a stale id; the map is bounded (measured: 21 entries at both 60 and 400 dropped renders).

themeEngines->has() is used rather than getThemeEngine(), because the latter's ->get() instantiates the engine service — including on the early-return path where core never resolves it — and adds a throw surface to a hot prehook. has() is side-effect-free; core itself guards get() behind it.

…removal

Drupal 11.3 moved template rendering to a theme_engine service
(ThemeEngineInterface::renderTemplate), and ThemeManager::render() only falls
back to the deprecated {engine}_render_template() global when no such service
resolves. Core still includes twig.engine, so function_exists() stays true
while the function is never called: the integration installed a hook per
render that never fired and never self-removed, so drupal.template.file was
missing on every drupal.theme.render span and datadog.trace.hook_limit was
reached within a single request.

Hook ThemeEngineInterface::renderTemplate once at init() to cover every engine
implementation, re-appending .html.twig for TwigThemeEngine so the tag keeps
its pre-11.3 value, and take the legacy per-render branch only when no engine
service resolves. The membership test goes through the themeEngines service
collection rather than getThemeEngine(), which would instantiate the engine
even on core's early-return path.

install_hook's callback is not gated by the span limit while trace_method is,
so past the limit a nested render has no span of its own and active_span()
returns the outer render's. Bail out when the tracer is limited rather than
tag that span with the nested template.

Independently, ThemeManager::render() can return without ever calling the
render function (unknown theme hook, exception) on every Drupal version, so
the callback's self-removal is no longer the only removal path: the posthook
now removes the id the prehook recorded for that span. Keyed by span rather
than a stack because the posthook is skipped for a dropped span.

APMS-20395
@datadog-official

datadog-official Bot commented Aug 27, 2026

Copy link
Copy Markdown

Pipelines  Tests

⚠️ Warnings

Your PR has failed checks. Please review the issues below and take necessary action before merging.

🚦 3 Pipeline jobs failed

DataDog/apm-reliability/dd-trace-php | ASAN test_c with multiple observers: [8.3]

View more details · View in GitLab

DataDog/apm-reliability/dd-trace-php | ASAN test_c: [8.0, arm64]

View more details · View in GitLab

DataDog/apm-reliability/dd-trace-php | publish docker image for system tests

View more details · View in GitLab

ℹ️ Info

No other issues found (see more)

🧪 All tests passed
❄️ No new flaky tests detected

🎯 Code Coverage (details)
Patch Coverage: 100.00%
Overall Coverage: 60.63% (+0.00%)

Useful? React with 👍 / 👎

This comment will be updated automatically if new data arrives.
🔗 Commit SHA: b4c5969 | Docs | View more details | Give us feedback!

@pr-commenter

pr-commenter Bot commented Aug 27, 2026

Copy link
Copy Markdown

Benchmarks [ tracer ]

Benchmark execution time: 2026-08-28 15:47:19

Comparing candidate commit b4c5969 in PR branch leiyks/fix-apms-20395 with baseline commit 57dcf90 in branch master.

Found 0 performance improvements and 1 performance regressions! Performance is the same for 192 metrics, 1 unstable metrics.

Explanation

This is an A/B test comparing a candidate commit's performance against that of a baseline commit. Performance changes are noted in the tables below as:

  • 🟩 = significantly better candidate vs. baseline
  • 🟥 = significantly worse candidate vs. baseline

We compute a confidence interval (CI) over the relative difference of means between metrics from the candidate and baseline commits, considering the baseline as the reference.

If the CI is entirely outside the configured SIGNIFICANT_IMPACT_THRESHOLD (or the deprecated UNCONFIDENCE_THRESHOLD), the change is considered significant.

Feel free to reach out to #apm-benchmarking-platform on Slack if you have any questions.

More details about the CI and significant changes

You can imagine this CI as a range of values that is likely to contain the true difference of means between the candidate and baseline commits.

CIs of the difference of means are often centered around 0%, because often changes are not that big:

---------------------------------(------|---^--------)-------------------------------->
                              -0.6%    0%  0.3%     +1.2%
                                 |          |        |
         lower bound of the CI --'          |        |
sample mean (center of the CI) -------------'        |
         upper bound of the CI ----------------------'

As described above, a change is considered significant if the CI is entirely outside the configured SIGNIFICANT_IMPACT_THRESHOLD (or the deprecated UNCONFIDENCE_THRESHOLD).

For instance, for an execution time metric, this confidence interval indicates a significantly worse performance:

----------------------------------------|---------|---(---------^---------)---------->
                                       0%        1%  1.3%      2.2%      3.1%
                                                  |   |         |         |
       significant impact threshold --------------'   |         |         |
                      lower bound of CI --------------'         |         |
       sample mean (center of the CI) --------------------------'         |
                      upper bound of CI ----------------------------------'

scenario:ContextPropagationBench/benchExtractTraceContext64Bit

  • 🟥 execution_time [+22.019ns; +65.981ns] or [+2.628%; +7.874%]

The ThemeManager::render prehook reset $renderHookIds[$spanKey] to 0
unconditionally. A dropped span skips the posthook (dd_uhook_end gates
the end hook on !dyn->dropped_span in tracer/hook/uhook_legacy.c), so
the slot could still hold a live, installed hook id. Once the freed
SpanData's object handle was recycled, spl_object_hash collided and the
reset discarded that id without removing the hook, orphaning it for the
rest of the request.

Remove the hook before resetting the slot. remove_hook() on an already
removed id is a no-op, so the common self-removal path is unaffected.

APMS-20395
@Leiyks
Leiyks marked this pull request as ready for review August 28, 2026 14:25
@Leiyks
Leiyks requested review from a team as code owners August 28, 2026 14:25
@Leiyks
Leiyks requested review from LobeTia and removed request for a team August 28, 2026 14:25
'prehook' => function (SpanData $span, $args) {
'prehook' => function (SpanData $span, $args) use (&$renderHookIds) {
// A dropped span skips the posthook, so its hook may still be installed.
$spanKey = \spl_object_hash($span);

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

spl_object_hash is deprecated since PHP 8.6. You can simply use spl_object_id.
You might want to simply use install_hook() to transfer the hook id via $hook->data from begin to end.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants