Skip to content

Re-enable full WASM marimo Playwright test suite #513

Open

Description

@paddymul

Context

The WASM marimo Playwright tests were trimmed to a single smoke test to keep CI fast and reliable. The full test suite should be re-enabled once WASM test infrastructure is more stable.

Tests to re-enable

These tests were in packages/buckaroo-js-core/pw-tests/wasm-marimo.spec.ts and should be added back:

test('WASM notebook displays version info and data', async () => {
  const pageText = await sharedPage.textContent('body');
  expect(pageText).toBeTruthy();
  const widgets = await sharedPage.locator('.buckaroo_anywidget').all();
  expect(widgets.length).toBeGreaterThanOrEqual(1);
});

test('small DataFrame renders in WASM', async () => {
  const firstWidget = sharedPage.locator('.buckaroo_anywidget').first();
  const count = await getRowCount(firstWidget);
  expect(count).toBeGreaterThan(0);
});

test('cell values are readable after WASM load', async () => {
  const firstWidget = sharedPage.locator('.buckaroo_anywidget').first();
  try {
    const firstCellText = await getCellText(firstWidget, 'a', 0);
    expect(firstCellText).toBeTruthy();
  } catch (e) {
    const cells = await firstWidget.locator('.ag-cell').all();
    expect(cells.length).toBeGreaterThan(0);
  }
});

test('column headers are visible in WASM', async () => {
  const firstWidget = sharedPage.locator('.buckaroo_anywidget').first();
  const headers = await firstWidget.locator('.ag-header-cell-text').all();
  expect(headers.length).toBeGreaterThan(0);
});

test('summary stats grid renders in WASM', async () => {
  const firstWidget = sharedPage.locator('.buckaroo_anywidget').first();
  const statsCells = await firstWidget.locator('.ag-cell').all();
  expect(statsCells.length).toBeGreaterThan(0);
});

test('second widget (if exists) also renders in WASM', async () => {
  const widgets = await sharedPage.locator('.buckaroo_anywidget').all();
  if (widgets.length >= 2) {
    await widgets[1].waitFor({ state: 'visible', timeout: 30_000 });
    const secondWidgetCells = await widgets[1].locator('.ag-cell').all();
    expect(secondWidgetCells.length).toBeGreaterThan(0);
  }
});

test('WASM notebook remains responsive after scroll', async () => {
  const widgets = await sharedPage.locator('.buckaroo_anywidget').all();
  if (widgets.length === 0) return;
  const firstWidget = widgets[0];
  const initialRowCount = await getRowCount(firstWidget);
  if (initialRowCount < 10) return;
  try {
    const targetRow = Math.floor(initialRowCount / 2);
    await scrollToRow(firstWidget, targetRow, sharedPage);
    const cellsAfterScroll = await firstWidget.locator('.ag-cell').all();
    expect(cellsAfterScroll.length).toBeGreaterThan(0);
  } catch (e) {
    const cells = await firstWidget.locator('.ag-cell').all();
    expect(cells.length).toBeGreaterThan(0);
  }
});

Helper functions needed

These utility functions are used by the tests above and should be preserved/restored:

async function getCellText(container, colId, rowIndex) { ... }
async function getRowCount(container) { ... }
async function scrollToRow(container, targetRow, page) { ... }

Notes

  • All tests share a single page load (Pyodide init takes 15-30s)
  • Tests run in serial mode on the shared page
  • Use npx serve (not Python http.server) for the WASM file server
  • Config: playwright.config.wasm-marimo.ts

🤖 Generated with Claude Code

Activity

  1. paddymul commented on Feb 26, 2026

    @paddymul
    CollaboratorAuthor

    Implementation Plan

    Overview

    Re-enable the WASM marimo Playwright tests that were trimmed to a single smoke test.

    Steps

    1. Inventory disabled tests — check packages/buckaroo-js-core/pw-tests/wasm-marimo.spec.ts for the full list of tests that were commented out or removed.

    2. Re-enable incrementally — add tests back one at a time:

      • Start with the simplest (e.g. basic widget render, column header visibility)
      • Then interactive tests (sorting, filtering, search)
      • Then edge cases (empty DataFrame, large DataFrame)
    3. Fix flakiness for each re-enabled test:

      • Increase timeouts for WASM load (pyodide cold start is slow)
      • Add waitForSelector or waitForFunction guards before assertions
      • Add retry logic for network-dependent operations (micropip installs)
    4. CI config — ensure the WASM test job has:

      • Sufficient timeout (10+ minutes for pyodide cold start)
      • Caching for pyodide artifacts to speed up subsequent runs
      • Clear failure output (screenshots on failure)
    5. Gate — once all tests pass reliably for 5+ consecutive CI runs, remove the "trimmed" comment and consider this done.

    Files to modify

    • packages/buckaroo-js-core/pw-tests/wasm-marimo.spec.ts — re-enable tests
    • CI workflow config — timeouts, caching

    @codex

  2. chatgpt-codex-connector commented on Feb 26, 2026

    @chatgpt-codex-connector

    Summary

    • Replaced the trimmed single-smoke WASM spec with a broader serial shared-page suite and restored the helper utilities (getCellText, getRowCount, scrollToRow), then added coverage for baseline rendering, headers/cells, optional second widget, scroll responsiveness, and basic sort/search interactions. packages/buckaroo-js-core/pw-tests/wasm-marimo.spec.tsL1-L158
    • Hardened WASM Playwright configuration by enabling CI retry, increasing timeout, and retaining richer failure diagnostics (trace/screenshot/video) with stable HTML reporting behavior. packages/buckaroo-js-core/playwright.config.wasm-marimo.tsL9-L40
    • Updated the WASM CI job to run with a longer timeout, cache generated WASM marimo artifacts, and always upload Playwright artifacts for clearer failure debugging. .github/workflows/checks.ymlL488-L537

    Testing

    • ⚠️ bash scripts/test_playwright_wasm_marimo.sh (failed in this environment because Playwright Chromium download returned upstream HTTP 403 from CDN).
    • ✅ cd packages/buckaroo-js-core && pnpm exec playwright test --config playwright.config.wasm-marimo.ts --list

    View task →

  3. paddymul commented on Feb 26, 2026

    @paddymul
    CollaboratorAuthor

    Related: PRs #509, #531, #532, #542

    The WASM marimo test infrastructure is now in place. The remaining work is re-enabling the specific tests listed in this issue one by one and fixing any flakiness. See also #502 (pyodide CI).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions