Skip to content

CanCreateEntitlementAtAnyBank stays out of consents but at one bank can go in. onBehalfOfUserId Phase 0. #1225

CanCreateEntitlementAtAnyBank stays out of consents but at one bank can go in. onBehalfOfUserId Phase 0.

CanCreateEntitlementAtAnyBank stays out of consents but at one bank can go in. onBehalfOfUserId Phase 0. #1225

name: Build on Pull Request
on:
pull_request:
branches:
- "**"
env:
DOCKER_HUB_ORGANIZATION: ${{ vars.DOCKER_HUB_ORGANIZATION }}
# ---------------------------------------------------------------------------
# compile — compiles everything once, packages the JAR, uploads classes
# test — 3-way matrix downloads compiled output and runs a shard of tests
#
# Wall-clock target:
# compile ~10 min (parallel with setup of test shards)
# tests ~8 min (8 shards in parallel after compile finishes)
# total ~18 min (vs ~40+ min single-job)
# ---------------------------------------------------------------------------
jobs:
# --------------------------------------------------------------------------
# Job 1: compile
# --------------------------------------------------------------------------
compile:
runs-on: ubuntu-latest
# The test job has carried a timeout since it was written; compile and report never did,
# so a hung Maven resolve blocks the build until GitHub's own 6-hour ceiling.
timeout-minutes: 25
steps:
- uses: actions/checkout@v4
- name: Set up JDK 25
uses: actions/setup-java@v4
with:
java-version: "25"
distribution: "temurin"
cache: maven # caches ~/.m2/repository keyed on pom.xml hash
- name: Setup production props
run: |
cp obp-api/src/main/resources/props/sample.props.template \
obp-api/src/main/resources/props/production.default.props
- name: Lint — test-isolation (no setPropsValues at class/feature body)
run: python3 .github/scripts/check_test_isolation.py
- name: Compile and install (skip test execution)
run: |
# -DskipTests — compile test sources but do NOT run them
# Test classes must be in target/test-classes for the test shards.
# `clean` for a guaranteed-correct build (see build_container.yml: a no-clean
# Zinc incremental cache saved only ~17s — compile is dominated by Maven +
# fat-jar packaging + install, not Scala compilation).
MAVEN_OPTS="-Xmx3G -Xss2m -XX:MaxMetaspaceSize=1G" \
mvn clean install -T 4 -Pprod -DskipTests
- name: Upload compiled output
uses: actions/upload-artifact@v4
with:
name: compiled-output
retention-days: 1
# Upload full target dirs — test shards download and run surefire:test
# without recompiling (surefire:test goal bypasses compile lifecycle)
path: |
obp-api/target/
obp-commons/target/
- name: Save .jar artifact
run: |
mkdir -p ./pull
cp obp-api/target/obp-api.jar ./pull/
cp -r obp-api/target/lib ./pull/lib
- uses: actions/upload-artifact@v4
with:
name: ${{ github.sha }}
path: pull/
# --------------------------------------------------------------------------
# Job 2: test (9-way matrix)
#
# Shard assignment (based on actual clean-run timings on ubuntu-latest 2-core):
# Shard 1 ~300s v4_0_0 non-Dynamic (58 classes; Dynamic* split to shard 9)
# Shard 2 ~281s v1_2_1 (largest single suite — isolated)
# Shard 3 ~250s v6_0_0 (split from v2_x; v2_x moved to shard 7)
# Shard 4 ~232s v5_1_0 + v5_0_0 + v3_0_0
# Shard 5 ~252s ResourceDocs + v3_1_0 + v1_4_0 + v1_3_0
# Shard 6 ~200s v7_0_0 + http4sbridge + UKOpenBanking
# Shard 7 ~280s model + views + customer + util + berlin + small data + v2_x
# Shard 8 ~240s connector + auth + login + mgmt + metrics + catch-all
# Shard 9 ~300s v4_0_0 Dynamic* (6 classes: 9 400+ lines each)
# --------------------------------------------------------------------------
test:
needs: compile
runs-on: ubuntu-latest
timeout-minutes: 35
strategy:
fail-fast: false
matrix:
include:
- shard: 1
name: "v4 non-Dynamic"
# v4_0_0 split: non-Dynamic classes only. Dynamic* on shard 9.
# test_filter is a marker, not a literal wildcardSuites value: the "Run tests"
# step below discovers code.api.v4_0_0.* classes at runtime and excludes
# Dynamic* — a static class list here would silently drop any class added
# after the list was written (the shard-8 catch-all treats the whole
# code.api.v4_0_0 package as covered by shard 1, so an omitted class would
# never run on ANY shard). Runtime discovery keeps shard 1 self-updating.
test_filter: >-
code.api.v4_0_0
- shard: 2
name: "v1_2_1 only (largest unsplittable suite, isolated)"
# API1_2_1Test is a single 6604-line suite (~333 scenarios, ~281s). Isolated
# on its own shard: mixing in berlin/management/metrics made this the slowest
# shard (314s); those moved to shards 7/8 to rebalance.
test_filter: >-
code.api.v1_2_1
- shard: 3
name: "v6 only"
# v6_0_0 isolated: 37 files / ~366 scenarios. Previously bundled with v2_x
# causing 700s+ runs; v2_x moved to shard 7 which had headroom (~98s).
test_filter: >-
code.api.v6_0_0
- shard: 4
name: "v5_1 + v5_0 + v3_0"
test_filter: >-
code.api.v5_1_0
code.api.v5_0_0
code.api.v3_0_0
- shard: 5
name: "ResourceDocs + v3_1 + v1_4 + v1_3"
test_filter: >-
code.api.ResourceDocs1_4_0
code.api.v3_1_0
code.api.v1_4_0
code.api.v1_3_0
- shard: 6
name: "v7 + http4sbridge + UKOpenBanking"
test_filter: >-
code.api.v7_0_0
code.api.http4sbridge
code.api.UKOpenBanking
- shard: 7
name: "model + views + customer + util + small data + berlin + v2_x"
# v2_0_0/v2_1_0/v2_2_0 moved here from shard 3 to rebalance after v6_0_0 was isolated.
test_filter: >-
code.model
code.views
code.customer
code.usercustomerlinks
code.api.util
code.errormessages
code.atms
code.branches
code.products
code.crm
code.accountHolder
code.api.berlin
code.api.v2_1_0
code.api.v2_2_0
code.api.v2_0_0
- shard: 8
name: "connector + auth + login + mgmt + metrics + remaining (catch-all)"
# catch-all shard: appends any test package not assigned to shards 1-7
# Root-level code.api tests use class-name prefix matching (lowercase classes).
# NOTE: classes that sit DIRECTLY in package code.api must be listed here by
# FQN-prefix — the catch-all marks the parent package code.api as "covered" once
# any child (code.api.v4_0_0, …) is assigned, so it never appends code.api itself.
test_filter: >-
code.connector
code.util
code.api.Authentication
code.api.dauthTest
code.api.DirectLoginTest
code.api.gateWayloginTest
code.api.OBPRestHelperTest
code.api.AliveCheckRoutesTest
code.api.Http4sOpenIdConnect
code.entitlement
code.bankaccountcreation
code.bankconnectors
code.container
code.management
code.metrics
code.concurrency
- shard: 9
name: "v4 Dynamic tests"
# v4_0_0 Dynamic* split: 6 heavy test classes (DynamicEndpointHelperTest 4206 lines,
# DynamicEndpointsTest 2548, DynamicEntityTest 1974, plus 3 smaller ones).
# Prefix code.api.v4_0_0.Dynamic matches all 6 classes; shard 1's runtime
# discovery excludes this same prefix so no test runs in both shards.
test_filter: >-
code.api.v4_0_0.Dynamic
services:
redis:
image: redis:7-alpine # pinned + small (~30MB) to cut redis docker-pull flakiness
ports:
- 6379:6379
options: >-
--health-cmd "redis-cli ping"
--health-interval 10s
--health-timeout 5s
--health-retries 5
steps:
- uses: actions/checkout@v4
- name: Set up JDK 25
uses: actions/setup-java@v4
with:
java-version: "25"
distribution: "temurin"
cache: maven
- name: Download compiled output
uses: actions/download-artifact@v4
with:
name: compiled-output
- name: Touch artifact files (prevent Zinc recompilation)
run: |
# actions/download-artifact preserves original compile-job timestamps.
# actions/checkout gives source files the current (later) time.
# Zinc sees sources newer than classes → full recompile (~215 s wasted).
# Touching everything in target/ makes all artifact files appear
# just-downloaded (current time) → newer than sources → Zinc skips.
find obp-api/target obp-commons/target -type f -exec touch {} + 2>/dev/null || true
echo "Touched $(find obp-api/target obp-commons/target -type f 2>/dev/null | wc -l) files"
- name: Install local artifacts into Maven repo
run: |
# The compile runner's ~/.m2 is discarded after that job completes.
# Install the two local multi-module artifacts so scalatest:test can
# resolve com.tesobe:* without hitting remote repos.
#
# 1. Parent POM — obp-commons' pom.xml declares obp-parent as its
# <parent>; Maven fetches it when reading transitive deps.
mvn install:install-file \
-Dfile=pom.xml \
-DgroupId=com.tesobe \
-DartifactId=obp-parent \
-Dversion=1.10.1 \
-Dpackaging=pom \
-DgeneratePom=false
# 2. obp-commons JAR with its full POM (lists compile deps inherited
# by obp-api at test classpath resolution time).
mvn install:install-file \
-Dfile=obp-commons/target/obp-commons-1.10.1.jar \
-DpomFile=obp-commons/pom.xml
- name: Setup props
run: |
cp obp-api/src/main/resources/props/sample.props.template \
obp-api/src/main/resources/props/production.default.props
echo connector=star > obp-api/src/main/resources/props/test.default.props
echo starConnector_supported_types=mapped,internal >> obp-api/src/main/resources/props/test.default.props
echo hostname=http://localhost:8016 >> obp-api/src/main/resources/props/test.default.props
echo tests.port=8016 >> obp-api/src/main/resources/props/test.default.props
echo End of minimum settings >> obp-api/src/main/resources/props/test.default.props
echo payments_enabled=false >> obp-api/src/main/resources/props/test.default.props
echo messageQueue.updateBankAccountsTransaction=false >> obp-api/src/main/resources/props/test.default.props
echo messageQueue.createBankAccounts=false >> obp-api/src/main/resources/props/test.default.props
echo allow_sandbox_account_creation=true >> obp-api/src/main/resources/props/test.default.props
echo allow_sandbox_data_import=true >> obp-api/src/main/resources/props/test.default.props
echo sandbox_data_import_secret=change_me >> obp-api/src/main/resources/props/test.default.props
echo allow_account_deletion=true >> obp-api/src/main/resources/props/test.default.props
echo allowed_internal_redirect_urls = /,/oauth/authorize >> obp-api/src/main/resources/props/test.default.props
echo transactionRequests_enabled=true >> obp-api/src/main/resources/props/test.default.props
echo transactionRequests_supported_types=SEPA,SANDBOX_TAN,FREE_FORM,COUNTERPARTY,ACCOUNT,SIMPLE >> obp-api/src/main/resources/props/test.default.props
echo SIMPLE_OTP_INSTRUCTION_TRANSPORT=dummy >> obp-api/src/main/resources/props/test.default.props
echo openredirects.hostname.whitlelist=http://127.0.0.1,http://localhost >> obp-api/src/main/resources/props/test.default.props
echo remotedata.secret = foobarbaz >> obp-api/src/main/resources/props/test.default.props
echo allow_public_views=true >> obp-api/src/main/resources/props/test.default.props
echo SIMPLE_OTP_INSTRUCTION_TRANSPORT=dummy >> obp-api/src/main/resources/props/test.default.props
echo ACCOUNT_OTP_INSTRUCTION_TRANSPORT=dummy >> obp-api/src/main/resources/props/test.default.props
echo SEPA_OTP_INSTRUCTION_TRANSPORT=dummy >> obp-api/src/main/resources/props/test.default.props
echo FREE_FORM_OTP_INSTRUCTION_TRANSPORT=dummy >> obp-api/src/main/resources/props/test.default.props
echo COUNTERPARTY_OTP_INSTRUCTION_TRANSPORT=dummy >> obp-api/src/main/resources/props/test.default.props
echo SEPA_CREDIT_TRANSFERS_OTP_INSTRUCTION_TRANSPORT=dummy >> obp-api/src/main/resources/props/test.default.props
echo allow_oauth2_login=true >> obp-api/src/main/resources/props/test.default.props
echo oauth2.jwk_set.url=https://www.googleapis.com/oauth2/v3/certs >> obp-api/src/main/resources/props/test.default.props
echo ResetPasswordUrlEnabled=true >> obp-api/src/main/resources/props/test.default.props
echo consents.allowed=true >> obp-api/src/main/resources/props/test.default.props
echo hikari.maximumPoolSize=20 >> obp-api/src/main/resources/props/test.default.props
# Enables the Berlin Group v1.3 alias so ResourceDocRegistryParityTest and
# ApiCollectionEndpointTest can exercise a real alias operation id end to end.
echo berlin_group_v1_3_alias_path=0.6/v1 >> obp-api/src/main/resources/props/test.default.props
echo write_metrics=false >> obp-api/src/main/resources/props/test.default.props
# Log emails instead of opening a real SMTP socket: without this,
# LocalMappedConnector.sendCustomerNotification's EMAIL branch calls
# CommonsEmailWrapper.sendTextEmail which throws ConnectException because
# there's no mail server in CI. That surfaces as 500 in any test that
# hits an endpoint triggering the notification (v5 consent flows, etc.).
echo mail.test.mode=true >> obp-api/src/main/resources/props/test.default.props
# Permissions granted to runtime-compiled dynamic-endpoint code inside the security sandbox
# (mirrors default.props / production.default.props). Required so dynamic resource-doc bodies
# can do JSON extraction (reflection) and read OBP props (getenv); without it the sandbox
# denies these and DynamicResourceDocTest's native-execution scenarios fail.
echo 'dynamic_code_sandbox_permissions=[new java.net.NetPermission("specifyStreamHandler"), new java.lang.reflect.ReflectPermission("suppressAccessChecks"), new java.lang.RuntimePermission("getenv.*"), new java.util.PropertyPermission("cglib.useCache", "read"), new java.util.PropertyPermission("net.sf.cglib.test.stressHashCodes", "read"), new java.util.PropertyPermission("cglib.debugLocation", "read"), new java.lang.RuntimePermission("accessDeclaredMembers"), new java.lang.RuntimePermission("getClassLoader")]' >> obp-api/src/main/resources/props/test.default.props
# allow_user_generated_scala_code defaults to false everywhere (including test/dev)
# with no run-mode fallback; CI must opt in explicitly so DynamicUtilTest,
# ConnectorMethodTest, AbacRuleTests, DynamicResourceDocTest, DynamicMessageDocTest
# and DynamicCodeKillSwitchTest's ON scenarios can still compile/execute dynamic code.
echo allow_user_generated_scala_code=true >> obp-api/src/main/resources/props/test.default.props
- name: Run tests — shard ${{ matrix.shard }} (${{ matrix.name }})
env:
# This job has declared a redis service since it was written, but nothing ever failed
# when the service was absent: ConcurrentRateLimiterRaceTest and
# MethodRoutingCacheInvalidationTest each `assume` a reachable Redis and cancel
# otherwise, and a cancelled test reports as a pass. Dropping the services: block, or
# a container that never became healthy, would have taken the rate-limiter and
# cache-invalidation races out of the run without changing a single report.
#
# RedisTestTarget turns that cancellation into a failure wherever this is set.
# Developers leave it unset and keep the skip.
OBP_TEST_REDIS_REQUIRED: "true"
run: |
# wildcardSuites requires comma-separated package prefixes (-w per entry).
# The YAML >- scalar collapses newlines to spaces, so we convert here.
FILTER=$(echo "${{ matrix.test_filter }}" | tr ' ' ',')
# Shard 1 (v4 non-Dynamic): discover code.api.v4_0_0 classes at runtime and
# exclude Dynamic* (shard 9's domain), instead of a hand-maintained class list.
# This is what makes the shard-8 catch-all's "code.api.v4_0_0 is covered by
# shard 1" assumption actually true — a class list here could drift from the
# real directory contents and leave a new test running on no shard at all.
if [ "${{ matrix.shard }}" = "1" ]; then
FILTER=$(grep -l '^class.*extends' obp-api/src/test/scala/code/api/v4_0_0/*.scala \
| xargs -I{} grep -hoP '^class \K[A-Z][A-Za-z0-9_]+' {} \
| grep -v '^Dynamic' \
| sed 's/^/code.api.v4_0_0./' \
| paste -sd, -)
fi
# Shard 8 is the catch-all: append any test package not explicitly
# assigned to shards 1–7 and 9, so new packages are never silently skipped.
if [ "${{ matrix.shard }}" = "8" ]; then
# Shard 1 discovers v4 non-Dynamic classes at runtime (see above); shard 9
# covers Dynamic*. Use code.api.v4_0_0 as the assigned prefix so the
# catch-all treats the whole v4_0_0 package as covered — true as long as
# shard 1's discovery and shard 9's Dynamic prefix together span it.
SHARD1="code.api.v4_0_0"
SHARD2="code.api.v1_2_1"
SHARD3="code.api.v6_0_0"
SHARD4="code.api.v5_1_0 code.api.v5_0_0 code.api.v3_0_0"
SHARD5="code.api.ResourceDocs1_4_0 code.api.v3_1_0 code.api.v1_4_0 code.api.v1_3_0"
SHARD6="code.api.v7_0_0 code.api.http4sbridge code.api.UKOpenBanking"
SHARD7="code.model code.views code.customer code.usercustomerlinks \
code.api.util code.errormessages code.atms code.branches \
code.products code.crm code.accountHolder code.api.berlin \
code.api.v2_1_0 code.api.v2_2_0 code.api.v2_0_0"
ASSIGNED="$SHARD1 $SHARD2 $SHARD3 $SHARD4 $SHARD5 $SHARD6 $SHARD7 ${{ matrix.test_filter }}"
# Discover all packages that contain at least one .scala test file
ALL_PKGS=$(find obp-api/src/test/scala obp-commons/src/test/scala \
-name "*.scala" 2>/dev/null \
| sed 's|.*/test/scala/||; s|/[^/]*\.scala$||; s|/|.|g' \
| sort -u)
EXTRAS=""
for pkg in $ALL_PKGS; do
covered=false
for prefix in $ASSIGNED; do
if [[ "$pkg" == "$prefix" || "$pkg" == "$prefix."* || "$prefix" == "$pkg."* ]]; then
covered=true; break
fi
done
[ "$covered" = "false" ] && EXTRAS="$EXTRAS,$pkg"
done
[ -n "$EXTRAS" ] && echo "Catch-all extras added to shard ${{ matrix.shard }}:$EXTRAS"
FILTER="${FILTER}${EXTRAS}"
fi
# `mvn process-resources scalatest:test` — run process-resources (copies the
# dynamically-generated props from src/main/resources onto the classpath at
# target/classes/props) then the scalatest:test goal. This SKIPS the
# compile/testCompile lifecycle phases (~208s/shard: classes already come from
# the compiled-output artifact + touch trick) while still placing
# test.default.props on the classpath. (Plain goal-only `scalatest:test` skips
# process-resources → Props init fails → 0 tests but BUILD SUCCESS = false green.)
# -pl obp-commons,obp-api: obp-commons' own 5 util suites run on whichever
# shard's filter matches com.openbankproject.* (the catch-all shard); on every
# other shard the filter matches nothing in obp-commons → 0 tests there.
# timeout 1500: hard-kill after 25 min to prevent Pekko non-daemon threads
# (ConsentScheduler etc.) from keeping the JVM alive after tests complete.
# Exit code 124 (timeout) is treated as success — tests are done, JVM just hung.
# set +e: GitHub Actions uses -eo pipefail by default; without it, a 124 exit from
# timeout would abort the step before the rc check below can run.
set +e
MAVEN_OPTS="-Xmx3G -Xss2m -XX:MaxMetaspaceSize=1G" \
timeout 1500 mvn process-resources scalatest:test -pl obp-commons,obp-api -DfailIfNoTests=false \
-DwildcardSuites="$FILTER" \
> maven-build-shard${{ matrix.shard }}.log 2>&1
rc=$?
set -e
# timeout returns 124 when the JVM was killed. That is only benign when the tests had
# successfully finished but Pekko non-daemon threads kept the JVM alive. We must
# require proof from the log instead of blindly converting 124 to success.
if [ $rc -eq 124 ]; then
if grep -q "BUILD SUCCESS" maven-build-shard${{ matrix.shard }}.log; then
rc=0
fi
fi
exit $rc
- name: Report failing tests — shard ${{ matrix.shard }}
if: always()
run: |
echo "Checking shard ${{ matrix.shard }} log for failing tests..."
if [ ! -f maven-build-shard${{ matrix.shard }}.log ]; then
echo "No build log found."; exit 0
fi
echo "=== RECOMPILATION CHECK ==="
if grep -c "Compiling " maven-build-shard${{ matrix.shard }}.log > /dev/null 2>&1; then
echo "WARNING: Scala recompilation occurred on this shard:"
grep "Compiling " maven-build-shard${{ matrix.shard }}.log | head -10
else
echo "OK: no recompilation (Zinc used pre-compiled classes)"
fi
echo ""
echo "=== BRIDGE / UNCAUGHT EXCEPTIONS ==="
grep -n "\[BRIDGE\] Exception\|Uncaught exception in dispatch\|requestScopeProxy=" \
maven-build-shard${{ matrix.shard }}.log | head -200 || true
echo ""
echo "=== FAILING TEST SCENARIOS (with 30 lines context) ==="
# maven.test.failure.ignore=true (root pom) makes mvn exit 0 even when a suite
# aborts entirely — "*** FAILED ***" alone misses that, since scalatest prints
# "*** RUN ABORTED ***" / "*** SUITE ABORTED ***" instead for e.g. an
# ExceptionInInitializerError, and neither pattern was being checked. That let
# a genuinely aborted suite report CI green silently.
if grep -C 30 -n -E "\*\*\* FAILED \*\*\*|\*\*\* RUN ABORTED \*\*\*|\*\*\* SUITE ABORTED \*\*\*" \
maven-build-shard${{ matrix.shard }}.log; then
echo "Failing/aborted tests detected in shard ${{ matrix.shard }}."
exit 1
else
echo "No failing tests detected in shard ${{ matrix.shard }}."
fi
- name: Upload Maven build log — shard ${{ matrix.shard }}
if: always()
uses: actions/upload-artifact@v4
with:
name: maven-build-log-shard${{ matrix.shard }}
if-no-files-found: ignore
path: maven-build-shard${{ matrix.shard }}.log
- name: Upload test reports — shard ${{ matrix.shard }}
if: always()
uses: actions/upload-artifact@v4
with:
name: test-reports-shard${{ matrix.shard }}
if-no-files-found: ignore
path: |
obp-api/target/surefire-reports/**
obp-commons/target/surefire-reports/**
**/target/scalatest-reports/**
**/target/site/surefire-report.html
**/target/site/surefire-report/*
# --------------------------------------------------------------------------
# Job 3: report — per-test speed (unit/pure vs integration, all http4s)
# --------------------------------------------------------------------------
report:
needs: test
runs-on: ubuntu-latest
if: always()
timeout-minutes: 10
steps:
- uses: actions/checkout@v4
- name: Download test reports — all shards
uses: actions/download-artifact@v4
with:
pattern: test-reports-shard*
path: all-reports
merge-multiple: true
- name: Per-test speed — unit/pure vs integration (all http4s)
run: python3 .github/scripts/test_speed_report.py all-reports