Skip to content

Functional tests risky when TYPO3 v15 Bootstrap no longer opens an output buffer #732

Description

@sbuerk

Problem

TYPO3 Core main (v15) is about to remove the implicit output buffer that
Bootstrap::init() opened as its very first action:

With that ob_start() gone, every functional test turns risky:

Test code or tested code closed output buffers other than its own

Core CI for that change fails in all functional jobs (sqlite 3/3, mariadb 6/6,
postgres 10/10) while unit, phpstan, cgl, lint, integration,
documentation rendering and the playwright e2e jobs stay green — the breakage
is confined to the functional-test bootstrap of this package.

Root cause

PHPUnit wraps every test in its own output buffer and remembers the level
(PHPUnit\Framework\TestCase::startOutputBuffering()). If the level is lower
when the test ends, the test is reported as risky.

This package contains two unconditional ob_end_clean() calls that exist
only to undo Bootstrap::startOutputBuffering(). Without that ob_start(),
they close PHPUnit's own buffer instead:

  1. Classes/Core/Testbase.php:757-760setUpBasicTypo3Bootstrap()

    $container = Bootstrap::init($classLoader);
    // Make sure output is not buffered, so command-line output can take place and
    // phpunit does not whine about changed output bufferings in tests.
    ob_end_clean();

    setUpBasicTypo3Bootstrap() is called from both branches of
    FunctionalTestCase::setUp() (:311 and :429), i.e. once per test — which
    is why all tests of a test case are affected, not just the first.

  2. Classes/Core/Functional/FunctionalTestCase.php:1008 + :1053-1054
    executeFrontendSubRequest()

    $container = Bootstrap::init(ClassLoadingInformation::getClassLoader());
    // ...
    } finally {
        // Somewhere an ob_start() is called in frontend that is not cleaned. Work around that for now.
        ob_end_clean();
        FrameworkState::pop();
    }

    The "somewhere" in that comment is the Bootstrap::init() call a few lines
    above.

These are the only two Bootstrap::init() call sites in this package;
Resources/Core/Build/UnitTestsBootstrap.php only calls
Bootstrap::initializeClassLoader(), which is why unit tests are unaffected.

Reproduction

TYPO3 Core main checkout with the Gerrit change cherry-picked:

Build/Scripts/runTests.sh -s functional -d sqlite typo3/sysext/core/Tests/Functional/Database/
→ Tests: 376, Assertions: 1309, Skipped: 34, Risky: 376
Build/Scripts/runTests.sh -s functional -d sqlite typo3/sysext/frontend/Tests/Functional/Rendering/TypeRenderingTest.php
→ Tests: 9, Assertions: 9, Risky: 9

Expected fix

Both call sites must become output-buffer-level aware rather than being
deleted outright: branch main supports typo3/cms-core: 14.*.*@dev || 15.*.*@dev, and TYPO3 v14 Bootstrap::init() still opens the buffer. A plain
removal would flip the failure mode on v14 to
Test code or tested code did not close its own output buffers.

Record ob_get_level() before Bootstrap::init() and only unwind buffers that
were opened above that level. That works unchanged on v14 and v15 and never
touches PHPUnit's own buffer.

Verification of the proposed fix

Same Core checkout, both call sites made level-aware:

Suite Before After
core/Tests/Functional/Database/ 376 risky Tests: 376, Skipped: 34 — 0 risky
frontend/…/Rendering/TypeRenderingTest.php + UriPrefixRenderingTest.php 9 risky OK (13 tests, 29 assertions)
core/Tests/Functional/ (breadth check) Tests: 4224, Assertions: 56298, Skipped: 36 — 0 risky, 0 failures

(PHP 8.5, sqlite, podman.)

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions