Prepare a separate test environment

Use the separate test environment when an explicitly requested check needs fresh storage instead of the normal local databases. It can remain prepared for later work. It does not start automatically or schedule tests.

Before you start

Have the project source, Docker access, and an existing shared proxy. Inspect host capacity and clone isolation. Select one complete active stack before cold bootstrap. Keep production volumes and backups separate.

Prepare the snapshot and routing

Before preparation, commit the intended implementation in every affected repository. Before interpreting build or browser results, check the snapshot source provenance. Confirm the selected revisions, source manifest, and actual resolved ECS_APPLICATION_SOURCE_ROOT. Later root edits do not refresh an existing bundle. Preserve its pinned evidence and prepare a new physical snapshot when current source is required.

1. From the original project root, run bash testInstance.sh prepare. It creates a physical source snapshot and private configuration without starting services.
2. Add approved test hosts to the existing proxy without editing original hosts. New preparation uses deployment-derived test hosts. For source deployment ttrmb, these are https://test.bff.ttrmb.ecas.local:8443 and https://test.account.ttrmb.ecas.local:8443.
3. Configure name resolution on the browser machine and server/WSL host to reach the proxy. Trust the selected public certificate in the actual browser environment. Import only .crt files, never private keys.
4. Before provider login, register the exact Microsoft callback on both selected hosts. Check certificates and callbacks before startup.
5. For the requested test, run bash testInstance.sh start. Check actual readiness before use.

The generated test policy disables provider/ingress flags. A test snapshot does not automatically contain a usable mailbox connector. Missing connector or login policy can produce LOGIN_PROVIDER_DISABLED. Do not bypass test policy by attaching production stores.

Fresh Microsoft login prerequisites

Provider login requires a separately reviewed, operator-approved snapshot with the required configuration. The fixed test wrapper remains personal-only, disables Microsoft, and generates a random ECAS_DEPLOYMENT_TENANT_ID authorization namespace. Keep that policy unchanged.

Microsoft login requires complete BFF_MICROSOFT_CLIENT_ID, BFF_MICROSOFT_CLIENT_SECRET, and BFF_MICROSOFT_TENANT_ID settings. It also requires a persisted tenant connection with granted consent, a service-principal ID, and a credential reference. The credential must not be expired. The connection must match the configured tenant selection. This gate applies even when Graph workers remain off. Copying OAuth settings and registering redirects alone does not satisfy it.

A fresh isolated database needs its first legitimate tenant connection before Microsoft app login. The normal seed does not create it. Complete the operator preparation below before login. Never fabricate active grants or mark unverified consent as granted. Keep production storage credentials, encryption keys, sessions, and data isolated from the test environment.

For an operator-approved separate custom snapshot with a usable persisted Microsoft tenant connection, keep MICROSOFT_CONNECTOR_ENABLED=true. This enables the Microsoft profile so normal Microsoft connector services run. Startup rejects personal-only mode (MICROSOFT_CONNECTOR_ENABLED=false) when such a connection exists. Do not bypass this consistency guard or change the fixed testInstance.sh wrapper policy. Its generated disabled flags still apply.

The Microsoft profile also requires the snapshot's own GRAPH_WEBHOOK_URL and GRAPH_LIFECYCLE_WEBHOOK_URL, plus complete Cloudflare lifecycle management settings. Do not borrow production Graph callbacks. Startup rejects missing Graph ingress or Cloudflare settings, even before mailbox onboarding.

The public Graph ingress hostname must be covered by the actual Cloudflare edge TLS certificate. In a Cloudflare full DNS setup with only Universal SSL, choose the apex or a one-level subdomain, such as test-graph-DEPLOYMENT.example.com. Deeper proxied hostnames require a separately provisioned edge certificate that covers them. See Universal SSL limitations. This differs from the DNS-only private Office callback, which uses its own DNS-01 certificate and can retain its supported nested hostname.

Follow the canonical notification setup and Graph cryptography setup. Use fresh, snapshot-owned client state and certificate identity, then preserve them across restarts and for existing subscriptions. Normal initialization prepares the instance-local certificate files.

GraphOneTimeUserOnboarder requires both GRAPH_WEBHOOK_URL and GRAPH_WEBHOOK_CLIENT_STATE. Generate fresh client state for the isolated instance. Store it through the existing private environment setup. Configure that same value for the onboarder and WebhookCatcher. Route the isolated webhook and lifecycle URLs to this instance's WebhookCatcher, with both paths intact. Preserve the value across restarts. A callback for another instance cannot satisfy these prerequisites.

The standard lifecycle provisions or reuses the exact clone-owned Cloudflare tunnel and checks both public endpoints with a validationToken challenge. Keep its private CLOUDFLARE_TOKEN, CLOUDFLARE_ACCOUNT_ID, and CLOUDFLARE_ZONE_ID in the snapshot's own lifecycle root .env. Compose exposes these to lifecycle management through x-runtime, not application service environments.

Enabling the Microsoft profile does not start mailbox onboarding or prove that it is complete. Use the normal Start onboarding action separately.

Prepare the separate app-only credentials through Microsoft Graph setup. The canonical environment reference defines BACKEND_TENANT_ID, BACKEND_CLIENT_ID, and BACKEND_CLIENT_SECRET. These credentials serve backend Graph access, separately from BFF_MICROSOFT_* login credentials.

Before bootstrap, an authorized operator must acquire an actual token and complete a read-only Graph request with the intended credentials. Only then use the existing canonical connection bootstrap against the isolated database. Its ensure helper writes granted consent from configuration alone. That write does not prove external consent or Graph access. The current application has no normal connector-create administration API or setup CLI. This prerequisite needs operator preparation, not an automated bootstrap command.

After app login, complete the separate Microsoft mailbox link. App login alone does not link the intended mailbox. Before onboarding, open Open email key setup when offered. Use the selected HTTPS origin with browser certificate trust established.

For a fresh owner, choose Generate and download email key in the encryption dialog. Store the backup securely. Confirm I saved my backup in a safe place. Then choose Activate email encryption.

For an existing owner, import that owner's original backup instead. Follow personal key backup and unlock. Then return to onboarding and choose Start onboarding. Graph notification keys and personal email keys serve separate purposes.

Select the Microsoft deployment tenant before bootstrap

For a separately prepared, operator-enabled Microsoft variant, select ECAS_DEPLOYMENT_TENANT_ID before the first authorization bootstrap. Set it to the intended, verified app-only BACKEND_TENANT_ID, which identifies the Entra tenant. Complete the credential checks above before bootstrap. This same-tenant requirement applies to the current legacy Microsoft onboarding admission path.

When the verified Microsoft identity belongs to another deployment tenant, BFF requires an existing verified Microsoft mailbox binding and application connection. Fresh onboarding has no such mailbox binding. Enabling Microsoft with the stock random deployment tenant can therefore return HTTP 409 from POST /api/onboarding/start. The C# Graph onboarder requires V1 authorization admission and a matching tenant. The existing legacy admission path does not support fresh Microsoft onboarding across different deployment tenants.

Keep independent test databases, auth/encryption keys, object stores, Kafka/Qdrant namespaces, and cookies. A tenant ID alone does not isolate storage. Matching the Entra tenant does not authorize shared production storage or keys.

If the test already contains initialized authorization state, do not change only its .env tenant value. Preserve a backup before any migration. A complete migration must cover the owned authorization namespace, grants, and history. Alternatively, stop the selected test runtime, preserve its data, and prepare a fresh isolated Microsoft variant with the intended tenant before bootstrap. Never modify immutable provider bindings or bypass capability guards. These prerequisites do not establish successful live onboarding.

Certificates and callbacks

Use the two optional origin selectors to select public names. Runtime identity and volumes remain isolated. The same reference lists all eight derived URL variables.

Proxy routing, certificate coverage, and client trust are separate prerequisites. Add separate BFF and Account routes to their respective runtime upstreams. Do not route Account through the BFF host. The proxy certificate must cover both selected hostnames in its Subject Alternative Names (SANs). Include the derived FormGen and documentation hosts when those services are used. The displayed Common Name (CN) does not establish SAN coverage.

The existing proxy helpers plan create-only additions and refuse conflicting hosts. Preserve production hosts, certificates, and proxy state. The test lifecycle cannot start or restart the shared proxy. Approved route additions can require a configuration reload.

For new BFF host plans with a calculated test origin, the proxy defers DNS resolution for /api/google-mailbox/complete. This protects proxy startup before the BFF upstream exists. The exact path and access-control behavior remain unchanged. Access and error logs for this route remain off. Account keeps its separate route.

Before application startup, check that the owned route is active in Nginx Proxy Manager (NPM) and that TLS works. Application HTTP requests can return 502 until the upstream starts. TLS success alone does not prove application readiness.

This change affects new plans only. An existing offline host can still report host not found for its old literal upstream. An operator must inspect the exact-owned repair helper, docker/proxy/office_deferred_dns.py::repair_plan. Apply the repair through authenticated NPM management, with the ownership and race checks in office_routes.reconcile and apply_changes. Preserve other hosts and custom routes. The application has no normal repair button or operator setup command for this procedure.

Saving the old host in NPM after its upstream starts can temporarily activate its literal configuration. The durable fix still requires the deferred-DNS repair helper. Do not rerun first-machine setup, reset data, or delete certificates for this error.

Trust the certificate in the browser environment

Creating or selecting a proxy certificate does not establish browser trust. Install only the selected public proxy.crt, never its private key. Check its X.509 SHA-256 fingerprint against the certificate actually served by each selected HTTPS host before import:

openssl x509 -in '<selected public proxy.crt>' -noout -fingerprint -sha256

Replace the placeholder with the selected public certificate path. Compare the result with the browser certificate viewer for each host. If fingerprints differ, stop and check the selected route and certificate. Check name resolution on both the browser machine and server/WSL host. Each must reach the intended proxy.

The T3 preview browser can run in Linux/WSL even when the T3 interface runs on Windows. Check the actual browser environment. A Windows Root import does not establish trust for a Linux/WSL preview.

Windows browser

Copy the selected public certificate to Downloads\proxy.crt. Run this command in PowerShell:

certutil.exe -user -addstore Root "$env:USERPROFILE\Downloads\proxy.crt"

The current user's trusted Root store can require interactive approval. Approve the Windows prompt to complete the import.

Linux/WSL Chromium preview

On Ubuntu, libnss3-tools supplies certutil. Use the account that runs the preview browser. Preserve its existing Network Security Services (NSS) database and entries.

Chromium's Linux certificate guide documents the M146 default: ~/.local/share/pki/nssdb. Chromium uses an existing legacy ~/.pki/nssdb instead. Check which database the affected browser actually opens before import. Do not create a replacement database.

Run these commands in that browser account's Linux shell. Set browser_nss_db to the confirmed directory. This example uses the M146 default:

browser_nss_db="$HOME/.local/share/pki/nssdb"
certutil -L -d "sql:$browser_nss_db"

For an existing legacy database that the browser uses, select $HOME/.pki/nssdb instead. Choose an unused nickname. Import only the selected, fingerprint-checked public leaf certificate. For this self-signed server certificate (CA:false), use peer trust P,,. Do not grant CA trust with C,,.

certutil -A -d "sql:$browser_nss_db" -n 'chosen unique nickname' -t 'P,,' -i '<selected public proxy.crt>'
certutil -L -d "sql:$browser_nss_db" -n 'chosen unique nickname'

Replace the nickname and certificate path before import. Check the entry's fingerprint and P,, trust afterward. Repeat for the other selected host only if it serves a different leaf certificate. Preserve all unrelated entries. Linux system trust alone does not establish Chromium NSS acceptance.

Restart and check

After a trust change, fully close and reopen only the affected browser or preview process. Use the app's preview controls when possible. A tab reload or closing and reopening a tab can retain stale certificate-verifier results. If a process restart is necessary, identify the browser parent by its profile path before stopping only that process. Do not terminate the T3 backend or all Chrome processes.

Open both selected HTTPS hosts in the actual browser environment. Check that neither reports a certificate error before login. Keep HTTPS certificate checks enabled. Do not use broad trust bypasses or ignore-HTTPS options.

Microsoft callbacks

Register these Microsoft redirect URIs for the example deployment:

https://test.bff.ttrmb.ecas.local:8443/api/auth/callback/microsoft
https://test.account.ttrmb.ecas.local:8443/api/auth/callback/microsoft

Replace the deployment, suffix, and port with the selected values. Both current handlers use /api/auth/callback/microsoft. An additional registration for /api/account/connect/microsoft/callback does not create a handler. The current source has no alias for that path, so it is not required for the current handler.

With Account login enabled, Account owns the Microsoft login callback. BFF owns the mailbox-link callback and derives its callback origin from MAILGEN_PUBLIC_URL. Register both exact URIs above. The origin allowlist retains exact origins, including scheme and port, and the two flows retain separate OAuth state cookies. A failed mailbox link uses the existing translated alert.

The paired test hosts use ecas-shared-test cookies and separate OAuth state names. The unprefixed deployment retains ecas-shared. See authentication isolation for the shared-domain limit and exact HTTP localhost exception.

After configuration, check actual login, logout isolation, and the Microsoft callback roundtrip in the selected runtime. Generated URLs, proxy plans, and certificate files alone do not prove these outcomes. Remote dependency probes also do not prove a complete hybrid application journey.

Retain and resume

Goal	Command
Inspect state	bash testInstance.sh status
Stop and preserve test data	bash testInstance.sh stop
Resume prepared containers	bash testInstance.sh quick-start
Ordinary restart	bash testInstance.sh restart

The snapshot does not automatically receive later edits. Preserve existing frozen bundles and their historical evidence. Use an explicitly approved new snapshot to include changed origin helpers and configuration while preserving existing test data. File-specific source inclusion does not disable secret/path filters.

The stock generated test default remains ECS_MEMORY_GIB=30, bounded, with a 4 GiB swap budget. The startup minimum guard is authoritative. A separately prepared, operator-enabled Microsoft test with the observed 33 selected services needs a larger existing profile. The onboarder minimum is 2048 MiB, and the full-development independent queue minimum is 768 MiB. This selection exceeds 30 GiB at its minimums. At the 35+ GiB profile minimums, it needs 38,592 MiB, which fits the selected 40 GiB profile. This is not a universal default or a claim of free host RAM.

Before a normal restart, keep only one full stack active and check host RAM and swap admission. Preserve existing databases, object stores, volumes, and keys. Apply an approved profile change through the selected runtime's normal restart, without a reset. A service cgroup out-of-memory (OOM) failure does not establish host OOM. Check host memory and kernel OOM evidence before stopping the full stack.

Graph FTS export now processes one item per chunk to reduce batch buffering. A single large item still needs an adequate service memory budget. Profile selection does not prove successful startup or mailbox completion. Require actual execution evidence before claiming either outcome.

See test runtime details for memory guidance, fresh keys, remote namespaces, and proxy restrictions. Do not mount original development databases, object storage, or unrelated backup-test volumes into this environment.

Use an existing registered browser origin

Prefer deployment-derived test hosts with their own registrations and certificates. If a check specifically requires an existing browser origin, an operator must review a separate snapshot and its private configuration. This is not a custom-identity option of testInstance.sh. Keep the fixed test bundle unchanged. See snapshot lifecycle commands.

For an authorized browser origin such as https://bff.ttrmb.ecas.local:8443, public browser and auth URLs must agree with the proxy routes. Review the corresponding Account origin and registered Microsoft callback before login. Ingress aliases such as ttrmb-bff and ttrmb-account can serve those existing proxy targets while internal identity and storage remain unique.

Keep the original production instance stopped throughout this exclusive alias window. Two instances must not serve the same aliases. Preserve original storage and keys, the shared proxy, and unrelated monitoring. Review these conditions before startup. A routing configuration alone does not prove that login or remote services work.

Prepare private placement variants

An operator can prepare separate private FullLocal and real PartialRemote environment variants. The active root .env selects the mode. Preserve local PostgreSQL, S3, auth/encryption keys, and volumes in both variants. Stop the selected runtime before switching. Ordinary start or restart applies the selected configuration. quickStarter.sh only resumes existing containers.

For retained local OpenSearch data in either mode, see local OpenSearch disk watermarks. The optional override belongs in the physical runtime root's .env and requires an ordinary restart. A copied enabled configuration also enables the override in its destination.

Before real PartialRemote startup, check the owned remote Kafka cluster identity and the existing Qdrant collection. Unique topics, consumer groups, and collection names do not replace Kafka cluster identity. See variant switching and dependency requirements and canonical remote deployment placement.

For a reusable private Office callback, follow the explicit callback setup from the original root. It requires a physical target runtime, the existing healthy shared NPM HTTPS mapping, local Tailscale, renderer Tailscale IPv4 /32 source CIDRs within 100.64.0.0/10, and original-root Cloudflare credentials. The optional suffix derives a separate private callback with a public DNS-01 certificate. It does not transfer production routes or change remote renderer trust. FullLocal remains local. Check the real renderer journey separately. The current prepared remote Coolify renderer also requires the full reviewed COLLABORA_CONTENT_SECURITY_POLICY, including the exact MAILGEN_PUBLIC_URL origin in frame-ancestors, persisted in the actual Coolify service configuration.

For a second callback, the remote operator copies the actual derived REMOTE_WOPI_ORIGIN once into remote WOPI_ADDITIONAL_CALLBACK_ORIGIN. The derived DNS example is https://test.wopi.ttrmb.ecs.t42.hu:8443. This optional setting supplies remote Collabora aliasgroup2; its empty default adds no host. Use a canonical exact HTTPS origin without a wildcard, credentials, or path. Keep required WOPI_CALLBACK_ORIGIN in aliasgroup1 and preserve existing approved origins. See the callback configuration table and scoped recreation procedure.

Keep persisted Coolify raw/generated Compose and service/rendered environments consistent before recreating only the verified Collabora service. Preserve other containers, data, and the proof key. Callback host trust is separate from browser CSP frame-ancestors and DNS/TLS. Configure both callback trust and browser embedding. Acceptance remains pending until an actual document loads in the root application.

Keep exact public-origin and route ownership for Office. Successful remote dependency probes do not prove a complete hybrid application journey or an Office callback.

For fresh storage, standard cold startup installs the canonical actor-rights and policy SQL before the normal seed. It then runs the authorization migrations only if betterauth.user and authorization.subjects are globally empty and the deployment tenant has no migration marker. Existing databases require the explicit authorization migration workflow. Do not replace that workflow with a marker-only insert or an authorization gate bypass.

The BFF development configuration allows the MAILGEN_PUBLIC_URL hostname for HMR alongside the existing Account auth hostnames, without a wildcard. Keep these public URLs aligned with the proxy routes. Restart the BFF development service after changes to next.config.js.

MAILGEN_PUBLIC_URL also adds its exact origin (scheme, hostname, and port) to BFF authenticated realtime and cookie sync. HTTP origin checks also include it for Complexity mutations, approvals, composer draft attachments, admin proxy mutations, and search mutations. These checks use the URL origin, while HMR uses only the hostname. Configure the specific public URL, without a wildcard. Session and permission checks still apply.