Encrypt stored mail and search content
When encryption is enabled or required, MailGen stores managed bodies and related content in owner-bound encrypted form. The user keeps the private-key backup. The server decrypts only during an active key lease: temporary permission to use that owner's private key. Read key setup first.
An authorized backend and external AI provider can temporarily receive readable content. Indexed search terms remain searchable. This is not complete search-index secrecy.
Existing-content conversion checks ownership and content identity before rewriting stored data. It does not require a reset. Current-object conversion does not automatically remove earlier versions or backups. Use encryption settings and Sync Center for managed conversion state.
Prepare an owner-scoped migration
The backend tool is mailGen/msmicroservices-architexture/scripts/email-encryption-migration.ts. Its implementation modules live under mailGen/msmicroservices-architexture/src/lib/utils/encryption/migration/. It inventories and converts one Better Auth owner, not every mailbox in a tenant.
Before running it, prepare:
- The exact Better Auth owner ID and the intended report path.
- Process configuration for
DATABASE_URL, S3, KeyStore, and search when requested. The script does not load.envautomatically. - The owner's public-key registry entry and active private-key lease for decrypt/readback checks.
- Conditional S3 writes, checked first with a separate disposable object on an S3-compatible gateway.
A missing or locked owner key leaves explicit pending work. Do not create a replacement key for existing ciphertext. The owner's app login ID differs from each provider mailbox ID. See identity and storage concepts.
Inventory and convert content
From mailGen/msmicroservices-architexture, replace the owner placeholder:
node --import tsx scripts/email-encryption-migration.ts \
--owner '<Better Auth owner id>' \
--report /tmp/email-encryption-report.json
The default is dry-run. Read the report's owner, counters, and pending categories before changing data. To apply conversion to that same owner:
node --import tsx scripts/email-encryption-migration.ts \
--owner '<Better Auth owner id>' \
--report /tmp/email-encryption-report.json \
--apply
--apply changes canonical objects and eligible SQL/history content. It does not reset the mailbox or change retention policy.
| Option | Contract |
|---|---|
--owner | Required Better Auth owner ID |
--report | Required report filename. The tool creates its parent directory |
--apply | Apply writes. Defaults to false |
--dry-run | Explicit read/inventory mode. Cannot combine with --apply |
--backfill-search | Rebuild search from canonical content. Requires --apply, a fresh target, and active owner unlock |
--search-index | Fresh index base for search backfill. Pattern: ^[a-z0-9_][a-z0-9_.-]{0,200}$ |
The tool writes the report atomically through a temporary file. It reports owner scope, counters, and pending work, not body content or key secrets.
Conversion guarantees and limits
Inventory covers canonical HTML, unique-body sidecars, raw FastTransfer bytes, legacy body objects, SQL bodies/previews, and eligible retained operation history. It also covers automatic-reply proposals/decisions, transcripts, provider history, and event payloads. Active commands and journals can remain pending to preserve worker revisions.
For an object, the converter gets bytes and ETag from the same read. It encrypts and decrypts the candidate for comparison. Then it writes an encrypted staging object. It reads that object back and checks the decrypted bytes. Only after these checks does an If-Match write target the original ETag. It reads and checks the canonical object again afterward.
A concurrent change becomes pending work rather than a blind overwrite. Binary FastTransfer content remains bytes. A malformed encryption envelope fails instead of falling back to plaintext.
SQL full-body content clears only after a verified canonical copy contains identical plaintext bytes. Compare-and-update conditions preserve concurrent row changes. A body without an object pointer gets a verified immutable object and a conditional pointer update. If that update fails, the extra object remains. Unique-body SQL clears only after a byte-identical sidecar passes its checks.
Eligible terminal outbox/admission history keeps the original PostgreSQL JSONB text bytes inside an encrypted container. After authorized decryption, proof verification receives those same bytes. Conversion does not replace the original signature or digest. Terminal command eligibility checks the operation identified by payload.operationId, not the outbox event ID. The write rechecks terminal state and the expected stored value.
An automatic-reply fork inherits owner scope through verified chat.forked provenance. Actor, tenant, and account must agree. Cycle/depth checks bound the lineage at 32. Old forks without valid provenance need separate review.
Each rerun inventories current content again, including decrypt checks for objects already encrypted. An earlier success does not make a later changed object automatically complete.
| Exit code | Meaning |
|---|---|
0 | Script finished without reported pending coverage or a runtime failure |
2 | Reported pending coverage or concurrent-change counters remain |
1 | Runtime/configuration/verification failure |
Inspect the report even after a successful write or search backfill. The inventory records separate browser-recovery, broker-retention, and historical-version/backup tasks. Exit 2 can therefore remain after canonical conversion succeeds. Resolve each category separately. No exit code proves historical secure deletion.
Restore the search projection
A search projection is the index copy derived from authoritative mailbox storage. Use canonical S3 content and a fresh privacy-compatible index base. Excluded _source fields cannot restore missing full content. Read search availability before changing search routing.
With the required configuration and active owner lease, run from the backend directory:
node --import tsx scripts/email-encryption-migration.ts \
--owner '<Better Auth owner id>' \
--report /tmp/email-encryption-report.json \
--apply --backfill-search --search-index '<fresh-base>'
Replace <fresh-base> with a valid new base name. The tool uses the existing revision-safe mailbox backfill in src/lib/system/opensearch/emailSearchBackfill.ts. It reads bounded database pages and canonical content. It does not blindly reindex excluded fields from an old index.
Each mailbox gets a checkpoint beside the report:
<fresh-base>-<mailbox-id>.search-checkpoint.json
The report records the chosen base and successful backfill completion. Before routing application readers to it, check pending report categories and per-mailbox checkpoints. Also check index coverage and search-result parity.
--search-index changes only the migration process target. It does not switch the application's configuration. After the owner-specific coverage review, the operator separately configures:
| Setting | Role |
|---|---|
OPENSEARCH_ENCRYPTED_INDEX | Fresh privacy-compatible base for the configured application path |
EMAIL_ENCRYPTION_MIGRATED_OWNERS | CSV of owner IDs whose coverage was checked for rollout |
OPENSEARCH_INDEX | Retained legacy base for owners that still need compatibility |
BODY_ENCRYPTION_ALLOW_LEGACY_READ | Legacy body-read gate. A listed migrated owner cannot use plaintext fallback even if this is true |
Use the supported runtime configuration procedure for that separate switch. These settings are not substitutes for completed conversion, correct owner keys, or coverage checks. The migration tool does not change them for the running application.
Retention and recovery
Encrypted staging keys include a ciphertext SHA-256 hash and remain after interruption. The tool preserves old indexes, staging objects, historical object versions, broker history, and backups. It does not claim a privacy-complete state or retroactive secure deletion.
Keep canonical conversion coverage and historical-retention remediation as separate checks. Review browser recovery storage and every pending history category before declaring an owner fully covered. For a failure, preserve the report and correct its scoped cause before rerunning. A reset is not part of this procedure.