=========================== Migrating to ActingWeb 3.13 =========================== Overview ======== ActingWeb 3.13 is a **DynamoDB scalability release**. It fixes two measured superlinear cost defects (full-table scans on per-actor reads; a ``DescribeTable`` control-plane call on every accessor construction), removes hot-path read amplification, defaults auto-created tables to on-demand billing, and — the one change that affects behaviour for deployments that change nothing — **flips the default property reverse-lookup mechanism to lookup-table mode**, backed by a redesigned (v2, digest-keyed) lookup table. Nothing in this release requires code changes. What it may require is **one operational step**: running the lookup-table backfill script so reverse lookups (find-actor-by-email/oauthId — typically your login path) are served from the new table instead of deprecated fallbacks. Which deployment am I? ====================== Find your row; do what its column says. "Implicit legacy" means you never called ``with_legacy_property_index()`` and never set ``USE_PROPERTY_LOOKUP_TABLE``. .. list-table:: :header-rows: 1 :widths: 34 33 33 * - State before 3.13 - After upgrading (no config change) - Action needed * - Implicit legacy; properties table **has** the ``property-index`` GSI (created by an older ActingWeb) - Reverse lookups keep working via the **deprecated GSI fallback** (a warning is logged per hit; an ERROR is logged at startup) - Run the backfill (below). Optionally delete the now-unused GSI to halve property write cost — only after verifying. * - Implicit legacy; properties table has **no** GSI (created before the GSI existed). Reverse lookups were **crashing** before 3.13. - Reverse lookups now work IF the lookup table is populated; until then they return ``None`` (startup ERROR tells you) - Run the backfill (below). * - Lookup mode already enabled (``with_legacy_property_index(False)``) on the **v1** lookup table - Reverse lookups keep working via the **deprecated v1 fallback** (warning per hit; startup ERROR) - Run the backfill, verify, then **drop the v1 ``_property_lookup`` table**. * - Explicit legacy (``with_legacy_property_index(True)`` or ``USE_PROPERTY_LOOKUP_TABLE=false``) - Unchanged: legacy GSI path stays active. If the table lacks the GSI you now get an actionable error instead of an opaque crash. - None immediately. Legacy mode is deprecated — plan the migration. * - Fresh deployment - Lookup-table mode, on-demand billing, no legacy GSI — the intended end state - None. Required: run the backfill (all upgrading deployments using reverse lookup) =========================================================================== The lookup table is **derived data**; the properties table is the source of truth. The script rebuilds the v2 table from it — streaming, rate-limited, resumable and idempotent:: # With the SAME environment as the app (AWS_DB_PREFIX, region, # credentials, INDEXED_PROPERTIES if customised): poetry run python scripts/backfill_property_lookup.py --dry-run poetry run python scripts/backfill_property_lookup.py --rps 50 Notes: - ``--rps`` caps items/second — a full-table scan against production should not brown-out serving traffic. ``--segments N`` parallelises. - Interrupted runs resume from ``--checkpoint-file``. - Values are copied **verbatim** (no normalisation) — runtime lookups are exact-match. - A value shared by two actors is reported as a **collision** and not overwritten (exit code 1). Resolve manually and re-run — re-runs are idempotent. Verify, then clean up: 1. Watch logs: reverse lookups should stop logging ``DEPRECATED: reverse lookup ... served from ...`` warnings, and the startup ERROR should disappear on the next restart. 2. **v1 cohort**: drop the old table — ``aws dynamodb delete-table --table-name _property_lookup``. (Never touch ``_properties`` — that is the source of truth.) 3. **GSI cohort (optional, saves ~half of property write cost)**: delete the legacy index:: aws dynamodb update-table --table-name _properties \ --global-secondary-index-updates '[{"Delete":{"IndexName":"property-index"}}]' Only after the backfill is verified: deleting the GSI removes the fallback the un-backfilled state depends on. Note that deleting the GSI makes a later return to legacy mode impossible without recreating the index. Rollback: setting ``USE_PROPERTY_LOOKUP_TABLE=false`` (or ``with_legacy_property_index(enable=True)``) restores the legacy path. (3.13 also fixed a bug where the env variable was silently ignored by apps using the fluent builder — the rollback now actually works.) Recommended: convert old auto-created tables to on-demand billing ================================================================= Tables the library auto-created before 3.13 are PROVISIONED with tiny capacities — typically ``_property_lookup`` (2 RCU / 1 WCU: a hard wall on the login path) and ``_peertrustees``. New tables are on-demand; **existing tables are not changed automatically**. Convert them in place:: aws dynamodb update-table --table-name _peertrustees \ --billing-mode PAY_PER_REQUEST - AWS allows one billing-mode switch per table per 24 hours. - **Never delete and recreate a table to change billing** — the v1 lookup table (pre-backfill) and every other table hold live data. - Check for other PROVISIONED stragglers: ``aws dynamodb list-tables`` + ``describe-table``. Recommended: disable auto-creation in production ================================================ If your tables are managed by CloudFormation/Terraform, turn off the library's table auto-creation and slim the runtime role:: AWS_DB_AUTO_CREATE_TABLES=false # env, or: app.with_dynamodb(auto_create_tables=False) With auto-creation off the library never calls ``DescribeTable`` or ``CreateTable``, so both can be dropped from the runtime IAM policy. If your IAM policy lists tables by name, add the new ``_property_lookup_v2`` table. Behaviour changes to be aware of ================================ - **Reverse-lookup matching is now (name, value)**, not value-only: the legacy GSI matched on value alone, so a value shared across *different* property names could resolve "cross-name". The lookup table cannot, and the migration fallback tiers (v1 table, legacy GSI, PostgreSQL scan) now also filter by property name — so the value-only cross-name match is gone on every path, not just the primary one. The fallbacks remain only to serve un-backfilled deployments and are removed in the next major release. - **Non-indexed property names return None** from ``Actor.get_from_property()`` / ``ActorInterface.get_by_property()`` (with a warning) instead of silently querying the legacy path. Add names to ``with_indexed_properties()`` to make them resolvable. - **Cross-actor lookup collisions are refused and logged** instead of silently overwritten (DynamoDB previously last-writer-wins; PostgreSQL already refused — the backends now agree, including idempotent re-creates). - **Trust/peer-trustee list ordering is deterministic** (range-key sorted) instead of arbitrary scan order. - **Subscription-suspension and peer-trustee-list accessors now auto-create their tables** like every other accessor (previously a fresh deployment crashed on first use of suspension). - The lookup table stores **SHA-256 digests, not plaintext values** — update your data map (see ``docs/reference/security.rst``). No changes needed for PostgreSQL schemas (the lookup table and its Alembic migration already exist); the default flip applies to both backends.