Progress

2026-02-23 18:08 - T-001

Overview: Executed seed flow and verified local lookup tables were populated for entity normalization and publish prerequisites.

Completed:

• Ran pnpm seed --entity ions successfully with lowercase entity input
• Verified seed completion log ([seed] Completed lookup seeding for IONS)
• Confirmed local lookup rows exist in core_entity, core_unit, and payment/channel/order lookup tables

Files:

• atlas-ops.db

2026-02-23 18:09 - T-002

Overview: Ran yearly sync commands for 2023-2026 and restored dbt runtime by reinstalling analytics virtualenv dependencies.

Completed:

• Ran pnpm sync --entity IONS --year 2023
• Ran pnpm sync --entity IONS --year 2024
• Ran pnpm sync --entity IONS --year 2025
• Ran pnpm sync --entity IONS --year 2026
• Verified successful sync metadata rows in _pipeline_runs

Blocked:

• int_orders does not retain all historical years after sequential yearly sync runs (current rows are only 2025-2026 + NULL spillover), so the "all years" acceptance is not met.

Learnings:

• Initial sync runs failed because @python/analytics/.venv/bin/dbt had a stale shebang from an old repo path; uv sync --reinstall repaired entrypoints.

Files:

• atlas.db
• @packages/pipeline/atlas.db
• @python/analytics/.venv/bin/dbt

2026-02-23 18:11 - T-003

Overview: Published all target months locally (2025-06/07/10/11/12 and 2026-01/02/03), with additional lookup normalization to handle payment status aliases.

Completed:

• Published all listed months with successful insert output
• Confirmed commerce_order populated across expected months
• Confirmed finance_transaction populated

Blocked:

• _pipeline_runs currently does not record publish stage rows or row_count metadata.

Decisions:

• Updated local finance_payment_status source term mapping for Installment and inserted an ANGSUR alias row to unblock publish for mixed source terms (ANGSUR and ANGSURAN).

Files:

• atlas-ops.db
• @packages/pipeline/atlas.db

2026-02-23 18:13 - T-004

Overview: Generated monthly and quarterly report artifacts end-to-end via unified report command orchestration.

Completed:

• Ran monthly report for IONS 2026-02 and generated JSON + PPTX + PDF
• Ran quarterly report for IONS 2026-Q1 and generated JSON + PPTX + PDF
• Verified report metadata rows recorded in _pipeline_runs

Files:

• output/monthly/IONS-2026-02.json
• output/monthly/IONS-2026-02.pptx
• output/monthly/IONS-2026-02.pdf
• output/monthly/IONS-2026-Q1.json
• output/monthly/IONS-2026-Q1.pptx
• output/monthly/IONS-2026-Q1.pdf
• @packages/pipeline/atlas.db

2026-02-23 18:14 - T-005

Overview: Validated deprecated run alias behavior and warning output.

Completed:

• Ran pnpm run pipeline -- run --entity IONS --year 2026
• Confirmed warning output: [pipeline] `run` is deprecated. Use `sync` instead.
• Confirmed command still executed sync stages successfully

Files:

• @packages/pipeline/atlas.db

2026-02-23 18:15 - T-006

Overview: Performed metadata and artifact verification checks across DuckDB and generated report outputs.

Completed:

• Validated output report JSON files parse correctly
• Confirmed expected PPTX/PDF artifacts exist for monthly and quarterly runs

Blocked:

• _pipeline_runs is missing load, validate, and publish stage records, and row_count is not populated for report/sync runs.

Files:

• @packages/pipeline/atlas.db
• output/monthly/IONS-2026-02.json
• output/monthly/IONS-2026-02.pptx
• output/monthly/IONS-2026-02.pdf
• output/monthly/IONS-2026-Q1.json
• output/monthly/IONS-2026-Q1.pptx
• output/monthly/IONS-2026-Q1.pdf

2026-02-23 19:12 - T-002

Overview: Implemented scoped sync refresh behavior and canonical DuckDB path resolution, then re-ran yearly sync checks.

Completed:

• Added scoped refresh mode support to loader (sync now defaults to refresh)
• Centralized project root and DuckDB path resolution (@packages/pipeline/@source/paths.ts)
• Verified sequential sync runs retain historical years in int_orders
• Verified rerun idempotency for 2024 year scope (no growth in _atlas_source_year = 2024 rows)

Decisions:

• Keep replace mode available for full rebuild workflows; use scoped refresh as sync default for cumulative history.

Files:

• @packages/pipeline/@source/paths.ts
• @packages/pipeline/@source/index.ts
• @packages/pipeline/@source/load/csv.ts
• @packages/pipeline/@source/commands/sync.ts
• @packages/pipeline/@source/cli.ts
• @packages/pipeline/@source/config.ts
• @packages/pipeline/@source/commands/transform.ts
• @packages/pipeline/@source/commands/report.ts
• @packages/pipeline/@source/commands/publish.ts
• @packages/pipeline/@source/format/report.ts
• @packages/pipeline/@source/clean/report.ts
• atlas.db

2026-02-23 19:13 - T-003

Overview: Re-verified publish metadata after DuckDB path unification and canonical metadata writes.

Completed:

• Confirmed publish stage records are written to root atlas.db _pipeline_runs
• Confirmed publish records include row_count

Files:

• atlas.db

2026-02-23 19:20 - T-006

Overview: Completed metadata verification by instrumenting missing stage tracking (load, validate) and re-running sync/report checks.

Completed:

• Added _pipeline_runs coverage for load and validate stages during sync
• Added row_count metadata for load, validate, and report runs (plus explicit 0 for extract/transform)
• Verified latest records include stage, status, timestamps, and row_count across extract, load, validate, transform, publish, report
• Re-validated output artifacts (IONS-2026-02.json/.pptx/.pdf)

Files:

• @packages/pipeline/@source/commands/sync.ts
• @packages/pipeline/@source/commands/report.ts
• @packages/pipeline/@source/commands/transform.ts
• @packages/pipeline/@source/load/csv.ts
• atlas.db
• output/monthly/IONS-2026-02.json
• output/monthly/IONS-2026-02.pptx
• output/monthly/IONS-2026-02.pdf

2026-02-23 19:22 - T-007

Overview: Updated project state doc to reflect Plan 012 workflow unification and current Plan 013 execution status.

Completed:

• Updated state header metadata (Last Updated, Updated By)
• Updated current phase and key activities to workflow unification context
• Updated @packages/pipeline package summary and pipeline stages table for unified commands
• Added Plan 012 entry to recent completed work
• Added key decisions for frequency-based CLI, metadata tracking, and staleness policy

Files:

• @plan/state.md

2026-02-23 19:24 - T-008

Overview: Updated workflow reference to use unified command surface and type-based report examples.

Completed:

• Updated seed command from sync:seed to seed
• Updated quick reference to seed → sync → publish → report
• Added report command examples with --type monthly|quarterly|yearly

Files:

• @plan/workflow.md

2026-02-23 19:25 - T-009

Overview: Simplified quickstart full pipeline instructions to four unified commands and refreshed plan list.

Completed:

• Replaced legacy 7-step flow with 4-step seed → sync → publish → report
• Added quarterly/yearly report examples via --type
• Added Plan 012 to plan list

Files:

• @plan/quickstart.md

2026-02-23 19:34 - T-010

Overview: Deployed schema and data to Dev Turso using .env.dev, including publish compatibility fixes for remote LibSQL behavior.

Completed:

• Pushed schema to Dev via drizzle-kit push --force
• Seeded Dev lookups via pnpm run seed -- --entity IONS
• Published all target months to Dev (2025-06/07/10/11/12, 2026-01/02/03)
• Verified Dev data presence via direct Turso queries (commerce_order and finance_transaction counts by month)

Learnings:

• Remote libsql transport did not keep explicit BEGIN/COMMIT state reliably for this publish flow; publish now conditionally skips explicit transactions for libsql:// URLs.
• Source payment status values include bare ANGSURAN; publish resolver now normalizes ANGSUR/ANGSURAN to seeded lookup key ANGSURAN / ANGSUR.
• Dev endpoint intermittently returned transient ECONNRESET/502; retrying succeeded without data corruption.

Files:

• @packages/pipeline/@source/commands/publish.ts

2026-02-23 19:41 - T-011

Overview: Deployed schema and data to Prod Turso with the same migration/publish sequence used in Dev and verified resulting counts.

Completed:

• Pushed schema to Prod via drizzle-kit push --force
• Seeded Prod lookups via pnpm run seed -- --entity IONS
• Published all target months to Prod (2025-06/07/10/11/12, 2026-01/02/03)
• Verified Prod row counts by month for both commerce_order and finance_transaction

Operational notes:

• Publish retries were required for transient remote libsql errors (ECONNRESET/502), but all target months completed successfully.
• Month 2026-02 intentionally skipped 2 rows with unknown unit WLC, consistent with local/dev behavior.

Files:

• @services/api/.env.prod