Ten domains
Every domain scored and evidenced. Each finding is a count or a pass/fail check you can reproduce.
Scale & composition
Covers: Codebase size, duplication, and how much is generated filler versus unique product logic.
Finding: 380K lines across 99 backend modules and 3,474 files — but 40–50% AI-assisted and 20+ duplicated grids inflate the total.
Development practice & provenance
Covers: How code is written and validated — and whether history matches what was reported delivered.
Finding: 40–50% AI-assisted (27–29% documented); commit messages instructed to omit AI attribution — provenance is obscured.
Dependency & lock-in
Covers: Whether the client can obtain and run every critical component without the vendor.
Finding: 63 private vantage/* PHP packages plus @vantage npm across 277 files; the vendor owns the repo, registry and CI/CD image.
Secrets & credential exposure
Covers: Credentials committed to source/history and whether secrets are managed safely.
Finding: Plaintext AWS access keys (prod + staging S3) and a default admin password committed to the repo; no secrets manager.
Test coverage & release control
Covers: Automated test coverage and whether anything gates a release.
Finding: 9 backend tests, 0 frontend, 0 E2E on 380K lines; CI runs only a TypeScript typecheck — nothing gates a release.
Technical debt
Covers: Accumulated shortcuts, lint/TODO density, duplication, and whether debt is tracked and paid down.
Finding: 821 lint problems, 152 TODO/FIXME across 109 files, 20+ duplicated DataGrids — debt untracked and growing.
Feature complexity & knowledge concentration
Covers: Feature breadth and how concentrated system knowledge is across people — the bus factor.
Finding: 17 feature areas built by effectively one developer — bus factor of 1; onboarding docs (CLAUDE.md) target AI, not people.
Infrastructure & recovery
Covers: Infrastructure-as-code, backups/DR, monitoring, and whether recovery has ever been tested.
Finding: No IaC for RDS/Elasticsearch/StarRocks/Kafka, no backup/DR, no monitoring — recovery is undocumented and untested.
Scalability & resilience
Covers: Redundancy and failover on the revenue- and filing-critical path.
Finding: Mission-critical during the live federal education-funding filing window, yet infra is vendor-run with no documented failover or DR.
Transition risk
Covers: How hard and costly it would be to move off the vendor, and whether a path exists.
Finding: A vendor exit needs 8–14 months and vantage/* source access; a costed, staged path now exists where none did.
Findings · evidence attached
- Dependency & lock-in
The backend can't be built or deployed without vendor registries: 63 private vantage/* packages plus @vantage npm.
Impact: If the vendor pauses or departs, Northwind cannot build, deploy or recover its own software — an existential operational risk.
composer.vantagedigital.com: 63 vantage/* packages@vantage: 277 files, 555 importsrepo: bitbucket.org/vantagedigitalCriticalconf 98% - Test coverage & release control
9 backend test files and zero frontend/E2E tests protect 380K lines; CI runs only a typecheck.
Impact: Every change is an unguarded bet on a live production system handling regulatory filings; regressions ship silently.
backend tests: 9frontend/E2E tests: 0CI: typecheck only — no tests, no lintCriticalconf 99% - Secrets & credential exposure
Plaintext AWS access keys (prod + staging S3) and a default admin password were committed to the repo.
Impact: Committed secrets can be extracted from history; each is a live breach vector until rotated and moved to a secrets manager.
2 AWS access keys in sourcedefault admin password in a plan fileno secrets managerHighconf 97% - Infrastructure & recovery
No IaC, backup/DR, or monitoring for RDS, Elasticsearch, StarRocks or Kafka.
Impact: An outage or data loss has no rehearsed recovery path — restoration time is unknown and unbounded.
no Terraform/CloudFormationno backup/DR docsno monitoring/alerting configHighconf 95% - Feature complexity & knowledge concentration
17 feature areas were built by effectively one developer — a bus factor of 1.
Impact: One person leaving would strand most of the system — both delivery and recovery are at risk.
5 CLAUDE.md files (965 lines)bus factor: 1onboarding docs target AI, not humansHighconf 92% - Development practice & provenance
Repeated missed targets (Jul → Sep → Dec 2025) before a filing-window MVP; AI attribution deliberately concealed in commits.
Impact: Velocity outran stabilization; quality issues surface in production during the filing window rather than in review.
missed: Jul, Sep–Oct, Dec 2025CLAUDE.md: omit Co-Authored-ByHighconf 90%