Player-coach technical range
Two products, built and largely launched solo: a healthcare claims-appeals SaaS platform (Project A) and a healthcare payer-rate data platform (Project B). Every claim below indexes to a specific decision or build in one of the two.
the sharpest example, per category
-
01 · product
Refused to ship a member-facing outcomes feature with placeholder statistics, holding it dark behind a hard, multi-part data and counsel threshold instead. (Project A)
-
02 · architecture
Built the AI-facing payload as a whitelist-constructed object holding only codes and IDs — narrative text is structurally excluded before it can reach a prompt, not filtered after the fact. (Project A — reasoning service's PHI firewall)
-
03 · dev craft
Found that a third-party SQL parser's traversal missed window specifications and DISTINCT ON clauses entirely, then added a second, unconditional sweep instead of trusting the library's documented API. (Project B — query-safety layer)
-
04 · devops
Root-caused a Terraform pipeline where an error-suppression flag combined with lost exit codes let failed plans report success and silently skip applies — then rebuilt the workflow to derive change counts straight from plan output. (Project A)
-
05 · AI/LLM
Named a failure mode specific to LLM-backed apps: every backend layer correctly reflected a state change within seconds, but the model kept asserting the old state because it anchored on its own prior turns — invisible to every database-level guard and every test that only asserts on rows. (Project B)
-
06 · data
Retired a disk-exhaustion failure class structurally by moving raw-volume staging out of the database into an on-task columnar pre-aggregation step, so database load scales with deduplicated volume instead of raw volume. (Project B — capture pipeline)
-
07 · security
Commissioned an independent security review before any real protected health data entered the system, and kept the system restricted to synthetic data until its top-severity finding closed. (Project A)
-
08 · incident
Deliberately opened a locked-down service's public perimeter for roughly nine minutes to measure real traffic at the API level, then treated the clean result as proof the mechanism works — not as proof it's safe to leave open. (Project B)
judgment calls, in sequence
- Early build (A): chose a fully deterministic, rule-based eligibility engine with zero LLM input by design, and tested that the resulting decision is never included in anything sent to the model.
- Pre-launch hardening (A): commissioned an independent security review and kept the system on synthetic data until its top-severity finding closed.
- Pre-launch hardening (A): found a send channel had been silently reporting false success with no backing infrastructure; chose to fail loudly rather than quietly patch around it.
- Launch week (A): ran a staged four-gate launch-readiness review, each gate requiring a full runbook verified in order before the next.
- Post-launch scale-up (A): cut a GPU-backed compute service over to CPU-only infrastructure without changing the surrounding queue interface, so the swap could be tested and rolled back independently.
- Early build (B): split the data model into a small served code set and a much larger capture-only set, preserving raw evidence without paying serving costs for it.
- Pre-launch hardening (B): inverted a SQL-safety control from denylist to allowlist after measuring that the denylist missed most of a dangerous function family.
- Launch week (B): deliberately opened a locked-down service's public perimeter for roughly nine minutes as an instrumented experiment, then treated the clean result as proof of mechanism, not proof of safety.
- Post-launch scale-up (B): root-caused a model-anchoring bug where every backend layer reflected a state change correctly but the model kept asserting the old state from its own prior turns.
- Post-launch scale-up (B): retired a disk-exhaustion failure class structurally by moving raw-volume staging out of the database into an on-task pre-aggregation step.
Product Management & Judgment Calls
-
Staged launch-readiness gates with quantified exit criteria
Ran a formal, staged launch-readiness sequence from Gate A through Gate D, in which a full-system review produced a severity-ranked P0/P1/P2 blocker list before any launch decision, and the final gate required all nine steps of a rewritten flag-flip runbook to be verified in order. (Project A -- the internal launch-readiness tracker's gate reviews)
-
Refused to ship fabricated efficacy statistics
Kept an aggregate-outcomes feature dark behind a hard, multi-part threshold on data and counsel review -- minimum recorded outcomes, minimum processed volume, minimum live duration, reviewed formats -- rather than launch it with placeholder numbers. (Project A -- the product roadmap's deferred-features table)
-
Sequenced a 9-to-157 product-vocabulary expansion as three parallel, flag-gated tracks
Chose three parallel, flag-gated tracks over one big-bang migration, explicitly deferring monetization of any new category until its own data-serving wave had actually landed. (Project B -- the backlog-expansion roadmap's track map)
-
Used a cost-ratio threshold, not a fixed number, as the launch gate
Modeled projected usage against a spend cap and set the trigger for raising that cap as a multiple of the cap itself, re-derived from freshly measured production costs rather than the original estimate. (Project B -- the platform roadmap's cost-gate entry)
-
Ran a post-launch acceptance walk and converted its findings into a process fix, not just a bug fix
After shipping an entitlements and paywall feature, found a severity-high defect where a purchase unlock didn't take effect inside the conversation that sold it, and recorded the real lesson -- that the acceptance walk happened after deploy -- as the process to change going forward, not just the defect to patch. (Project B -- the web product's post-launch review and roadmap entry)
show 11 more
-
Distinguished "remove the lie" from "restore the capability"
When a fax-send channel was falsely reporting success with no backing infrastructure behind it, chose to fail closed immediately by returning a 409 error rather than wait for the full capability to be rebuilt, while treating that fix explicitly as a partial fix, not a resolution. (Project A -- the known-issues tracker's P0 writeup on the send channel)
-
Decoupled build order from launch order via dark-shipping
Standardized on building features fully behind flags so engineering throughput wasn't gated by business launch timing, with each feature carrying its own later go-live checklist. (Project A -- the operations log's design-day wave notes)
-
Roadmap pivot driven by market research synthesis
Restructured the near-term roadmap around a proactive exception-handling pathway after webinar and market research showed a federally available option was used by only a small minority of eligible professionals, reframing the product from a reactive tool into a proactive access platform. (Project A -- the product team's feature-roadmap document)
-
Codified anti-fabrication policy as an engineering control
Converted a no-fake-stats, no-fake-testimonials, no-fake-compliance-claims content policy into CI-enforced (GitHub Actions), mutation-tested test gates, replicated identically across three separate codebases (Python and TypeScript), after one banned statistic had already reached production copy. (Project A -- the known-issues tracker's content-policy CI section)
-
Build-vs-buy on regulated logistics
Chose vendor integrations for certified mail and fax transmission over building physical-mail and fax infrastructure in-house, keeping engineering focus on the core extraction, reasoning (Anthropic's Claude API), and letter-assembly pipeline. (Project A -- the letter-and-send subsystem notes)
-
KPI-first feature sequencing
Every roadmap item in a major feature wave carried an explicit dependency graph and a named success metric before implementation began. (Project A -- the feature-roadmap document's sequencing-and-measurement section)
-
Treated a proven capability as a business decision, not an engineering task
Deliberately exercised a public-perimeter flip -- opening it, measuring live traffic via AWS CloudTrail and AWS WAF logs for about nine minutes, then reverting -- to prove the mechanism worked, then explicitly refused to conflate proven-ready with should-ship, leaving the actual go/no-go to a stakeholder call. (Project B -- the infrastructure log's perimeter-exposure writeup)
-
Gated a scope expansion behind an unrun measurement pass
Rather than estimating capacity needs up front, required a data-quality and volume scan (DuckDB) of the target codes before sizing database growth (PostgreSQL), verification floors, or wave order. (Project B -- the backlog-expansion roadmap's measurement-gate section)
-
Pre-emptively red-teamed and disabled a legally sensitive feature pending counsel review
Rather than shipping the feature and hoping for the best, disabled it ahead of launch and tracked the decision as a named, dated item pending legal sign-off. (Project B -- the backlog-expansion roadmap's legal-decision log)
-
Enforced a legal/ToU gate by structural absence rather than a feature flag
Built one payer's data pipeline (Python) with no extraction path at all, rather than a disable switch, so the restriction can't be silently re-enabled by a config change. (Project B -- the payer-configuration reference and roadmap's legal-gates section)
-
Named and tracked "honestly deferred" scope at each ship boundary
Tracked upload support, per-row read state, and a backend rate-history ledger (PostgreSQL) as explicit, named backlog rather than quietly cutting them, so stakeholders always knew what a release did and didn't include. (Project B -- the platform roadmap and web product's honest-limits notes)
Software Architecture
-
PHI firewall built as a data-shape contract, not a filter
The Python extraction pipeline's LLM-facing payload is a whitelist-constructed object holding only codes, enums, and IDs — narrative text is structurally excluded before it can ever reach a prompt sent to Anthropic's Claude API, backed by a second-layer scanner as defense in depth. (Project A — the reasoning service's PHI firewall)
-
Split the domain into two aggregates joined loosely, not by foreign key
Modeled the system as a forward-only document-pipeline state machine and a separate 14-state user-facing case aggregate in PostgreSQL, connected by a plain reference so the two could evolve and scale independently. (Project A — the data model's two-record-type design)
-
Enforced row-level entitlements by rewriting the SQL AST, not by filtering in application code
Model-authored SQL is parsed (via node-sql-parser, in TypeScript) and every gated table reference is replaced with a pre-filtered subquery before execution, so a locked resource can't come back even if it's named directly in the query. (Project B — the web layer's SQL entitlement fence)
-
Separated the served product from the raw option value in the data model itself
A small served code set is the only thing loaded into the operational PostgreSQL database, while a much larger capture-only code set streams straight to S3, preserving unrecoverable raw evidence without paying serving costs for it. (Project B — the rate-extraction pipeline's code-set split)
-
Ran two independent provider migrations without touching the surrounding interface
Cut over the extraction LLM provider to Anthropic's Claude API, then separately cut the GPU compute over to CPU (on AWS ECS Fargate), in both cases leaving the queue/worker interface untouched so each swap could be tested and rolled back on its own. (Project A — the extraction worker service's provider-cutover history)
show 11 more
-
Replaced hardcoded per-letter rendering with a declarative registry
A spec/style/format-version registry, implemented in Python, lets one assembly walk produce four distinct letter kinds, and adding a new kind requires no changes to the render path itself. (Project A — the letter-formatting subsystem's registry design)
-
Resolved a recurring CD-vs-infrastructure-as-code conflict by giving each system a single role
Made the deploy pipeline (GitHub Actions) the sole writer of the current image pointer, held in AWS SSM Parameter Store, and made the infrastructure-as-code layer (Terraform) a reader only — removing the race between deploy-driven and infra-driven image changes. (Project A — the infrastructure module's image-pointer wiring)
-
Designed the AI-output schema so evolution can't silently break old consumers
Every new field on the reasoning service's output contract is optional and validated with strict extra-field rejection (Pydantic, in Python), letting old and new payload shapes coexist and be tested for parity mechanically. (Project A — the reasoning service's output-schema versioning)
-
Pinned the rules engine to the document's date, not today's date
The statutory-deadline engine, written in Python, resolves rule "vintage" from the denial letter's own date rather than the current date, so historical determinations stay reproducible even as the underlying rule corpus keeps getting updated. (Project A — the reasoning service's statutory-deadline engine)
-
Scoped an exception to a strict invariant as narrowly as possible instead of weakening the rule
Added exactly one gated backward transition to an otherwise strictly forward-only state machine, for crash recovery only, bounded by a PostgreSQL transaction-local flag rather than a general relaxation of the invariant. (Project A — the case data model's migration history)
-
Split retention policy from the data model so history survives independently of raw storage limits
One PostgreSQL fact table is retention-pruned for cost, while a separate percentile-aggregate table, refreshed transactionally alongside it, is never pruned — historical trends survive regardless of the raw-data retention window. (Project B — the rate-extraction pipeline's permanent aggregate table)
-
Built a repeatable privilege-bracket procedure for schema changes against a database with no direct network path
A standing, audited procedure temporarily re-points IAM credentials, runs the migration (Alembic), then reverts access — used across multiple schema releases in the absence of a bastion host. (Project B — the infrastructure runbook's migration privilege-bracket procedure)
-
Mapped database access pools to blast radius, not convenience
A row-grain "drill" tool runs on a separate, least-privileged connection pool from aggregate-only tools, with the boundary enforced as a PostgreSQL grant rather than application logic. (Project B — the web layer's three-pool database access design)
-
Serve four independently-versioned services from one container image
Each of four AWS ECS task definitions is pinned to a different tag of the same container image in AWS ECR, so any one service can roll forward without moving the other three. (Project B — the capture pipeline's container/task-definition design)
-
Turned cross-repo content drift into a build failure instead of a silent divergence
A large structured content corpus lives as version-controlled config in one repo, syncs idempotently into the PostgreSQL database via a GitHub Actions pipeline, and exports as a checksummed artifact consumed by a second repo, so the two can't quietly drift apart. (Project B — the channels subsystem's content-corpus sync pipeline)
-
Split ingestion into fetch and load stages connected by cloud-native events
Fetch and load run as separate services (Python, on AWS ECS Fargate) chained by state-change events, so a fetch failure never loses already-fetched data and a load never blocks on network I/O. (Project B — the pipeline's ingestion-architecture design)
Development Craft
-
Root-caused a silent whole-pipeline classification bug.
Traced every production document being misclassified back to a read-path mismatch between what the Python extraction pipeline produced and what the rules layer expected, then fixed it with a canonicalizing seam robust to model field-misassignment. (Project A -- the reasoning service's read-path classification fix)
-
Found a CI gap that hid a payment-critical defect for five days.
Discovered that a missing pytest marker was silently deselecting 20 golden tests, masking a checkout defect, and fixed the underlying testing-infrastructure gap rather than just the one test. (Project A -- an internal engineering-lessons log entry)
-
Found and closed a traversal blind spot in a third-party SQL parser library (node-sql-parser, in TypeScript).
Reproduced the gap directly -- window specifications and DISTINCT ON clauses were invisible to the library's AST visitor -- then added a second, unconditional defense-in-depth sweep rather than trusting the library's documented API. (Project B -- the platform's SQL guard module)
-
Inverted a security control from denylist to allowlist based on measurement, not intuition.
Discovered the existing denylist missed most of a dangerous function family, and that Postgres exposes an SQL-injection-capable function no static list could safely enumerate, then rebuilt the control around an allowlist with categorized function families and a test asserting the two sets never overlap. (Project B -- the platform's SQL guard module)
-
Identified and named a new class of bug specific to LLM-backed applications.
Every backend layer -- resolver, access-control fence, audit log -- was correct and had flipped state within seconds of an entitlement change, yet the Claude API model kept asserting the stale fact because it anchored on its own prior conversation turns, a defect invisible to the fence, the guard, every grant, and every test that asserts on PostgreSQL rows. (Project B -- an entitlement-state incident writeup)
show 10 more
-
Wrote property-based regression coverage for a cross-cutting invariant.
Wrote a pytest test asserting, across the full state-by-plan-class matrix, that a regional rule can never make an outcome worse than a federal baseline, protecting the invariant permanently rather than pinning individual cases. (Project A -- the reasoning service's statutory decision-due test suite)
-
Built a purpose-built PII/PHI scrubber and iterated it against real false positives.
Wrote a fail-closed, regex-based PHI scanner in Python, then iteratively tuned a curated allowlist after production data revealed benign terms -- agency names, payer names -- tripping it. (Project A -- the reasoning service's PHI scanner)
-
Matched test-suite architecture to CI cost and signal tradeoffs.
Separated unit, integration (Docker-backed), compliance, and live-provider smoke suites so GitHub Actions could run the cheap, deterministic majority on bare runners and reserve container-backed runs for what actually needed them. (Project A -- the claims-processing service's test suite and CI configuration)
-
Built image supply-chain hygiene into CI.
Enforced per-service Docker image size budgets, a production-image import check to catch dev-only dependencies leaking into prod images, and vulnerability scans (Trivy) on every build. (Project A -- the claims-processing service's CI pipeline configuration)
-
Diagnosed an unrecoverable CI-only browser-automation flake.
Identified that a test framework's form-fill call could leave React's internal value tracker out of sync with the DOM, meaning the same failing input could never be fixed by simply retrying. (Project A -- an internal engineering-lessons log entry)
-
Designed CI triggers around a GitHub Actions limitation.
Deliberately made a required cross-repo consistency check run unconditionally, rather than path-filtered, after learning that GitHub Actions never reports status for a skipped path-filtered required check -- which would otherwise block pull requests forever. (Project A -- the claims-processing service's cross-repo consistency CI workflow)
-
Diagnosed a CI-cost defect down to a specific third-party GitHub Action's quantifier semantics.
A path filter meant to skip an expensive job on unrelated pull requests was actually firing on every one, and its exclusion line never worked either, because of how the underlying pattern-matcher combines patterns; fixed by re-deriving the filter as positive enumeration and pinning the fix with a test that compiles the GitHub Actions workflow file itself. (Project B -- the web layer's CI eval-workflow configuration)
-
Root-caused a resource-leak class in query cancellation.
Measured that destroying a database client on cancel leaves the PostgreSQL backend connection permanently active -- never idle in transaction -- so a timeout-based reaper could never reclaim it, and fixed it by closing the read cursor instead of the connection. (Project B -- the query-execution and cancellation layer)
-
Built cache-cost math directly from the Claude API's documented mechanics.
Derived a per-round tool-call cap from the Claude API's cache-lookback block limit via an explicit inequality, rather than picking a number empirically. (Project B -- the chat engine's tool-call configuration)
-
Caught accessibility defects invisible to the full automated test suite.
Manual review found a fake aria-modal attribute, a silent success state with no live region, and a roving-tabindex bug that a click-driven test suite structurally cannot see, and used the findings to argue for accessibility-tree assertions as a first-class test category. (Project B -- a manual accessibility review of the web product)
DevOps / Infrastructure-as-Code / AWS
-
"Merge is the apply" as an explicit governance model
Ran the Terraform infrastructure repository, applied via GitHub Actions, with no separate approval step between merge and production application, compensating with a strict rule holding every flag-flip pull request unmerged until a deliberate go decision, tracked per-PR in a shared status ledger. (Project A -- the infrastructure repo's merge policy)
-
Found and fixed a silent CI false-negative
Root-caused a Terraform pipeline bug in a GitHub Actions workflow where an error-suppression flag combined with a shell-pipeline exit-code loss caused failed plans to report success and skip applies, then rebuilt the workflow as a single GitHub Actions job that derives change counts directly from Terraform plan output. (Project A -- the infrastructure CI pipeline's apply workflow)
-
Measured a live security change at the API rather than trusting the merge timestamp
Used AWS CloudTrail logs on the specific security group, AWS Application Load Balancer (ALB) request logs, and AWS WAF logs to prove the actual open/close window and exactly what traffic arrived during a perimeter test, instead of reporting duration from pull-request history. (Project B -- the platform's network-perimeter verification)
-
Found and documented a silent Terraform footgun
A Terraform `replace()` function call used to template per-payer schedules returns its input unchanged when the search string doesn't match, so adding or reordering a value silently converts six isolated, payer-scoped jobs into six duplicate full-fleet jobs with no Terraform plan-output signal; tracked as a named, numbered finding with a standing do-not-touch rule during freeze windows. (Project B -- the data-capture scheduling infrastructure's landmines log)
-
Built a deploy-verification primitive independent of "green CI"
Added a single header on one unauthenticated health endpoint that carries the exact deployed source commit, built specifically because "merged" and "deployed" had drifted apart multiple times across AWS ECS release waves. (Project B -- the platform's health-check endpoint)
show 10 more
-
Sequenced a live capacity-provider migration around a discovered ordering bug
Migrated a GPU-backed compute service to CPU-based serverless containers (AWS ECS Fargate) in two separate Terraform applies after finding that removing an AWS ECS capacity provider and its cluster association in one apply times out; fixed by disassociating first. (Project A -- the GPU-worker service's compute-provider cutover)
-
Closed an apply/deploy divergence that had orphaned revisions for weeks
Diagnosed that a Terraform lifecycle ignore-changes setting let new AWS ECS task-definition revisions get registered that the running service never adopted, then removed it fleet-wide so a Terraform apply and a deploy became the same guaranteed event. (Project A -- the infrastructure's task-definition ownership model)
-
CD hardened against two independent failure classes
Added a boot-smoke run of every new Docker image before swapping a live AWS ECS service, catching a bad build before it serves traffic, and an AWS ECR immutable-tag probe before rebuilding, fixing a rerun failure mode. (Project A -- the platform's CD and migration safeguards)
-
Cost-engineering tradeoffs made and documented as explicit levers
Ran a single-NAT-gateway AWS VPC network topology and a scale-to-zero AWS ECS Fargate posture as deliberate pre-revenue cost cuts, then later produced a rigorously verified scale-to-zero design for the most compute-heavy service by soliciting multiple independent design drafts and fact-checking every load-bearing AWS-behavior claim against primary AWS documentation before synthesizing a final approach. (Project A -- the network topology and worker scale-to-zero design)
-
Plan-time preconditions to prevent crash-looping services
Added Terraform preconditions that fail a plan, rather than a live boot, when a feature flag would be enabled without its required configuration present, and chose non-empty default string values specifically to avoid an empty-string enum crash at boot. (Project A -- the infrastructure's feature-flag plan preconditions)
-
Removed a security-sensitive CI artifact
Stopped publishing the binary Terraform plan file as a GitHub Actions CI artifact after recognizing it embeds refreshed state, including secrets, in plaintext, replacing it with a masked text summary. (Project A -- the infrastructure CI pipeline's plan-artifact handling)
-
"Merge is the apply" CI/CD discipline enforced consistently
Ran the same merge-is-the-apply convention across the platform's Terraform infrastructure, applied via GitHub Actions, paired with a documented exception where the marketing stack deliberately uses plan-on-pull-request plus manual apply, so operators never assume one convention applies everywhere. (Project B -- the infrastructure repo's CI convention and its documented exception)
-
Designed for cost via architecture, not tuning
Used a scale-to-zero-by-default posture with the AWS Application Load Balancer (ALB) as the one standing exception, and public-subnet AWS ECS Fargate tasks with no NAT gateway as a deliberate cost/security tradeoff, documented as such rather than left implicit. (Project B -- the platform's network and compute topology)
-
Declared and observed formal change freezes around first-ever scheduled batch windows
Coordinated freezes spanning multiple independent AWS ECS services (capture, extraction, verification), explicitly reasoning about which changes were in or out of scope for each freeze. (Project B -- the platform's operational change-freeze log)
-
Ran a repeatable, four-times-executed credential-elevation runbook
Used a standing procedure of temporarily re-pointing credentials, running the Alembic-managed migration, then reverting, against a PostgreSQL database with no administrative access path, with each of the four executions timed, verified, and reverted, and separately flagged the one component never brought into the pattern as an open critical finding rather than assuming coverage. (Project B -- the database migration privilege-elevation procedure)
AI/LLM Engineering
-
Live incident-driven model/provider fallback
When a third-party inference vendor silently retired the pinned extraction model, the extraction stage — a Python service — was re-routed to a different provider through an OpenAI-compatible bridge within the same release cycle, uncovering and working around an undocumented schema-strictness incompatibility along the way. (Project A — the GPU worker service's LLM-extraction stage)
-
Deterministic-first design as a stated safety invariant
The eligibility and routing decision engine was built as pure rule-based Python code with zero LLM input by design, with a dedicated pytest test proving the resulting decision object is never included in anything sent to the model. (Project A — the pathway-resolution engine and the reasoning service's rule-output construction)
-
Hallucination guardrails on every LLM call
Every response from Anthropic's Claude API is validated against the exact set of source citations and required output items sent in the prompt, rejecting phantom citations or incomplete or added items, with failure reasons restricted to a closed enum rather than free text. (Project A — the reasoning service's output-validation step)
-
Two independent eval harnesses, both scored against the live model
One Python harness scores structured-extraction accuracy — triage and extraction field correctness, pass@3 — and a separate Python harness scores tool-use correctness, covering answer accuracy plus cache-hit proof and zero-unauthorized-write assertions, kept as documented separate systems because they are easy to conflate. (Project B — the messaging-channel and web-chat eval suites)
-
Built row-level entitlement enforcement as SQL rewriting rather than prompting
The model may name any allowed resource in generated SQL, but a TypeScript rewriting layer injects the actual access predicate before execution, so a compliant answer never depends on the model choosing to behave. (Project B — the web layer's AST predicate-injection fence)
show 11 more
-
Sub-agent-orchestrated design synthesis with a verification pass
A Terraform infrastructure design was produced by collecting multiple independently generated draft proposals and running a judging pass that fact-checked every load-bearing technical claim against primary sources before scoring and merging the strongest elements of each into a final design. (Project A — the worker scale-to-zero design process)
-
Eval harness spanning PR-time and nightly cadence
A curated gold-set of denial-classification cases runs on every PR via GitHub Actions without calling an LLM, a smaller LLM-backed eval subset using Anthropic's Claude API runs on PRs, and a full Claude API eval runs nightly, with an automatic graceful skip when no eval credentials are configured. (Project A — the gold-set test suite and CI / nightly-eval workflows)
-
TDD applied to AI-authored/AI-adjacent code
CI-enforced pytest tests, run through GitHub Actions, pin exact hallucination-guardrail behaviors, exact PHI-firewall exclusions, and exact fallback taxonomies, written before or alongside the corresponding production code paths. (Project A — the reasoning service's test suite)
-
Confidence-scored classification with an audited fallback path
The denial classifier prefers a corpus-matched code and only falls back to a category inferred by Anthropic's Claude API at a deliberately lowered confidence score set just under the human-review threshold, tracking which path produced each result for audit. (Project A — the reasoning service's denial-classification step)
-
Licensing-aware content guardrail
Member-facing procedure descriptions are restricted to a curated, human-reviewed, checksum-pinned glossary stored in PostgreSQL rather than any verbatim licensed descriptor text the model might otherwise echo. (Project A — the reasoning service's curated procedure-translation table)
-
A streaming tool-use loop with two distinct prompt-cache breakpoints
One prompt-cache breakpoint in the calls to Anthropic's Claude API sits at a byte-static system prefix and the other moves per round, with an explicit rule that the static prefix may never interpolate a date or id, because doing so would silently invalidate the cache for every user. (Project B — the web layer's chat engine)
-
A cheap-model-gates-expensive-model extraction pipeline
A fast triage pass, built in Python, filters most incoming content before an expensive structured-extraction pass using Anthropic's Claude API ever runs, with a separate dedupe-judgment pass to avoid creating duplicate structured records from the same underlying event. (Project B — the messaging-channel ingestion pipeline)
-
Guardrail against a real hallucination-adjacent failure mode
A React charting component refuses to draw rather than mislabel when the underlying data mixes incompatible units, added after a live incident where a percentage value was drawn on the same axis as a dollar figure. (Project B — the web layer's chart-rendering component)
-
A dual-persona voice system implemented entirely through prompt text
The two-persona voice runs with no routing code path at all, deliberately choosing self-selection by Anthropic's Claude API over an explicit branch so the single cached system prompt stays intact. (Project B — the persona-refresh and cadence design)
-
Diagnosed a model-behavior defect distinct from a backend defect
After an entitlement changed mid-conversation, every backend layer correctly reflected the new state, but Anthropic's Claude API kept asserting the old state because it anchored on its own prior turns; fixed with an explicit directive that the current context is authoritative, engineered as a pure function with zero extra PostgreSQL reads. (Project B — the web layer's entitlement-and-chat integration)
-
Designed the teaser answer path to be computed by deterministic SQL rather than by the model
The teaser path, computed directly in PostgreSQL, is asserted by a pytest test to contain none of the values that would make an incomplete answer look complete, keeping unentitled or rejected interactions cheap relative to full tool-using answers. (Project B — the product's locked-turn response pipeline)
Data Engineering at Scale
-
Retired an entire failure class with a structural fix instead of a workaround.
Moved raw-volume-sized staging out of the database (PostgreSQL) and into an on-task columnar pre-aggregation step (DuckDB), so database staging scales with deduplicated volume instead of raw volume — closing a disk-exhaustion failure that had previously killed a production load outright. (Project B -- the rate-extraction pipeline's DuckDB pre-aggregation stage)
-
Demonstrated dedupe and collapse ratios at production scale.
One payer's raw feed collapsed roughly 9-20x through pre-aggregation, and a single payer's monthly raw volume of about 200 million rows reduced to a multi-million-row deduplicated serving grain. (Project B -- rate-extraction pipeline pre-aggregation metrics and the July 2026 serving-state report)
-
Designed ingestion to survive task termination instead of restarting from zero.
For multi-hour fetches, each source file uploads to S3 as soon as it completes rather than at run end, and a restart lists what has already been banked and fetches only the gap — turning a killed multi-day fetch into a cheap top-up instead of a full re-run. (Project B -- ingestion pipeline's per-file upload and resume design)
-
Built automated, per-source verification that catches a missing feed on its own.
A read-only monthly job, written in Python, asserts that every expected data source landed above a floor, with the floors candidly documented as rough placeholder estimates to be recalibrated once real levels are known, rather than presented as already tuned. (Project B -- the pipeline's serving-completeness verification job)
-
Built a living dataset catalog that ties schema to business questions.
Cross-references every measurable field against multiple analytical lenses and maps each to a persona-grounded SQL query library, giving a self-service trace from raw schema to a product-facing question. (Project B -- the pipeline's dataset catalog and query-library documentation)
show 2 more
-
Derived data-quality heuristics from real per-payer data instead of one global rule.
Built dead-man checks in Python for a structural anomaly class (plan-grain collapse or explosion) with per-payer exceptions, each justified from an actual measured ratio against that payer's real monthly data rather than a single global threshold. (Project B -- the pipeline design's observability and dead-man-check specification)
-
Kept a data-quality signal advisory rather than using it to silently drop rows.
Computed a suspicious-provider-share metric in Python and enforced, in code comments and schema documentation, that it exists to inform downstream judgment rather than to filter automatically — preventing silent data loss from a heuristic. (Project B -- rate-extraction pipeline's serving-state documentation)
Security & Compliance
-
Commissioned an independent security review before any real patient data entered the system.
Static-code and live AWS cloud-posture audits were deduplicated into a single severity-ranked remediation roadmap checked against standard HIPAA and CIS frameworks, and the system stayed restricted to synthetic data until the top finding closed. (Project A -- the pre-launch security audit and remediation roadmap)
-
Enforced an audit log's immutability as a PostgreSQL grant, proven by a failing test, not a coding convention.
The append-only event table's write role has update and delete revoked at the PostgreSQL level, and the pytest suite runs under that restricted role and asserts the exact permission-denied error, so a superuser test run fails loudly instead of passing without proving anything. (Project B -- database-level grant restrictions on the append-only event log)
-
Made legal and compliance gates mechanically enforced in CI rather than doc-reviewed.
A payer's public-listing clearance, a banned-claims list, and a rule against showing certain brand and data combinations together all fail the GitHub Actions build if violated, checked against the live content graph on every build rather than relying on manual review before publish. (Project B -- marketing-site content-graph build gates)
-
Surfaced and documented a latent multi-tenancy gap rather than leaving it as an unstated assumption.
Found that both authentication layers hardcoded a single tenant value, so apparent tenant isolation was a no-op and real isolation depended entirely on per-user ownership filters, and wrote that down explicitly instead of leaving it implicit. (Project A -- data-model tenancy documentation)
-
Proactively catalogued secrets-hygiene gaps as findings to close, not incidents to react to.
Documented that no application secret in the AWS account namespace had rotation enabled and that one credential was shared across two production services, tracked as an open standing risk rather than something that surfaces only after a postmortem. (Project B -- infrastructure secrets-rotation audit)
show 7 more
-
Built PHI-specific data protection into the architecture rather than adding it on afterward.
Used dedicated, rotating AWS KMS-managed encryption keys scoped by data purpose, S3 Object Lock's write-once storage locking on PHI data, and copied an existing service's strong per-user authorization pattern into a service found to lack it. (Project A -- static-scan findings and infrastructure module layout)
-
Tracked vendor risk as a versioned artifact instead of tribal knowledge.
Maintained a machine-readable vendor compliance-agreement status file that was updated the same day a compute-provider swap changed which vendor handled raw PHI egress. (Project A -- vendor compliance-agreement status reference)
-
Built out threat detection incrementally, tied to real alerting rather than dashboards nobody watches.
Stood up AWS GuardDuty and AWS Security Hub threat-detection and security-posture services with severity-based alert routing, plus encrypted, centralized AWS CloudTrail audit logging, as part of a phased security hardening program. (Project A -- static-scan findings, infrastructure hardening section)
-
Engineered right-to-erasure with fail-closed safety properties.
Built a Python data-erasure capability shared by self-service requests and operational hygiene sweeps, requiring a matching read-only dry run before any destructive run, refusing to orphan case-anchored records, and leaving a permanent PHI-free audit trail. (Project A -- data-erasure hygiene-sweep runbook)
-
Used one service's identity design as the reference pattern and prioritized fixing a sibling service that fell short of it.
Fail-closed token verification plus PostgreSQL row-level security (RLS) checks were the established pattern in one service; the same review found a sibling service relying on unguessable identifiers alone, and that gap got prioritized for remediation. (Project A -- static-scan findings and identity subsystem review)
-
Tracked standing-privilege and secrets-hygiene gaps to closure instead of fixing them ad hoc.
Flagged a broad-privilege standing administrative IAM identity and secrets without automatic rotation, scheduling remediation for both as part of the same structured roadmap rather than one-off fixes. (Project A -- security vulnerability roadmap, standing-privilege and secrets items)
-
Layered network defenses by traffic shape rather than by rule count.
Separated volumetric and reputation AWS WAF rules, which were enforced, from content-inspection rules, which were deliberately left observation-only, reasoning explicitly about why a legitimate query payload would otherwise trip generic input-filtering rules. (Project B -- regional WAF access-control configuration)
Incident Response & Operational Maturity
-
Judged fail-closed over silent restoration for a broken channel.
For a document-send channel that had been falsely reporting success, chose to make it fail loudly with an explicit error rather than quietly resurrect a capability that hadn't been properly rebuilt, and wrote up that distinction precisely for the next operator. (Project A -- incident notes on the phantom-send P0 fix)
-
Treated a live perimeter exposure as a controlled, measured experiment.
Briefly opened public ingress on a locked-down service, instrumented it with AWS WAF logs to capture every request and client IP during the window, and used that evidence to distinguish "rehearsable" from "safe under sustained exposure" rather than letting the clean result stand in for the latter. (Project B -- infrastructure notes on the perimeter-exposure test)
-
Self-audited alarm coverage and named the exact gaps.
Reviewed every AWS CloudWatch alarm's "missing data" behavior across the account and found that most of them could never fire if a subsystem went silent, cataloguing the result as a numbered list of concrete holes rather than a general "monitoring could be better." (Project B -- operations notes on the alerting-ladder audit)
-
Rehearsed a risky operational procedure before relying on it live.
Before trusting a new mid-flight worker-drain mechanism in an AWS ECS (Fargate) service in production, ran deliberate rehearsal probes exercising both a graceful-deploy drain and an abrupt stop-task drain to confirm safe recovery under each. (Project A -- worker-service drain rehearsal notes)
-
Freeze/gate discipline under a solo-operator model.
Held every Terraform infrastructure change in a launch wave unmerged until an explicit go decision, applied strictly in a pre-defined order via GitHub Actions, with each step's verification recorded in a shared status ledger before the next step proceeded. (Project A -- launch-wave status ledger)
show 5 more
-
Monitoring caught a real production drift during an active change window.
An out-of-band emergency IAM policy patch made mid-incident correctly triggered its intended AWS CloudWatch alarm; rather than treat the alarm firing as the end of the story, the fix was codified back into the Terraform infrastructure-as-code with a matching runbook update. (Project A -- infrastructure notes on the IAM-drift codification)
-
Root-caused a payment-integrity class of bug, not just the symptom.
Traced a defect that let members pay before discovering their letter wasn't actually sendable to an ordering bug in the Python backend between a paywall check and a readiness check, then fixed the ordering and added a purchase-time guard rather than a point patch. (Project A -- payments subsystem "money integrity" findings)
-
Alarm hygiene as a continuous practice, not a one-time cleanup.
Identified a month-long unacknowledged AWS CloudWatch alarm and a flapping false-positive alarm caused by an internet-facing AWS Application Load Balancer (ALB) left running against a scaled-to-zero backend, then fixed the underlying cause instead of muting the noise. (Project A -- operations log on alarm cleanup)
-
Closed a schema/behavior gap that had already caused a production incident.
After a widened application-level enum without a matching PostgreSQL constraint change broke a live payment webhook, added a dedicated schema-parity pytest test class specifically so that category of gap could not silently recur. (Project A -- data-model notes on the Alembic migration-autogenerate guard)
-
Ran a repeatable credential-elevation runbook as standing operational practice.
For schema migrations against a PostgreSQL database with no administrative access path, executed a temporarily-repoint-credentials (AWS SSM Parameter Store) / migrate / revert procedure four times, each run timed, verified, and reverted, and separately flagged the one component never brought into the pattern as an open finding rather than assuming coverage. (Project B -- infrastructure notes on the migration privilege-bracket procedure)