Wallet.Balance is a cache. The ledger is the truth.
Every number a user sees on a balance screen is a projection, kept in lockstep with the ledger by discipline, not by magic. This page is about that discipline: how the projection is kept honest inside one transaction, what the ledger structurally refuses to accept, how to independently verify the two never drifted apart, and — since it did drift once, in a real code path — how to investigate it when it happens.
The core invariant
Wallet.Balance is a cached, denormalized number that exists purely so a balance
read doesn't have to SUM() a ledger table on every request. It is kept correct
by one rule, enforced by convention across every caller: any code that changes a
balance must call ledger.Post and ledger.ApplyToWallet inside the
same database transaction. There is no other path — see the full catalog at
Money Flows, where every one of the 22 operations does exactly
this pairing (two of them, card holds, deliberately don't — below).
The zero-sum invariant
ledger.Post refuses to write anything unless every currency present in the
entry set sums to exactly zero — quoted directly from internal/ledger/service.go:
var (
ErrNoEntries = errors.New("ledger: no entries supplied")
ErrUnbalanced = errors.New("ledger: entries do not sum to zero")
ErrOverdraft = errors.New("ledger: insufficient balance")
ErrZeroAmount = errors.New("ledger: entry amount must not be zero")
ErrMissingInfo = errors.New("ledger: entry missing required field")
)
sums := make(map[string]decimal.Decimal)
for _, e := range entries {
if e.AccountID == 0 || e.Currency == "" {
return nil, fmt.Errorf("%w: account_id and currency are required", ErrMissingInfo)
}
if e.Amount.IsZero() {
return nil, fmt.Errorf("%w: account %d", ErrZeroAmount, e.AccountID)
}
sums[e.Currency] = sums[e.Currency].Add(e.Amount)
}
for currency, sum := range sums {
if !sum.IsZero() {
return nil, fmt.Errorf("%w: currency %s sums to %s", ErrUnbalanced, currency, sum.String())
}
}
A cross-currency operation like FX convert is therefore really two independent zero-sum postings sharing one LedgerTransaction row — see the four-leg example at Money Flows → FX conversion. Rejects with a fully specific message: "ledger: entries do not sum to zero: currency USD sums to 0.13".
Locking discipline
Every distinct AccountID touched by an entry set is locked FOR UPDATE,
in ascending ID order — not the order accounts appear in the entry list. This
is what prevents two concurrent transfers between the same two wallets (in opposite
directions) from deadlocking each other: both transactions always acquire locks in the same
global order.
ids := make([]uint, 0, len(accountIDSet))
for id := range accountIDSet {
ids = append(ids, id)
}
sort.Slice(ids, func(i, j int) bool { return ids[i] < ids[j] })
locked := make(map[uint]*LedgerAccount, len(ids))
for _, id := range ids {
acct, err := s.lockAccount(ctx, tx, id)
if err != nil { return nil, err }
locked[id] = acct
}
// lockAccount locks a ledger account row FOR UPDATE. SQLite has no such
// clause and serializes access at the connection level anyway, so the
// locking clause is skipped there — tests can run against sqlite without a
// real Postgres deployment.
func (s *Service) lockAccount(ctx context.Context, tx *gorm.DB, id uint) (*LedgerAccount, error) {
var acct LedgerAccount
q := tx.WithContext(ctx)
if tx.Dialector != nil && tx.Dialector.Name() != "sqlite" {
q = q.Clauses(clause.Locking{Strength: "UPDATE"})
}
if err := q.Where("id = ?", id).First(&acct).Error; err != nil {
return nil, fmt.Errorf("ledger: lock account %d: %w", id, err)
}
return &acct, nil
}
The dialect check renders real SELECT ... FOR UPDATE on Postgres via GORM's clause.Locking{Strength: "UPDATE"}, and silently no-ops on sqlite — the exact mechanism that lets internal/ledger/service_test.go run the full test suite against an in-memory sqlite DB with no real Postgres deployment.
The overdraft guard
After locking, Post computes every touched account's resulting balance and
rejects the whole posting if any account with AllowNegative = false would go
negative:
for id, delta := range deltas {
acct := locked[id]
current, err := s.accountBalance(ctx, tx, id)
if err != nil { return nil, err }
resulting := current.Add(delta)
if resulting.IsNegative() && !acct.AllowNegative {
return nil, fmt.Errorf("%w: account %d (%s/%s) would go to %s",
ErrOverdraft, id, acct.OwnerType, acct.OwnerID, resulting.String())
}
}
| Owner type | AllowNegative | Why |
|---|---|---|
| User accounts | false | A user wallet cannot be overdrawn by construction — every one of the 22 operations in Money Flows that debits a user account hits this guard |
| System accounts (all six) | true | fees, fx_spread, naqd_reserve, card_settlement, partner_settlement:<rail>, external_funding — these represent the platform's own liabilities/counterparties and are structurally allowed to run negative mid-period |
Balance is read live from accountBalance — a fresh SUM() over that account's LedgerEntry rows inside the same locked transaction — never from a cached field on LedgerAccount itself. ErrOverdraft surfaces to the API as INSUFFICIENT_BALANCE (400).
The reconcile tool
ledger.Reconcile(db) is the independent check that the paired-write convention
above actually held, for every wallet, from scratch.
type Mismatch struct {
WalletID uint
Currency string
WalletBalance decimal.Decimal
LedgerBalance decimal.Decimal
Diff decimal.Decimal // WalletBalance - LedgerBalance
}
Reconcile lists every models.Wallet row, and for each one looks up its LedgerAccount by (owner_type="user", owner_id=<wallet.ID as string>, currency=wallet.Currency). If no account exists yet (nothing has ever posted to this wallet), the ledger balance is treated as zero. It compares that sum against wallet.Balance and appends a Mismatch for every disagreement — Diff is signed so you can tell at a glance whether the cache is ahead or behind the truth.
// Command reconcile compares every wallet's cached balance projection
// against the sum of its ledger entries and reports any mismatches.
// Exits 0 with no output issues when the ledger is clean, 1 otherwise.
package main
func main() {
cfg := config.Load()
db := database.Connect(cfg)
mismatches, err := ledger.Reconcile(db)
if err != nil {
log.Fatalf("reconcile failed: %v", err) // os.Exit(1) via log.Fatalf
}
if len(mismatches) == 0 {
fmt.Println("✅ ledger reconciliation clean: 0 mismatches")
return // exit 0
}
fmt.Printf("⚠️ %d wallet(s) out of sync with the ledger:\n", len(mismatches))
for _, m := range mismatches {
fmt.Printf(" wallet=%d currency=%s wallet_balance=%s ledger_balance=%s diff=%s\n",
m.WalletID, m.Currency, m.WalletBalance.String(), m.LedgerBalance.String(), m.Diff.String())
}
os.Exit(1)
}
go run ./cmd/tools/reconcile from mynex-backend/ — reads the same .env-driven config as the server, no flagslog.Fatalf, printed to stderros.Exit(1) — the same exit code as a hard failure, so a monitoring wrapper should always read the output, not just the code, to tell the two apartIt means some code path changed Wallet.Balance without posting a matching,
equal-and-opposite ledger entry in the same transaction — a violation of the one rule this
whole page is about. It is never expected in normal operation; a positive count from this
tool is a real bug report, not noise. See Investigating a discrepancy
below, and Known gaps for one confirmed source of mismatches in the
current code.
Card holds — the deliberate exception
Card authorization holds are the one place a "balance" moves outside ledger.Post
entirely — and this is why the invariant above still holds, not despite it.
// handleAuth places (or declines) an authorization hold. ... a hold only
// reserves funds via Wallet.LockedBalance — no ledger posting happens until
// handleClearing actually settles it. This mirrors every real card network
// (authorize reserves, capture moves money) and keeps ledger.Reconcile's
// Wallet.Balance-vs-ledger invariant untouched by holds that might never
// capture.
- The reasoning, made explicit: if a hold posted to the ledger, a wallet with an outstanding uncaptured hold would show a real ledger balance lower than its
Wallet.Balance— by design, not by bug.Reconcilehas no way to distinguish "expected hold-shaped drift" from "an actual missing posting." Keeping holds entirely out of the ledger keeps the zero-sum, wallet-matches-ledger invariant unconditionally true, with no special-casing anywhere inReconcilefor in-flight holds. - Only
Wallet.LockedBalancemoves on auth (repo.AdjustLockedBalance(+holdAmount)) and reversal (-holdAmount) — confirmed: neither call site touchesledger.Post. - Clearing is the only card event that posts — and when it does, it releases the full original hold from
LockedBalancein the same transaction as the ledger entries, so the two mutations (ledger legs + lock release) commit or roll back together.
Full ledger-entry tables for all four card events: Money Flows → Card lifecycle.
Idempotency
middleware.Idempotency(redisClient) is attached per-route (a third handler
argument) to money-moving POSTs, and is what makes a network-retried request
provably safe rather than merely "probably fine."
SetNX.const idempotencyTTL = 24 * time.Hour
redisKey := fmt.Sprintf("idempotency:%s:%s:%s", userPart, c.Path(), key)
lockKey := redisKey + ":lock"
responseKey := redisKey + ":response"
claimed, err := redisClient.SetNX(ctx, lockKey, "1", idempotencyTTL).Result()
if !claimed {
stored, err := redisClient.Get(ctx, responseKey).Result()
if err == nil && stored != "" {
return replayStoredResponse(c, stored)
}
return apperr.Respond(c, apperr.Duplicate("a request with this idempotency key is already in progress"))
}
handlerErr := c.Next()
status := c.Response().StatusCode()
if handlerErr != nil || status >= fiber.StatusInternalServerError {
_ = redisClient.Del(ctx, lockKey).Err() // don't poison the key on our own failures
return handlerErr
}
// else: marshal {status, body} and Set(responseKey, payload, idempotencyTTL)
| Property | Behavior |
|---|---|
| Key format | idempotency:<userID-or-"anon">:<request path>:<Idempotency-Key header> |
| Cached payload | {status int, body []byte} — the exact final HTTP response, only cached on success |
| Replay injection | meta.idempotent_replay = true spliced into the cached JSON body; falls back to raw-body replay if it wasn't a JSON object |
| Concurrent duplicate | 409 via apperr.Duplicate → CodeDuplicate → fiber.StatusConflict |
| TTL | 24h on both the lock and the cached response |
| Crash before cleanup | the lock still expires naturally after 24h — no permanent stuck state |
Deterministic idempotency-key derivation elsewhere
The HTTP-layer middleware isn't the only idempotency mechanism — several services derive their own deterministic keys so retries at their layer are safe too, independent of the Redis lock above:
| Layer | Mechanism |
|---|---|
payment-service (bill-pay, NFC) | GenerateIdempotencyKey(userID, walletID, amount, reference) — pure function of stable inputs; the doc comment explicitly notes it used to include time.Now().Unix(), which broke idempotency, and was fixed |
partner-service (TnG) | partnerRef(idempotencyKey) derives "TNG-SBX-XXXXXXXX" from the operation UUID — same key always yields the same ref; DeterministicFailureRate uses FNV-32a hashing, not math/rand, so simulated failure is also reproducible per key |
blockchain-service (sandbox chain) | apply(kind, address, delta, reference) is idempotent per (kind, reference) — a worker crash-recovery retry returns the already-recorded receipt instead of double-crediting; see NAQD Minting → custody model |
stablecoin-service worker | alreadyPosted(tx, reference) checks for an existing LedgerTransaction by reference before every ledger post, including reversals |
Together these two layers are what actually prevents double-spend on retry: the HTTP layer stops a duplicate request from re-running the handler at all, and the deterministic-key layer stops a duplicate operation (e.g. a worker retry after a crash) from double-posting even if it does run again.
Transaction tracking
Two models carry the state that makes async settlement crash-safe: models.Transaction (the user-facing record) and models.PaymentSettlement (the settlement engine's own state machine, one-to-one with a top-up transaction).
Transaction.Reference"TXN-" + uuid.New().String() in BeforeCreate if not set — this is the same string reused as the ledger.Post reference, tying a user-facing transaction directly to its ledger posting by a shared keyTransaction.Statuspending, processing, completed, failed, cancelled — not a Go const enum, so a grep for a typo'd status string won't be caught at compile timeconst (
SettlementStatusPending = "pending"
SettlementStatusProcessing = "processing"
SettlementStatusCompleted = "completed"
SettlementStatusFailed = "failed"
)
type PaymentSettlement struct {
...
Provider string
ProviderRef string
Status string
Attempts int
NextPollAt *time.Time // drives the sweeper
SettledAt *time.Time
}
All sweeper state lives in this row, never in memory — a server restart simply resumes wherever the DB left off:
// GetDue returns non-final settlements whose next_poll_at has arrived (or
// was never set), oldest-due first, capped at limit rows per sweep tick.
func (r *SettlementRepository) GetDue(ctx context.Context, now time.Time, limit int) ([]models.PaymentSettlement, error) {
var rows []models.PaymentSettlement
err := r.db.WithContext(ctx).
Where("status IN ? AND (next_poll_at IS NULL OR next_poll_at <= ?)",
[]string{models.SettlementStatusPending, models.SettlementStatusProcessing}, now).
Order("next_poll_at ASC").
Limit(limit).
Find(&rows).Error
return rows, err
}
The sweeper's own loop (Sweeper.Start) ticks every PAYMENTS_SWEEPER_INTERVAL (30s) and calls sweepOnce → GetDue → PollAndSettle per row. Two sweeper instances racing the same due row are safe: complete()/fail() re-lock the settlement and re-check IsFinal() inside their own transaction before writing anything.
Audit trail
models.AuditLog is the durable, queryable record of who-did-what — distinct from the best-effort events.Bus (drop-if-full, in-memory, for live dashboards) described in Data Flows → Admin WebSocket. An audit-write failure is logged, not fatal, except when the write shares a DB transaction with the operation it's recording (wallet adjustment approval), where a failed audit write rolls back the money movement too.
| Call site | Records |
|---|---|
admin-service/wallet_adjustments.go | WALLET_ADJUSTMENT_REQUESTED / _APPROVED / _REJECTED — the approve write is inside the same tx as the ledger posting |
admin-service/service_fiber.go | UPDATE_USER_STATUS, KYC_REVIEW, withdrawal approval |
admin-service/auth_handler.go | admin login / logout |
stablecoin-service/audit.go | NAQD operation approve/reject |
card-service/service.go | card lifecycle admin actions (freeze, approve, reject, ship) |
partner-service/service.go | PARTNER_OPERATION_EXECUTED |
kyc-service/repository.go | LogKYCActivity |
fx-service/service.go | FX rate admin override |
GET /admin/audit-logs (role admin) supports admin_user_id, entity_type, action (all exact match), from/to on created_at, paginated (limit capped at 100), ordered newest-first — confirmed directly against the handler and repository code, matching what API Reference → Admin and Admin Console describe as backend-ready with no view built yet.
NAQD reserve accounting
The proof-of-reserve numbers an admin sees are computed live from the ledger, every request — no snapshot table to drift. Full depth (mint/burn lifecycle, oracle, economics) is on NAQD Minting & Reserves; the formula itself belongs here because it's a reconciliation-style computation:
// naqd_reserve/NAQD is credited on burn, debited on mint — its balance is
// the negative of circulating supply.
circulatingSupply := reserveNAQDLiability.Neg()
totalReserveUSD := reserveUSD.Add(reserveMVR.Mul(mvrToUSD))
naqdValueUSD := circulatingSupply.Mul(silverPriceUSD.Div(31.1034768))
if naqdValueUSD.GreaterThan(decimal.Zero) {
collateralRatio = totalReserveUSD.Div(naqdValueUSD).Mul(100).Round(2)
} else {
collateralRatio = decimal.Zero
}
isHealthy = collateralRatio >= MinCollateralRatio // default 110
rebalancingRequired = naqdValueUSD > 0 && collateralRatio < RebalanceThreshold // default 115
// oracle down → collateralRatio forced to 0, rebalancingRequired forced true (fail-safe)
Both reserve balances are read the same way every other reconciliation check reads a balance in this system: ledger.AccountBalance, a live SUM() over that account's entries — the reserve figure and the ledger can never disagree, by construction, in a way Reconcile couldn't also catch for a user wallet.
Investigating a discrepancy
A practical sequence for an operator who just saw reconcile report a non-zero count.
- Run the tool and capture full output.
go run ./cmd/tools/reconcile— every line gives youwallet_id,currency,wallet_balance,ledger_balance, and the signeddiff. A positive diff means the cache is ahead of the ledger (money appears from nowhere); negative means the cache is behind (money looks like it vanished). - Find the ledger account and read its full entry history.
illustrative — adapt to your DB client
SELECT id, owner_type, owner_id, currency, allow_negative FROM ledger_accounts WHERE owner_type = 'user' AND owner_id = '<wallet_id>' AND currency = '<currency>'; SELECT le.created_at, lt.kind, lt.reference, le.amount FROM ledger_entries le JOIN ledger_transactions lt ON lt.id = le.ledger_transaction_id WHERE le.ledger_account_id = <id> ORDER BY le.created_at ASC; - Line up ledger entries against the transaction history for the same wallet.
GET /wallet/transactions(or a directtransactionstable query filtered byuser_id) gives every user-facing transaction in order; everycompletedmoney-moving transaction should have exactly one matchingreferencein the ledger query above. A transaction with no matching ledger reference is the smoking gun. - Check the audit log for admin-initiated changes to this wallet.
GET /admin/audit-logs?entity_type=wallet_adjustmentorentity_type=user, filtered by date range around the suspected drift — dual-approval adjustments and withdrawal approvals are the two admin paths that touch a wallet outside the normal user-facing flows. - Check against the known gap below before assuming a new bug: if the pattern is "cached balance higher than ledger by exactly a past withdrawal amount, and that withdrawal's status shows an admin rejection," it's the documented
ApproveWithdrawalissue, not a new one. - Never edit a posted
LedgerEntryrow directly. The ledger is append-only by convention — every correction in this codebase (NAQD reversal, partner refund, card refund) is a new, independent posting that nets to the right answer, never a mutation of history. If the wallet cache itself needs correcting to match a ledger that's already right, that's a projection-cache fix (safe to recompute from the ledger sum); if real money needs to move, that's a freshKindAdjustmentposting through the normal dual-approval/admin/wallets/:id/adjustflow, not a manual UPDATE.
Known gaps
FiberService.ApproveWithdrawal (internal/admin-service/service_fiber.go),
on rejection, refunds the wallet with a raw
tx.Model(&wallet).Update("balance", gorm.Expr("balance + ?", withdrawal.Amount))
— bypassing ledger.Post/ledger.ApplyToWallet entirely. Since
wallet-service.Withdraw already posted a real KindWithdraw ledger
transaction debiting the user, a rejected withdrawal leaves the ledger showing the money gone
while the cached wallet balance shows it refunded. This is a genuine, reproducible source of
the exact Mismatch shape Reconcile exists to catch — confirmed by
reading the handler directly, not inferred. See Money Flows
→ Withdrawal for the full context.
No other code path audited for this documentation wave was found to mutate Wallet.Balance outside ledger.ApplyToWallet — this appears to be an isolated gap in one admin approval branch, not a systemic pattern.