DEV Community

KOMARI Subheeksh
KOMARI Subheeksh

Posted on • Edited on

Your Midnight deploy will crash on a fresh install - here are the wallet-sdk patches (and why they happen)

Every Midnight developer eventually hits this. You clone a working project, run npm install, and the deploy script that worked yesterday crashes with something like:

TypeError: state.pendingOutputs.values.map is not a function or its return value is not iterable
  at Object.pickAllCoins (CoreWallet.js:28:66)
Enter fullscreen mode Exit fullscreen mode

Or your wallet sync hangs for minutes and dies with:

Wallet.Sync: [object Object]
  at file:///.../wallet-sdk-shielded/dist/v1/Sync.js:126:169
Enter fullscreen mode Exit fullscreen mode

The first time this happened to me I assumed it was a Node version issue. It is not. It's a bug in the Midnight wallet SDK itself — a family of bugs, actually, spread across five packages — and the fix is a set of patches you have to re-apply after every npm install.

The root cause: plain iterators without Iterator helpers

The wallet SDK (version 1.2.0 core, shielded 3.0.2) defines its own ledger types — maps of transaction intents, outputs, and coin nonces. Those custom types implement .entries(), .values(), and .keys() methods that return plain iterators.

Here's the problem. Modern JavaScript's Iterator protocol has helper methods: .map(), .filter(), .find(), .every(), .toArray(). Native Map and Set objects get these helpers from Iterator.prototype — and in Node 22+, you can do:

new Map().values().map(x => x)  // works on native Map
Enter fullscreen mode Exit fullscreen mode

But the SDK's custom ledger types return iterators that do NOT inherit from Iterator.prototype. They're plain objects that only implement the next() method. So:

state.pendingOutputs.values().map(...)
// TypeError: state.pendingOutputs.values.map is not a function
Enter fullscreen mode Exit fullscreen mode

This is not a Node version problem. It fails on every Node version, because the SDK's custom iterators never had the helper methods to begin with. The SDK code was written against an environment where these helpers existed (or the code was never run against the custom types), and it shipped assuming .map() would be there.

The patch: spread into a real array first

The fix for each crash site is mechanical: convert the plain iterator into a real array (which has all the methods) before chaining:

- state.pendingOutputs.values().map(...)
+ [...state.pendingOutputs.values()].map(...)
Enter fullscreen mode Exit fullscreen mode

Here are the affected files and the exact patches, verified against wallet-sdk-shielded 3.0.2 / unshielded 3.1.0 / facade 4.1.0 / dust-wallet 4.2.0.

1. wallet-sdk-shielded/dist/v1/CoreWallet.js.values().map()

The crash from the top of this post. This one hits first because pickAllCoins runs during coin selection, before any transaction is built.

sed -i 's/state.pendingOutputs.values().map/([...state.pendingOutputs.values()]).map/' \
  node_modules/@midnight-ntwrk/wallet-sdk-shielded/dist/v1/CoreWallet.js
Enter fullscreen mode Exit fullscreen mode

2. wallet-sdk-shielded/dist/v1/TransactionImbalances.js.entries().every() (x2)

sed -i 's/imbalances.guaranteed.entries().every/[...imbalances.guaranteed.entries()].every/' \
  node_modules/@midnight-ntwrk/wallet-sdk-shielded/dist/v1/TransactionImbalances.js
sed -i 's/segmentImbalances.entries().every/[...segmentImbalances.entries()].every/' \
  node_modules/@midnight-ntwrk/wallet-sdk-shielded/dist/v1/TransactionImbalances.js
Enter fullscreen mode Exit fullscreen mode

3. wallet-sdk-shielded/dist/v1/TransactionOps.js.entries().filter() (x2)

These call sites span multiple lines, so a plain sed won't do. The chain looks like:

tx
  .imbalances(0)
  .entries()
  .filter(...)
Enter fullscreen mode Exit fullscreen mode

Patch with a small Node script:

const fs = require('fs');
let f = 'node_modules/@midnight-ntwrk/wallet-sdk-shielded/dist/v1/TransactionOps.js';
let c = fs.readFileSync(f, 'utf8');
c = c.replace(/tx\n(\s+)\.imbalances\(0\)\n\1\.entries\(\)\n\1\.filter\(/g,
  '[...tx\n$1.imbalances(0)\n$1.entries()]\n$1.filter(');
c = c.replace(/tx\n(\s+)\.imbalances\(segment\)\n\1\.entries\(\)\n\1\.filter\(/g,
  '[...tx\n$1.imbalances(segment)\n$1.entries()]\n$1.filter(');
fs.writeFileSync(f, c);
Enter fullscreen mode Exit fullscreen mode

4. wallet-sdk-unshielded-wallet/dist/v1/TransactionOps.js.keys().toArray() + .entries().filter().map().toArray()

sed -i 's/transaction.intents?.keys().toArray()/Array.from(transaction.intents?.keys() ?? [])/' \
  node_modules/@midnight-ntwrk/wallet-sdk-unshielded-wallet/dist/v1/TransactionOps.js
Enter fullscreen mode Exit fullscreen mode

The .entries().filter().map().toArray() chain is multi-line and also ends with .toArray() that has no replacement — the cleanest approach is a Python one-liner:

f = 'node_modules/@midnight-ntwrk/wallet-sdk-unshielded-wallet/dist/v1/TransactionOps.js'
c = open(f).read()
c = c.replace(
    'const imbalances = transaction\n            .imbalances(segment)\n            .entries()\n            .filter',
    'const imbalances = [...transaction\n            .imbalances(segment)\n            .entries()]\n            .filter'
)
c = c.replace('.toArray();\n        return Imbalances.fromEntries', ';\n        return Imbalances.fromEntries')
open(f, 'w').write(c)
Enter fullscreen mode Exit fullscreen mode

5. wallet-sdk-facade/dist/transaction.js.values().toArray() (x2)

sed -i 's/tx.intents?.values().toArray()/[...(tx.intents?.values() ?? [])]/g' \
  node_modules/@midnight-ntwrk/wallet-sdk-facade/dist/transaction.js
Enter fullscreen mode Exit fullscreen mode

6. wallet-sdk-dust-wallet/dist/v1/Transacting.js.entries().find()

const fs = require('fs');
let f = 'node_modules/@midnight-ntwrk/wallet-sdk-dust-wallet/dist/v1/Transacting.js';
let c = fs.readFileSync(f, 'utf8');
c = c.replace(
  /const \[_, imbalance\] = transaction\n(\s+)\.imbalances\(0, totalFee\)\n\1\.entries\(\)\n\1\.find\(/,
  'const [_, imbalance] = [...transaction\n$1.imbalances(0, totalFee)\n$1.entries()]\n$1.find('
);
fs.writeFileSync(f, c);
Enter fullscreen mode Exit fullscreen mode

Bonus bug: Set.difference() doesn't exist on older Node

While patching CoreWallet.js, you'll find a second landmine right next to the first:

coinNonces.difference(definedNonces)
Enter fullscreen mode Exit fullscreen mode

Set.prototype.difference() is ES2025 — it exists on Node 22.13+ and 24, but NOT on Node 20, which the Midnight toolchain still supports. On Node 20 this fails with coinNonces.difference is not a function during wallet state restore.

sed -i 's/coinNonces.difference(definedNonces)/new Set([...coinNonces].filter(x => !definedNonces.has(x)))/' \
  node_modules/@midnight-ntwrk/wallet-sdk-shielded/dist/v1/CoreWallet.js
Enter fullscreen mode Exit fullscreen mode

A correction on wallet sync hangs

An earlier version of this post claimed the preprod RPC closes idle WebSocket connections and that client keepalive pings were needed. That was wrong. Direct testing against wss://rpc.preprod.midnight.network shows an idle connection stays open well past a minute, and the server sends a ping every 30 seconds that the client auto-pongs — idle connections do not drop for lack of client keepalives. The verification snippet in the earlier version also passed { pingInterval, pingTimeout } to new WebSocket(), which are not valid ws options and are silently ignored, so it did not actually reproduce anything.

The sync hang itself is real — long preprod syncs can still fail with Wallet.Sync errors from the shielded/unshielded sync modules — but the idle-timeout explanation was incorrect. The practical workaround that got syncs moving was the sync-gap tolerance patch (see #8-10 below), and the honest advice is to capture the actual sync error before reaching for any patch. Treat the keepalive theory as retracted.

The complete patch script

Here's everything in one place — save it as patch-sdk.sh in your project and run it after every npm install:

#!/bin/bash
# Comprehensive Midnight wallet-sdk patches (11 patches, 13 fixes across 7 files)
# Run after every `npm install` before deploying.
# The SDK's custom ledger types define .entries()/.values()/.keys() that return
# plain iterators WITHOUT Iterator helper methods (.filter, .map, .find, .every).
# This is an SDK bug, not a Node version issue — affects all Node versions.
set -e

ROOT="${1:-.}"
cd "$ROOT"

echo "=== Patching Midnight wallet-sdk iterator bugs ==="

# ── Shielded wallet (5 patches) ──

# 1. CoreWallet.js — .values().map()
sed -i 's/state.pendingOutputs.values().map/([...state.pendingOutputs.values()]).map/' \
  node_modules/@midnight-ntwrk/wallet-sdk-shielded/dist/v1/CoreWallet.js
echo "  ✓ CoreWallet.js"

# 1b. CoreWallet.js — Set.difference() (ES2025, missing on Node 20)
sed -i 's/coinNonces.difference(definedNonces)/new Set([...coinNonces].filter(x => !definedNonces.has(x)))/' \
  node_modules/@midnight-ntwrk/wallet-sdk-shielded/dist/v1/CoreWallet.js
echo "  ✓ CoreWallet.js Set.difference"

# 2. TransactionImbalances.js — .entries().every() (2 occurrences)
sed -i 's/imbalances.guaranteed.entries().every/[...imbalances.guaranteed.entries()].every/' \
  node_modules/@midnight-ntwrk/wallet-sdk-shielded/dist/v1/TransactionImbalances.js
sed -i 's/segmentImbalances.entries().every/[...segmentImbalances.entries()].every/' \
  node_modules/@midnight-ntwrk/wallet-sdk-shielded/dist/v1/TransactionImbalances.js
echo "  ✓ TransactionImbalances.js"

# 3. TransactionOps.js (shielded) — .entries().filter() (2 occurrences)
node -e "
const fs = require('fs');
let f = 'node_modules/@midnight-ntwrk/wallet-sdk-shielded/dist/v1/TransactionOps.js';
let c = fs.readFileSync(f, 'utf8');
c = c.replace(/tx\n(\s+)\.imbalances\(0\)\n\1\.entries\(\)\n\1\.filter\(/g,
  '[...tx\n\$1.imbalances(0)\n\$1.entries()]\n\$1.filter(');
c = c.replace(/tx\n(\s+)\.imbalances\(segment\)\n\1\.entries\(\)\n\1\.filter\(/g,
  '[...tx\n\$1.imbalances(segment)\n\$1.entries()]\n\$1.filter(');
fs.writeFileSync(f, c);
"
echo "  ✓ TransactionOps.js (shielded)"

# ── Unshielded wallet (2 patches in 1 file) ──

# 4. TransactionOps.js (unshielded) — .keys().toArray()
sed -i 's/transaction.intents?.keys().toArray()/Array.from(transaction.intents?.keys() ?? [])/' \
  node_modules/@midnight-ntwrk/wallet-sdk-unshielded-wallet/dist/v1/TransactionOps.js
echo "  ✓ TransactionOps.js keys (unshielded)"

# 5. TransactionOps.js (unshielded) — .entries().filter().map().toArray()
# The Node regex approach is fragile due to escaping. Python fallback is more reliable.
if python3 -c "
f = 'node_modules/@midnight-ntwrk/wallet-sdk-unshielded-wallet/dist/v1/TransactionOps.js'
c = open(f).read()
c = c.replace(
    'const imbalances = transaction\\n            .imbalances(segment)\\n            .entries()\\n            .filter',
    'const imbalances = [...transaction\\n            .imbalances(segment)\\n            .entries()]\\n            .filter'
)
c = c.replace('.toArray();\\n        return Imbalances.fromEntries', ';\\n        return Imbalances.fromEntries')
open(f,'w').write(c)
print('ok')
" 2>/dev/null; then
    echo "  ✓ TransactionOps.js imbalances (unshielded) [python]"
else
    # Node regex fallback
    node -e "
const fs = require('fs');
let f = 'node_modules/@midnight-ntwrk/wallet-sdk-unshielded-wallet/dist/v1/TransactionOps.js';
let c = fs.readFileSync(f, 'utf8');
c = c.replace(
  /const imbalances = transaction\\n(\\s+)\\.imbalances\\(segment\\)\\n\\1\\.entries\\(\\)\\n\\1\\.filter\\(/,
  'const imbalances = [...transaction\\n\$1.imbalances(segment)\\n\$1.entries()]\\n\$1.filter('
);
c = c.replace(/\\.toArray\\(\\);\\n(\\s+)return Imbalances\\.fromEntries/, ';\\n\$1return Imbalances.fromEntries');
fs.writeFileSync(f, c);
" && echo "  ✓ TransactionOps.js imbalances (unshielded) [node]"
fi

# ── Facade (1 patch, 2 occurrences) ──

# 6. transaction.js — .values().toArray() (2 occurrences)
sed -i 's/tx.intents?.values().toArray()/[...(tx.intents?.values() ?? [])]/g' \
  node_modules/@midnight-ntwrk/wallet-sdk-facade/dist/transaction.js
echo "  ✓ transaction.js (facade)"

# ── Dust wallet (1 patch) ──

# 7. Transacting.js — .entries().find()
node -e "
const fs = require('fs');
let f = 'node_modules/@midnight-ntwrk/wallet-sdk-dust-wallet/dist/v1/Transacting.js';
let c = fs.readFileSync(f, 'utf8');
c = c.replace(
  /const \[_, imbalance\] = transaction\n(\s+)\.imbalances\(0, totalFee\)\n\1\.entries\(\)\n\1\.find\(/,
  'const [_, imbalance] = [...transaction\n\$1.imbalances(0, totalFee)\n\$1.entries()]\n\$1.find('
);
fs.writeFileSync(f, c);
"
echo "  ✓ Transacting.js (dust)"

# ── Wallet sync workarounds (3 patches) ⚠️ UNSAFE — see note below ──

# 8. Skip shielded sync
sed -i 's/this.shielded.waitForSyncedState(),//' \
  node_modules/@midnight-ntwrk/wallet-sdk-facade/dist/index.js
echo "  ✓ facade index.js"

# 9-10. Increase sync gap tolerance
sed -i 's/waitForSyncedState(allowedGap = 0n)/waitForSyncedState(allowedGap = 100000n)/' \
  node_modules/@midnight-ntwrk/wallet-sdk-unshielded-wallet/dist/UnshieldedWallet.js
sed -i 's/waitForSyncedState(allowedGap = 0n)/waitForSyncedState(allowedGap = 100000n)/' \
  node_modules/@midnight-ntwrk/wallet-sdk-dust-wallet/dist/DustWallet.js
echo "  ✓ sync gap tolerance"

echo ""
echo "=== All 11 patches applied (13 fixes across 7 files) ==="
Enter fullscreen mode Exit fullscreen mode

Unsafe workarounds (patches #8-10). These three do NOT fix a bug — they disable sync-completeness checks. Skipping the shielded waitForSyncedState() and raising allowedGap to 100000n let a wallet act on incomplete chain state, which risks acting on stale or missing data. Use them only for local dev / demos where a slightly-off balance is acceptable, and never on a wallet handling real funds. The iterator patches (#1-7) are safe; #8-10 are not, and were kept separate for that reason.

Note: the unshielded .entries().filter().map().toArray() chain is safest with the Python snippet above — the regex escaping in sed/node gets fragile. Check node_modules/@midnight-ntwrk/wallet-sdk-unshielded-wallet/dist/v1/TransactionOps.js after patching and confirm no .entries().filter() chain remains un-prefixed.

How to check whether you need the patches

If your deploy crashes with any of these, you need them:

state.pendingOutputs.values.map is not a function
imbalances.guaranteed.entries().every is not a function
transaction.intents?.keys().toArray is not a function
coinNonces.difference is not a function
Wallet.Sync: [object Object]  (after a long hang on preprod)
Enter fullscreen mode Exit fullscreen mode

And you can grep your installed SDK directly:

grep -c "\.values()\.map\|\.entries()\.every\|\.keys()\.toArray\|\.difference(" \
  node_modules/@midnight-ntwrk/wallet-sdk-shielded/dist/v1/CoreWallet.js
Enter fullscreen mode Exit fullscreen mode

Why this matters

Every Midnight project that deploys from a fresh clone hits this wall — including the official example repos, because the SDK is broken the same way for everyone. The patches are mechanical, but discovering them cost real debugging time, and they must be re-applied after every npm install (the patches live in node_modules, so a fresh install restores the broken code). Put patch-sdk.sh in your repo, run it in CI before npm run deploy, and your fresh-clone experience will match your working-tree experience.

A more durable alternative is patch-package: apply the patches once, run npx patch-package @midnight-ntwrk/wallet-sdk-shielded ... to generate a patches/ directory, add a postinstall script (patch-package), and every npm install re-applies them automatically — no manual re-run, and the patches are reviewed as part of your repo instead of living only in a script.

Verified against wallet-sdk 1.2.0, wallet-sdk-shielded 3.0.2, wallet-sdk-unshielded-wallet 3.1.0, wallet-sdk-facade 4.1.0, wallet-sdk-dust-wallet 4.2.0, proof-server 8.1.0.

Top comments (0)