Compare commits

..

3 Commits

Author SHA1 Message Date
Renato Alcara d119cb064a fix(lint): flatten max-depth in storeTcTokensFromHistorySync
CI lint failed with:
  process-message.ts:127:6 error Blocks are nested too deeply (5). Maximum allowed is 4 max-depth

The previous shape nested `if (pnsToResolve.length) → try → if (mappings) → for → if (pn && lid)` = 5 levels. Flattened to 4 by:
- Using `mappings ?? []` so the for-of can iterate without an outer guard
- Inverting the inner predicate to `if (!pn || !lid) continue`

No behavior change. CI should now pass.
2026-04-25 23:49:55 -03:00
Renato Alcara b5910c591a perf(history-sync): true batch + partial-failure resilience for tcToken sync
Addresses all 4 PR #387 review findings (Copilot + CodeRabbit):

#1 (Copilot, Major) — Promise.all still issued one getLIDForPN call per
chat. Replaced with a single getLIDsForPNs batch over deduped PN inputs:
turns O(N) round-trips into 1, AND shares USync retry across PNs that
miss cache. LID inputs (and @hosted.lid) skip the lookup entirely.

#4 (CodeRabbit, Major) — Promise.all is all-or-nothing: a single
resolveTcTokenJid rejection (transient DB error) would reject the whole
storeTcTokensFromHistorySync call AND abort the surrounding
HISTORY_SYNC_NOTIFICATION handler — meaning messaging-history.set
never fires and EVERY tctoken in the chunk is lost. Wrapped the batch
call in try/catch with per-chat fallback (storage under unresolved jid),
matching resolveTcTokenJid's null-LID branch.

#2 (Copilot/CodeRabbit, Minor) — `if (!candidates.length) return` was
dead code: after the early return for empty tokenChats, candidates is a
1:1 map of tokenChats and can never be empty. Removed.

#3 (CodeRabbit, Minor) — `as { ... }[]` type assertion was redundant
(TypeScript already infers the shape from the async map callback).
Removed — inferred typing now catches future shape drift.

Bonus cleanup: dropped unused `resolveTcTokenJid` import.

Tests: 35/35 suites, 824/824 passing. Customizations untouched.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-25 23:43:48 -03:00
Renato Alcara ae333b2910 perf(history-sync): parallelise tcToken JID resolution per chunk
PR #386 added storeTcTokensFromHistorySync which iterates chats in a
sync chunk and calls resolveTcTokenJid (an async getLIDForPN lookup)
per candidate. The original shape used sequential `await` inside a
for-of, so each chunk blocked for O(N) round-trips before
messaging-history.set could fire downstream.

In production this surfaced as systemic message-delivery latency:
under heavy history sync (re-scan / multi-device pairing) the event
buffer backed up, the adaptive flush mode escalated to "aggressive",
and outbound sends competed with the sync queue for socket time.
QR re-scans made it worse because a fresh history sync triggered
another round of sequential resolutions.

Fix:
1. Pre-filter chats with `tcToken && tcTokenTimestamp > 0` so we
   never spin up promises for the empty cases (most chats in a chunk).
2. Parallelise the resolveTcTokenJid lookups via Promise.all — same
   semantics, but all N getLIDForPN calls run concurrently instead
   of serially.

Downstream (loop that builds `entries`) is untouched: it remains
synchronous and keeps the same-batch dedup guard, so monotonicity
guarantees stay intact.

Customizations untouched: zero diff in carousel/buttons/lists/
LID-PN normalization/Bad MAC/proto.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-25 23:26:17 -03:00
8 changed files with 64 additions and 299 deletions
+1 -1
View File
@@ -1,7 +1,7 @@
syntax = "proto3";
package proto;
/// WhatsApp Version: 2.3000.1038164556
/// WhatsApp Version: 2.3000.1038024963
message ADVDeviceIdentity {
optional uint32 rawId = 1;
+1 -1
View File
@@ -1 +1 @@
{"version":[2,3000,1038167900]}
{"version":[2,3000,1038147544]}
+23 -149
View File
@@ -54,7 +54,7 @@ import {
resolveLidToPn
} from '../Utils'
import { makeKeyedMutex, makeMutex } from '../Utils/make-mutex'
import processMessage, { getChatId } from '../Utils/process-message'
import processMessage from '../Utils/process-message'
import { buildTcTokenFromJid } from '../Utils/tc-token-utils'
import {
type BinaryNode,
@@ -130,18 +130,6 @@ export const makeChatsSocket = (config: SocketConfig) => {
/** this mutex ensures that notifications from the same chat are processed in order, while allowing parallel processing across chats */
const notificationMutex = makeKeyedMutex()
/**
* Per-chat mutex dedicated to post-upsert work (history app-state sync +
* processMessage side effects). Kept separate from `messageMutex` because
* the inbound caller already holds `messageMutex(chatId)` while running
* decrypt + upsertMessage; sharing the same mutex would let a concurrently-
* arrived message N+1 enqueue *between* msg N's outer callback and msg N's
* post-upsert task, so msg N+1's processMessage could run before msg N's
* (breaking per-chat ordering of side effects). With a separate mutex,
* post-upsert tasks enqueue strictly in upsertMessage call order.
*/
const postUpsertMutex = makeKeyedMutex()
// Timeout for AwaitingInitialSync state
let awaitingSyncTimeout: NodeJS.Timeout | undefined
@@ -1411,19 +1399,7 @@ export const makeChatsSocket = (config: SocketConfig) => {
blockedCollections.clear()
logger.info('Doing app state sync')
try {
await resyncAppState(ALL_WA_PATCH_NAMES, true)
} catch (err) {
// Failure recovery: without this, syncState would stay at Syncing
// and ev.flush() would never run, leaving the event buffer pinned
// until the buffer's own safety timeout expires. Force the state
// machine forward so live inbound events can flow even if the
// app-state resync failed (collections are already cleared, so
// blocked patches will be retried on the next creds.update tick).
syncState = SyncState.Online
ev.flush()
throw err
}
await resyncAppState(ALL_WA_PATCH_NAMES, true)
// Sync is complete, go online and flush everything
syncState = SyncState.Online
@@ -1435,131 +1411,29 @@ export const makeChatsSocket = (config: SocketConfig) => {
}
}
// Post-upsert work: history app-state sync + processMessage side effects.
// Awaiting here keeps `messages.upsert` pinned in the event buffer
// (createBufferedFunction only schedules flush after work() resolves), so
// the hot path detaches this work to release the emit on the next debounce
// tick.
//
// Use Promise.allSettled so the combined promise only settles after BOTH
// tasks finish. With a plain Promise.all, an early rejection from one task
// would release the keyed mutex while the other task is still mutating
// chat state — letting the next message of the same chat overtake it and
// break per-chat ordering.
//
// Returns the per-task settle status so the keyShare branch can know
// whether processMessage actually persisted the new app-state-sync key
// before triggering doAppStateSync (otherwise the sync would hit
// isMissingKeyError and park collections in blockedCollections).
const postUpsertTasks = async (): Promise<{ processMessageOk: boolean }> => {
const [historyResult, processResult] = await Promise.allSettled([
shouldProcessHistoryMsg ? doAppStateSync() : Promise.resolve(),
processMessage(msg, {
signalRepository,
shouldProcessHistoryMsg,
placeholderResendCache,
ev,
creds: authState.creds,
keyStore: authState.keys,
logger,
options: config.options,
getMessage
})
])
if (historyResult.status === 'rejected') {
logger?.warn(
{ err: historyResult.reason, messageId: msg.key?.id, remoteJid: msg.key?.remoteJid },
'history doAppStateSync failed'
)
}
if (processResult.status === 'rejected') {
logger?.warn(
{ err: processResult.reason, messageId: msg.key?.id, remoteJid: msg.key?.remoteJid },
'processMessage failed'
)
}
return { processMessageOk: processResult.status === 'fulfilled' }
}
// Use getChatId + jidNormalizedUser so the mutex key matches the chat-id
// scheme processMessage uses for chat updates (broadcasts target the
// participant). When getChatId/jidNormalizedUser yields nothing usable
// (missing or malformed JID), prefer a message-derived fallback over a
// single global 'unknown' bucket — that bucket would head-of-line block
// every malformed message behind a shared queue. msg.key.id is unique
// per message so unrelated malformed inputs no longer serialize together;
// for valid messages we still hit the normalized chat-id path, so the
// per-chat ordering guarantee is unchanged where it matters.
const rawChatId = getChatId(msg.key)
const normalizedChatId = rawChatId ? jidNormalizedUser(rawChatId) : ''
const postUpsertChatId = normalizedChatId || msg.key?.id || 'unknown'
// Wrap in `postUpsertMutex(chatId)` (a SEPARATE keyed mutex from the outer
// `messageMutex` held by the inbound caller) so per-chat ordering of
// processMessage side effects (chat.unreadCount, LID/PN mapping,
// messages.update, history downloads) is preserved across messages of the
// same chat.
//
// Why a separate mutex: if we re-used messageMutex, a concurrently-arrived
// message N+1 could enqueue on the outer mutex BEFORE msg N's post-upsert
// task gets enqueued (because N's outer callback yields on `await decrypt()`
// before reaching the inner enqueue site). The queue would then be
// [OuterN+1, InnerN, ...], so InnerN+1 would beat InnerN to processMessage.
// With its own mutex, post-upsert tasks enqueue strictly in upsertMessage
// call order (which IS message arrival order because the outer
// messageMutex serializes the upserts per-chat).
const postUpsertWork = postUpsertMutex.mutex(postUpsertChatId, postUpsertTasks)
const isKeyShareDuringSync =
!!msg.message?.protocolMessage?.appStateSyncKeyShare && syncState === SyncState.Syncing
if (isKeyShareDuringSync) {
// appStateSyncKeyShare path: processMessage persists the new app-state-sync
// key in its APP_STATE_SYNC_KEY_SHARE handler (via keyStore.transaction).
// The follow-up doAppStateSync() needs that key to decrypt patches, so
// we MUST wait for processMessage to actually succeed before kicking off
// the sync — otherwise it would hit isMissingKeyError and park
// collections in blockedCollections, regressing the very issue this
// branch was added to fix.
//
// No deadlock with the inbound caller's messageMutex because
// postUpsertMutex is a different mutex instance.
logger.info('App state sync key arrived, awaiting persistence before triggering sync')
const { processMessageOk } = await postUpsertWork
if (!processMessageOk) {
logger?.warn(
{ messageId: msg.key?.id, remoteJid: msg.key?.remoteJid },
'processMessage failed during key-share — skipping doAppStateSync to avoid isMissingKeyError'
)
} else {
try {
await Promise.all([
(async () => {
if (shouldProcessHistoryMsg) {
await doAppStateSync()
} catch (err) {
logger?.warn(
{ err, messageId: msg.key?.id, remoteJid: msg.key?.remoteJid },
'doAppStateSync failed after key-share persistence'
)
}
}
} else {
// `postUpsertWork` is not expected to reject — `Promise.allSettled`
// inside `postUpsertTasks` never rejects, and per-task failures are
// already logged inline. The defensive catch routes any truly
// unexpected rejection (e.g. `postUpsertMutex` internal corruption,
// future synchronous throws inside processMessage) through
// `onUnexpectedError` instead of letting it surface as an
// UnhandledPromiseRejection — which on Node ≥15 can terminate the
// long-running socket process.
postUpsertWork.catch(err =>
onUnexpectedError(
err,
`processing post-upsert work for message ${msg.key?.id || 'unknown'} on ${msg.key?.remoteJid || 'unknown chat'}`
)
)
})(),
processMessage(msg, {
signalRepository,
shouldProcessHistoryMsg,
placeholderResendCache,
ev,
creds: authState.creds,
keyStore: authState.keys,
logger,
options: config.options,
getMessage
})
])
// If the app state key arrives and we are waiting to sync, trigger the sync now.
if (msg.message?.protocolMessage?.appStateSyncKeyShare && syncState === SyncState.Syncing) {
logger.info('App state sync key arrived, triggering app state sync')
await doAppStateSync()
}
})
+29 -39
View File
@@ -2330,54 +2330,44 @@ export const makeMessagesRecvSocket = (config: SocketConfig) => {
)
const alt = msg.key.participantAlt || msg.key.remoteJidAlt
// Handle LID/PN mappings with optimized hot-path:
// - storeLIDPNMappings is fire-and-forget (background) — does NOT block decrypt
// - migrateSession is SYNC (await) — REQUIRED for decrypt to find session
//
// SAFETY: normalizeMessageJids has a fast-path that uses key.*Alt directly without
// hitting the store, so the just-arrived message normalizes correctly even before
// the background store completes. Subsequent messages in the same chat hit the
// store after the background write is done (ms later).
//
// Pre-check (getPNForLID/getLIDForPN) was removed — storeLIDPNMappings has internal
// LRU cache + dedup, the pre-check was a redundant store round-trip per inbound
// message that added latency under load.
//
// HISTORICAL: this restores the intent of d73cd28d39 (2026-02-03) which was
// partially reverted by c3fc792351 the same day due to a race-condition concern
// with migrateSession (kept sync here). storeLIDPNMappings was over-protected:
// it persists a mapping that downstream consumers can re-derive from key.*Alt,
// while migrateSession actually moves the Signal session record that decrypt()
// will load microseconds later — those two have very different criticality.
//
// DO NOT make migrateSession async — decrypt() depends on the session being at
// the correct identifier (LID vs PN) when it runs. Other code paths (USync
// device lookup in messages-send.ts) create LID/PN mappings without migrating
// the session, so we cannot skip migration even when the mapping already exists.
// Handle LID/PN mappings with hybrid approach:
// - Store mapping operation runs in background (non-critical for decrypt)
// - Session migration MUST complete before decrypt() to avoid "No session record" errors
// This addresses Codex/Copilot review concerns about race conditions with decrypt()
if (!!alt) {
const altServer = jidDecode(alt)?.server
const primaryJid = msg.key.participant || msg.key.remoteJid!
if (altServer === 'lid') {
// Fire-and-forget: storeLIDPNMappings has internal cache+dedup,
// pre-check (getPNForLID) was redundant.
signalRepository.lidMapping
.storeLIDPNMappings([{ lid: alt, pn: primaryJid }])
.catch(error => logger.warn({ error, alt, primaryJid }, 'background LID mapping store failed'))
// Check if mapping already exists to avoid unnecessary storage operations
const existingMapping = await signalRepository.lidMapping.getPNForLID(alt)
if (!existingMapping) {
// MUST await: normalizeMessageJids() runs after this and needs the mapping
// in the LIDMappingStore to resolve LID→PN for events delivered to consumers
await signalRepository.lidMapping
.storeLIDPNMappings([{ lid: alt, pn: primaryJid }])
.catch(error => logger.warn({ error, alt, primaryJid }, 'LID mapping storage failed'))
}
// CRITICAL: ALWAYS migrate session SYNC, even if mapping exists.
// Other code paths (e.g., USync device lookup in messages-send.ts) may create
// mappings via storeLIDPNMappings() without calling migrateSession(). This
// leaves sessions under PN format while decrypt() expects LID format.
// Skipping migration based on mapping existence causes "No session record" errors.
// CRITICAL: ALWAYS migrate session, even if mapping exists
// Other code paths (e.g., USync device lookup in messages-send.ts:310-319)
// may create mappings via storeLIDPNMappings() without calling migrateSession()
// This leaves sessions under PN format while decrypt() expects LID format
// Skipping migration based on mapping existence causes "No session record" errors
await signalRepository.migrateSession(primaryJid, alt)
} else {
// Fire-and-forget: same rationale as above.
signalRepository.lidMapping
.storeLIDPNMappings([{ lid: primaryJid, pn: alt }])
.catch(error => logger.warn({ error, alt, primaryJid }, 'background LID mapping store failed'))
// Check if reverse mapping exists
const existingMapping = await signalRepository.lidMapping.getLIDForPN(alt)
if (!existingMapping) {
// MUST await: normalizeMessageJids() runs after this and needs the mapping
// in the LIDMappingStore to resolve LID→PN for events delivered to consumers
await signalRepository.lidMapping
.storeLIDPNMappings([{ lid: primaryJid, pn: alt }])
.catch(error => logger.warn({ error, alt, primaryJid }, 'LID mapping storage failed'))
}
// CRITICAL: ALWAYS migrate session SYNC.
// CRITICAL: ALWAYS migrate session, even if mapping exists
// Same reasoning as above - mapping existence doesn't guarantee session migration
await signalRepository.migrateSession(alt, primaryJid)
}
}
+1 -12
View File
@@ -341,18 +341,7 @@ export const addTransactionCapability = (
return result
} catch (error) {
// SessionError is part of the normal Bad MAC recovery flow
// (retry receipt → sender resends as pkmsg → new session within ~1.3s).
// Logging it as ERROR creates 2 noise lines per recoverable Bad MAC cycle.
// Downgrade to debug for SessionError; keep ERROR for everything else.
// The error is still re-thrown — recovery behavior is unchanged.
const errName = (error as { name?: string })?.name
if (errName === 'SessionError') {
logger.debug({ error }, 'transaction failed (SessionError — recoverable via retry receipt)')
} else {
logger.error({ error }, 'transaction failed, rolling back')
}
logger.error({ error }, 'transaction failed, rolling back')
throw error
}
})
+2 -23
View File
@@ -58,11 +58,7 @@ export const BAD_MAC_ERROR_TEXT = 'Bad MAC'
export const DECRYPTION_RETRY_CONFIG = {
maxRetries: 3,
baseDelayMs: 100,
// 'No matching sessions found' is the libsignal error when decryptWithSessions exhausts
// all stored sessions for a JID. Same recovery flow (retry receipt → pkmsg → new session)
// — categorise it as session-record so the caller logs DEBUG on retry, ERROR only when
// retries are exhausted (instead of dumping the full stack as an unknown error).
sessionRecordErrors: ['No session record', 'SessionError: No session record', 'No matching sessions found'],
sessionRecordErrors: ['No session record', 'SessionError: No session record'],
corruptedSessionErrors: ['Bad MAC', 'MessageCounterError', MISSING_KEYS_ERROR_TEXT]
}
@@ -425,26 +421,9 @@ export const decryptMessageNode = (
const isCorrupted = isCorruptedSessionError(originalError)
const isSessionRecord = isSessionRecordError(originalError)
// Slim error projection — keep name/message/type for diagnosis,
// drop `stack` which adds 4-5 lines of node_modules paths per log
// for known-recoverable libsignal errors.
//
// CRITICAL: only slim for KNOWN-RECOVERABLE categories (corrupted /
// session-record). The unknown-error branch keeps the full Error so
// protobuf/parsing/runtime bugs still emit a stack trace where it
// matters most. Catches Copilot/Codex P2 review on PR #391.
const slimErr = originalError
? {
name: (originalError as { name?: string }).name,
message: (originalError as { message?: string }).message,
type: (originalError as { type?: string }).type
}
: undefined
const isRecoverableCategory = isCorrupted || isSessionRecord
const errorContext = {
key: fullMessage.key,
err: isRecoverableCategory ? slimErr : originalError,
err: originalError,
messageType: tag === 'plaintext' ? 'plaintext' : attrs.type,
sender,
author,
+5 -63
View File
@@ -82,47 +82,6 @@ const REAL_MSG_REQ_ME_STUB_TYPES = new Set([WAMessageStubType.GROUP_PARTICIPANT_
* (TC_TOKEN_INDEX_KEY) via buildMergedTcTokenIndexWrite, so the 24h prune sweep in
* messages-recv picks them up across sessions.
*/
/**
* Single-concurrency queue for `storeTcTokensFromHistorySync` calls.
*
* Why: the function does read-then-write merges (`keyStore.get('tctoken', ...)`
* compute `keyStore.set(...)`) which are NOT atomic at the store level. If two
* history-sync chunks invoke this concurrently (common during reconnect / QR
* scan), an older chunk that started first can `keyStore.set` AFTER a newer
* chunk, overwriting the newer entry and worse, the merged `__index` write
* can drop JIDs the other chunk just added. Result: stale tcTokens / repeat 463
* sends until the next opportunistic refetch.
*
* Serialising via a chained Promise keeps the runs ordered while still freeing
* the calling `processMessage` to emit `messaging-history.set` immediately
* (the chain is fire-and-forget at the call site). Errors don't break the chain
* each `catch` resets it to `Promise.resolve()` so a single failure can't
* stall future runs.
*
* The chain is module-scoped (one per Node process). Multiple Baileys instances
* sharing this module will serialise across instances too, but their writes
* target different keyStores so there's no correctness gain only a tiny loss
* of inter-instance parallelism for tcToken syncs, which is acceptable given
* how rarely this runs vs. how rare cross-instance contention is.
*/
let historyTcTokenChain: Promise<void> = Promise.resolve()
function scheduleHistoryTcTokenSync(
chats: Chat[],
signalRepository: SignalRepositoryWithLIDStore,
keyStore: SignalKeyStoreWithTransaction,
logger?: ILogger
): void {
historyTcTokenChain = historyTcTokenChain
.catch(() => {
/* swallow prior error so chain stays alive */
})
.then(() => storeTcTokensFromHistorySync(chats, signalRepository, keyStore, logger))
.catch(err => {
logger?.warn({ err }, 'background tctoken history-sync persistence failed')
})
}
async function storeTcTokensFromHistorySync(
chats: Chat[],
signalRepository: SignalRepositoryWithLIDStore,
@@ -606,28 +565,11 @@ const processMessage = async (
}
}
// Persist tctokens carried by history-sync chats in BACKGROUND, serialised.
//
// Originally awaited (PR #386) to avoid 463 on first multi-device send, but in
// production this drained the event buffer per-chunk and added visible delivery
// latency (especially after restart / QR scan when many chunks arrived at once).
//
// `scheduleHistoryTcTokenSync` enqueues onto a single-concurrency promise chain
// (see definition above) — chunks persist sequentially in the order they were
// emitted, preserving timestamp monotonicity AND keeping the `__index` write
// safe from concurrent merge clobbers. The call returns immediately so the
// `messaging-history.set` emit is not blocked.
//
// TRADE-OFF: a listener that fires an outbound send IMMEDIATELY after the emit
// may race the still-pending persistence and get a 463 on that specific send.
// The existing 463 handler in messages-recv.ts triggers a getPrivacyTokens()
// refetch that auto-recovers within seconds. Net result is much better UX than
// per-chunk stalls.
//
// DO NOT add `await` back here without re-evaluating production latency, AND
// DO NOT call storeTcTokensFromHistorySync directly — it must go through the
// chain to preserve write ordering across overlapping chunks.
scheduleHistoryTcTokenSync(data.chats, signalRepository, keyStore, logger)
// Persist tctokens carried by history-sync chats BEFORE emitting messaging-history.set
// — listeners may immediately fire outbound sends that need the tctoken, and the store
// has to be populated first to avoid an error 463 on the first multi-device send.
// Runs AFTER storeLIDPNMappings (see comment above) so LID resolution works.
await storeTcTokensFromHistorySync(data.chats, signalRepository, keyStore, logger)
ev.emit('messaging-history.set', {
...data,
+2 -11
View File
@@ -28,16 +28,7 @@ console.info = function (...args: unknown[]) {
// Track errors by type + JID to avoid duplicates (using Map for better performance)
const _errorTimestamps = new Map<string, number>()
// Dedup window for repeated decrypt-error console lines (Bad MAC / Counter / etc).
// Was 150ms, but retry attempts of the SAME message are typically ~300-1000ms apart,
// so the second attempt fell outside the window and double-printed.
//
// TRADE-OFF: dedup key is `errorType + JID` (no message-id). With 5s, a burst of
// errors for the SAME JID — even of slightly different categories or different
// messages — collapses to one log line every 5s. This is intentional for a noisy
// production stream; if you need per-message visibility, set BAILEYS_LOG_LEVEL=debug
// to bypass this console-side dedup and see the structured pino logs in full.
const DEDUP_WINDOW_MS = 5000
const DEDUP_WINDOW_MS = 150
console.error = function (...args: unknown[]) {
if (args.length > 0 && typeof args[0] === 'string') {
@@ -79,7 +70,7 @@ console.error = function (...args: unknown[]) {
const lastTime = _errorTimestamps.get(dedupeKey)
if (lastTime && now - lastTime < DEDUP_WINDOW_MS) {
return // Skip duplicate within DEDUP_WINDOW_MS window
return // Skip duplicate within 150ms window
}
_errorTimestamps.set(dedupeKey, now)