Compare commits

...

6 Commits

Author SHA1 Message Date
Renato Alcara 9e3af35b54 fix(history-sync): serialise tcToken persistence + drop external doc reference
Addresses 4 PR #390 review findings (Codex P1 + Copilot ×3):

#1 Codex P1 (real, MAJOR) — `storeTcTokensFromHistorySync` fire-and-forget
   created a write race when multiple history-sync chunks arrived in parallel
   (common during reconnect/QR scan). The function does a non-atomic
   read-then-merge-then-write: chunk B could read `existing` before chunk A
   wrote, then commit later and overwrite chunk A's newer timestamp. Even
   worse, the merged `__index` write could drop JIDs added by the other
   chunk, leaving stale tcTokens and persistent 463 send errors.

   FIX: introduced `scheduleHistoryTcTokenSync()` — a module-scoped
   single-concurrency promise chain that sequentialises all calls in order.
   The chain swallows prior errors so a single failure can't stall future
   runs. Call site stays fire-and-forget so the `messaging-history.set`
   emit is still instant (which is the whole point of PR #389).

#2 Copilot (same root as #1) — already covered by the chain.

#3+#4 Copilot — code comments referenced
   `Downloads/InfiniteAPI-Inbound-Latency-Fix-Documentation.md` which is
   intentionally out-of-tree (user keeps it locally). Replaced the broken
   refs with self-contained explanations covering the same rationale —
   future maintainers no longer need an external file to understand the
   trade-offs.

Skipped: CodeRabbit nitpick to extract a `fireAndForgetMappingStore` helper
(stylistic, current explicit form makes the lid/pn argument order obvious to
reviewers — keeping it).

Tests: 35/35 suites, 824/824 still passing. Customizations untouched.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-26 10:25:27 -03:00
Renato Alcara a631c8c3c3 perf(inbound-latency): restore async LID mapping + fire-and-forget tctoken history sync
PRODUCTION ISSUE: Inbound messages from smartphone to ZPRO frontend were
arriving with seconds of delay. Outbound (ZPRO → smartphone) was instant.
Started after PR #386 (tctoken lifecycle) deploy.

ROOT CAUSE: Three compounding factors:

1. The historical fix d73cd28d39 (2026-02-03, "fix inbound latency by making
   LID mapping async") was partially reverted the same day by c3fc792351
   ("hybrid approach") due to a valid race-condition concern with decrypt().
   The reversion was over-protective: storeLIDPNMappings does NOT need to be
   sync — only migrateSession does. The hybrid kept all 3 awaits sync.

2. PR #386 added `await storeTcTokensFromHistorySync(...)` BEFORE the
   `messaging-history.set` emit. Per chunk this drains the event buffer with
   2-4 store ops, which compounds when many chunks arrive at once (restart,
   QR scan, multi-device login).

3. Each pre-check `await getPNForLID(alt)` / `getLIDForPN(alt)` before
   storeLIDPNMappings was redundant — the store has its own LRU cache + dedup.

Combined under production load (multi-instance store contention, post-PR #386
extra ops per send) the per-message hot-path penalty became user-visible delay.

THIS FIX:

#1+#3: messages-recv.ts ~line 2332 — `storeLIDPNMappings` becomes
fire-and-forget, pre-check `getPNForLID/getLIDForPN` removed. `migrateSession`
stays SYNC (REQUIRED for decrypt — see Codex/Copilot review on PR #72 / commit
c3fc792351). normalizeMessageJids has a fast-path that uses key.*Alt directly
without hitting the store, so the just-arrived message normalizes correctly
even before the background store completes.

#2: process-message.ts ~line 451 — `storeTcTokensFromHistorySync` becomes
fire-and-forget. Trade-off: a listener firing an outbound send IMMEDIATELY
after the emit may race the background persistence and hit error 463 on that
specific send. Existing 463 handler in messages-recv.ts triggers
getPrivacyTokens() refetch that auto-recovers in seconds. Net UX is much
better than per-chunk stalls.

INVARIANTS PRESERVED:
- migrateSession remains SYNC — decrypt() depends on it (race condition guard)
- normalizeMessageJids remains SYNC — events need correct JIDs before emit
- messageMutex remains SYNC — per-chat ordering preserved
- All 824 tests still pass

DOCUMENTATION:
Full rationale, before/after code, test scenarios, rollback procedure and
guidance for future maintainers in:
Downloads/InfiniteAPI-Inbound-Latency-Fix-Documentation.md

DO NOT REVERT WITHOUT READING THE DOC.

Customizations untouched: zero diff in src/Utils/messages.ts (carousel/buttons/
lists), src/Socket/groups.ts, WAProto/*.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-26 09:59:54 -03:00
Renato Alcara 4fe708445a chore: update WhatsApp Web version to v2.3000.1038167900 (#389)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2026-04-26 06:18:52 -03:00
github-actions[bot] 336ed64a44 chore: update proto/version to v2.3000.1038164556 (#388)
Co-authored-by: rsalcara <rsalcara@users.noreply.github.com>
2026-04-26 01:07:43 -03:00
Renato Alcara 65cb09e4df perf(history-sync): parallelise tcToken JID resolution per chunk (#387)
perf(history-sync): parallelise tcToken JID resolution per chunk (#387)
2026-04-25 23:52:37 -03:00
Renato Alcara 9ff21db749 feat(tctoken): complete lifecycle (TIER 1 + 2 + 3 of upstream PR)
feat(tctoken): complete lifecycle (TIER 1 + 2 + 3 of upstream PR)
2026-04-25 18:52:45 -03:00
10 changed files with 960 additions and 102 deletions
+1 -1
View File
@@ -1,7 +1,7 @@
syntax = "proto3"; syntax = "proto3";
package proto; package proto;
/// WhatsApp Version: 2.3000.1038024963 /// WhatsApp Version: 2.3000.1038164556
message ADVDeviceIdentity { message ADVDeviceIdentity {
optional uint32 rawId = 1; optional uint32 rawId = 1;
+1 -1
View File
@@ -1 +1 @@
{"version":[2,3000,1038147544]} {"version":[2,3000,1038167900]}
+42 -2
View File
@@ -98,6 +98,24 @@ export const makeChatsSocket = (config: SocketConfig) => {
let privacySettings: { [_: string]: string } | undefined let privacySettings: { [_: string]: string } | undefined
/**
* Server-assigned AB props that gate tctoken-related protocol behavior.
* Defaults match WA Web (safe avoids spurious error 463 if the prop never
* arrives). Populated from `fetchProps()` on connection.
*
* - `privacyTokenOn1to1` (AB prop 10518 / `privacy_token_sending_on_all_1_on_1_messages`):
* include tctoken in 1:1 messages.
* - `profilePicPrivacyToken` (AB prop 9666 / `profile_scraping_privacy_token_in_photo_iq`):
* include tctoken in profile picture IQs.
* - `lidTrustedTokenIssueToLid` (AB prop 14303 / `lid_trusted_token_issue_to_lid`):
* issue privacy tokens to the contact's LID instead of the PN.
*/
const serverProps = {
privacyTokenOn1to1: true,
profilePicPrivacyToken: true,
lidTrustedTokenIssueToLid: false
}
let syncState: SyncState = SyncState.Connecting let syncState: SyncState = SyncState.Connecting
/** this mutex ensures that messages from the same chat are processed in order, while allowing parallel processing of messages from different chats */ /** this mutex ensures that messages from the same chat are processed in order, while allowing parallel processing of messages from different chats */
@@ -791,7 +809,10 @@ export const makeChatsSocket = (config: SocketConfig) => {
me && (normalizedJid === jidNormalizedUser(me.id) || (me.lid && normalizedJid === jidNormalizedUser(me.lid))) me && (normalizedJid === jidNormalizedUser(me.id) || (me.lid && normalizedJid === jidNormalizedUser(me.lid)))
let content: BinaryNode[] | undefined = baseContent let content: BinaryNode[] | undefined = baseContent
if (isUserJid && !isSelf) { // Gate inclusion on AB prop 9666 (profile_scraping_privacy_token_in_photo_iq).
// WA Web defaults to true; if the server flips it off, we mirror that to
// avoid divergence with the spec-compliant client.
if (serverProps.profilePicPrivacyToken && isUserJid && !isSelf) {
content = await buildTcTokenFromJid({ content = await buildTcTokenFromJid({
authState, authState,
jid: normalizedJid, jid: normalizedJid,
@@ -1093,7 +1114,25 @@ export const makeChatsSocket = (config: SocketConfig) => {
props = reduceBinaryNodeToDictionary(propsNode, 'prop') props = reduceBinaryNodeToDictionary(propsNode, 'prop')
} }
logger.debug('fetched props') // Extract protocol-relevant AB props (defaults match WA Web; see serverProps doc).
// We accept both numeric IDs and human-readable names so the parser is resilient
// to upstream renaming and to future-versioned WA Web servers.
const privacyTokenProp = props['10518'] ?? props['privacy_token_sending_on_all_1_on_1_messages']
if (privacyTokenProp !== undefined) {
serverProps.privacyTokenOn1to1 = privacyTokenProp === 'true' || privacyTokenProp === '1'
}
const profilePicProp = props['9666'] ?? props['profile_scraping_privacy_token_in_photo_iq']
if (profilePicProp !== undefined) {
serverProps.profilePicPrivacyToken = profilePicProp === 'true' || profilePicProp === '1'
}
const lidIssueProp = props['14303'] ?? props['lid_trusted_token_issue_to_lid']
if (lidIssueProp !== undefined) {
serverProps.lidTrustedTokenIssueToLid = lidIssueProp === 'true' || lidIssueProp === '1'
}
logger.debug({ serverProps }, 'fetched props')
return props return props
} }
@@ -1611,6 +1650,7 @@ export const makeChatsSocket = (config: SocketConfig) => {
return { return {
...sock, ...sock,
serverProps,
createCallLink, createCallLink,
getBotListV2, getBotListV2,
messageMutex, messageMutex,
+132 -89
View File
@@ -73,7 +73,14 @@ import {
recordMessageReceived, recordMessageReceived,
recordMessageRetry recordMessageRetry
} from '../Utils/prometheus-metrics.js' } from '../Utils/prometheus-metrics.js'
import { isTcTokenExpired, resolveTcTokenJid, storeTcTokensFromIqResult } from '../Utils/tc-token-utils' import {
buildMergedTcTokenIndexWrite,
isTcTokenExpired,
resolveIssuanceJid,
resolveTcTokenJid,
storeTcTokensFromIqResult,
TC_TOKEN_INDEX_KEY
} from '../Utils/tc-token-utils'
import { import {
areJidsSameUser, areJidsSameUser,
type BinaryNode, type BinaryNode,
@@ -164,7 +171,12 @@ export const makeMessagesRecvSocket = (config: SocketConfig) => {
let sendActiveReceipts = false let sendActiveReceipts = false
// ======= tctoken index tracking for cross-session pruning ======= // ======= tctoken index tracking for cross-session pruning =======
const TC_TOKEN_INDEX_KEY = '__index' // TC_TOKEN_INDEX_KEY is imported from tc-token-utils so the value stays in sync
// with messages-send/process-message writes (avoids string drift on rename).
// Race note: this prune-driven index write may interleave with
// buildMergedTcTokenIndexWrite calls from issuance/history-sync paths. Worst
// case: a JID resurrected by a stale read gets pruned again on the next 24h
// sweep — no data loss, just one extra cycle.
const TC_TOKEN_PRUNE_TS_KEY = '__prune_ts' const TC_TOKEN_PRUNE_TS_KEY = '__prune_ts'
const tcTokenKnownJids = new Set<string>() const tcTokenKnownJids = new Set<string>()
const tcTokenRetriedMsgIds = new Set<string>() const tcTokenRetriedMsgIds = new Set<string>()
@@ -198,16 +210,25 @@ export const makeMessagesRecvSocket = (config: SocketConfig) => {
} }
})() })()
/** Debounced save of the tctoken JID index (5s) */ /**
* Debounced save of the tctoken JID index (5s).
*
* Merges with the persisted index instead of overwriting other layers
* (messages-send fire-and-forget issuance, process-message history sync) may
* write JIDs to the index via `buildMergedTcTokenIndexWrite` without updating
* `tcTokenKnownJids`. Without the merge those JIDs would be silently dropped
* the next time this debounced save fires.
*/
const scheduleTcTokenIndexSave = () => { const scheduleTcTokenIndexSave = () => {
if (tcTokenIndexSaveTimer) clearTimeout(tcTokenIndexSaveTimer) if (tcTokenIndexSaveTimer) clearTimeout(tcTokenIndexSaveTimer)
tcTokenIndexSaveTimer = setTimeout(async () => { tcTokenIndexSaveTimer = setTimeout(async () => {
try { try {
const arr = Array.from(tcTokenKnownJids) const merged = await buildMergedTcTokenIndexWrite(authState.keys, tcTokenKnownJids)
await authState.keys.set({ await authState.keys.set({
tctoken: { tctoken: {
...merged,
[TC_TOKEN_INDEX_KEY]: { [TC_TOKEN_INDEX_KEY]: {
token: Buffer.from(JSON.stringify(arr), 'utf8'), ...merged[TC_TOKEN_INDEX_KEY],
timestamp: unixTimestampSeconds().toString() timestamp: unixTimestampSeconds().toString()
} }
} }
@@ -1434,6 +1455,58 @@ export const makeMessagesRecvSocket = (config: SocketConfig) => {
}, authState?.creds?.me?.id || 'sendRetryRequest') }, authState?.creds?.me?.id || 'sendRetryRequest')
} }
/**
* Fire-and-forget tctoken re-issuance after a peer's device identity changed.
* Mirrors WAWebSendTcTokenWhenDeviceIdentityChange runs in PARALLEL with the
* session refresh (not after it).
*
* Why parallel and not sequential:
* - WA Web invokes this BEFORE assertSessions to maximise the chance the contact
* has a fresh tctoken by the time the next outbound send executes.
* - Running after assertSessions (the fork's previous behaviour) races with the
* next send and risks error 463 when the contact reinstalls and the user replies
* immediately afterwards.
*
* Gated on `entry.senderTimestamp` (we previously issued a token to this peer in the
* current bucket window). Preserves the existing senderTimestamp instead of issuing
* a fresh one keeps WA Web's bucket coalescing semantics: same token, same window.
*/
const reissueTcTokenAfterIdentityChange = (from: string): void => {
void (async () => {
const normalizedJid = jidNormalizedUser(from)
const tcJid = await resolveTcTokenJid(normalizedJid, getLIDForPN)
const tcTokenData = await authState.keys.get('tctoken', [tcJid])
const senderTs = tcTokenData?.[tcJid]?.senderTimestamp
if (senderTs === null || senderTs === undefined || isTcTokenExpired(senderTs)) {
return
}
logTcToken('reissue', { jid: normalizedJid, reason: 'identity_changed', senderTimestamp: senderTs })
const getPNForLID = signalRepository.lidMapping.getPNForLID.bind(signalRepository.lidMapping)
const issueJid = await resolveIssuanceJid(
normalizedJid,
sock.serverProps.lidTrustedTokenIssueToLid,
getLIDForPN,
getPNForLID
)
const result = await getPrivacyTokens([issueJid], senderTs)
await storeTcTokensFromIqResult({
result,
fallbackJid: tcJid,
keys: authState.keys,
getLIDForPN,
onNewJidStored: storedJid => {
tcTokenKnownJids.add(storedJid)
scheduleTcTokenIndexSave()
}
})
logTcToken('reissue_ok', { jid: normalizedJid, reason: 'identity_changed' })
})().catch(err => {
logTcToken('reissue_fail', { jid: from, error: err?.message })
})
}
const handleEncryptNotification = async (node: BinaryNode) => { const handleEncryptNotification = async (node: BinaryNode) => {
const from = node.attrs.from const from = node.attrs.from
if (from === S_WHATSAPP_NET) { if (from === S_WHATSAPP_NET) {
@@ -1453,57 +1526,12 @@ export const makeMessagesRecvSocket = (config: SocketConfig) => {
validateSession: signalRepository.validateSession, validateSession: signalRepository.validateSession,
assertSessions, assertSessions,
debounceCache: identityAssertDebounce, debounceCache: identityAssertDebounce,
logger logger,
// Fire reissue in parallel with the session refresh, NOT after it.
// See reissueTcTokenAfterIdentityChange doc for the rationale.
onBeforeSessionRefresh: reissueTcTokenAfterIdentityChange
}) })
// When a session is refreshed (identity change), re-issue tctoken fire-and-forget
// WABA Android: reissue stores senderTimestamp + realIssueTimestamp after IQ success
if (result.action === 'session_refreshed') {
const normalizedJid = jidNormalizedUser(from)
resolveTcTokenJid(normalizedJid, getLIDForPN)
.then(async tcJid => {
const tcData = await authState.keys.get('tctoken', [tcJid])
const entry = tcData[tcJid]
if (entry?.token?.length && !isTcTokenExpired(entry.timestamp)) {
const senderTs = unixTimestampSeconds()
logTcToken('reissue', { jid: normalizedJid, reason: 'session_refreshed' })
getPrivacyTokens([normalizedJid], senderTs)
.then(async (iqResult) => {
await storeTcTokensFromIqResult({
result: iqResult,
fallbackJid: normalizedJid,
keys: authState.keys,
getLIDForPN,
onNewJidStored: (storedJid) => {
tcTokenKnownJids.add(storedJid)
scheduleTcTokenIndexSave()
}
})
// Persist senderTimestamp + realIssueTimestamp after IQ success
const currentData = await authState.keys.get('tctoken', [tcJid])
const currentEntry = currentData[tcJid]
await authState.keys.set({
tctoken: {
[tcJid]: {
...currentEntry,
token: currentEntry?.token ?? Buffer.alloc(0),
senderTimestamp: senderTs,
realIssueTimestamp: 0
}
}
})
logTcToken('reissue_ok', { jid: normalizedJid, reason: 'session_refreshed' })
})
.catch(err => {
logTcToken('reissue_fail', { jid: normalizedJid, error: err?.message })
})
}
})
.catch(() => {
/* ignore resolution errors */
})
}
if (result.action === 'no_identity_node') { if (result.action === 'no_identity_node') {
logger.info({ node }, 'unknown encrypt notification') logger.info({ node }, 'unknown encrypt notification')
} }
@@ -2302,44 +2330,54 @@ export const makeMessagesRecvSocket = (config: SocketConfig) => {
) )
const alt = msg.key.participantAlt || msg.key.remoteJidAlt const alt = msg.key.participantAlt || msg.key.remoteJidAlt
// Handle LID/PN mappings with hybrid approach: // Handle LID/PN mappings with optimized hot-path:
// - Store mapping operation runs in background (non-critical for decrypt) // - storeLIDPNMappings is fire-and-forget (background) — does NOT block decrypt
// - Session migration MUST complete before decrypt() to avoid "No session record" errors // - migrateSession is SYNC (await) — REQUIRED for decrypt to find session
// This addresses Codex/Copilot review concerns about race conditions with decrypt() //
// SAFETY: normalizeMessageJids has a fast-path that uses key.*Alt directly without
// hitting the store, so the just-arrived message normalizes correctly even before
// the background store completes. Subsequent messages in the same chat hit the
// store after the background write is done (ms later).
//
// Pre-check (getPNForLID/getLIDForPN) was removed — storeLIDPNMappings has internal
// LRU cache + dedup, the pre-check was a redundant store round-trip per inbound
// message that added latency under load.
//
// HISTORICAL: this restores the intent of d73cd28d39 (2026-02-03) which was
// partially reverted by c3fc792351 the same day due to a race-condition concern
// with migrateSession (kept sync here). storeLIDPNMappings was over-protected:
// it persists a mapping that downstream consumers can re-derive from key.*Alt,
// while migrateSession actually moves the Signal session record that decrypt()
// will load microseconds later — those two have very different criticality.
//
// DO NOT make migrateSession async — decrypt() depends on the session being at
// the correct identifier (LID vs PN) when it runs. Other code paths (USync
// device lookup in messages-send.ts) create LID/PN mappings without migrating
// the session, so we cannot skip migration even when the mapping already exists.
if (!!alt) { if (!!alt) {
const altServer = jidDecode(alt)?.server const altServer = jidDecode(alt)?.server
const primaryJid = msg.key.participant || msg.key.remoteJid! const primaryJid = msg.key.participant || msg.key.remoteJid!
if (altServer === 'lid') { if (altServer === 'lid') {
// Check if mapping already exists to avoid unnecessary storage operations // Fire-and-forget: storeLIDPNMappings has internal cache+dedup,
const existingMapping = await signalRepository.lidMapping.getPNForLID(alt) // pre-check (getPNForLID) was redundant.
if (!existingMapping) { signalRepository.lidMapping
// MUST await: normalizeMessageJids() runs after this and needs the mapping .storeLIDPNMappings([{ lid: alt, pn: primaryJid }])
// in the LIDMappingStore to resolve LID→PN for events delivered to consumers .catch(error => logger.warn({ error, alt, primaryJid }, 'background LID mapping store failed'))
await signalRepository.lidMapping
.storeLIDPNMappings([{ lid: alt, pn: primaryJid }])
.catch(error => logger.warn({ error, alt, primaryJid }, 'LID mapping storage failed'))
}
// CRITICAL: ALWAYS migrate session, even if mapping exists // CRITICAL: ALWAYS migrate session SYNC, even if mapping exists.
// Other code paths (e.g., USync device lookup in messages-send.ts:310-319) // Other code paths (e.g., USync device lookup in messages-send.ts) may create
// may create mappings via storeLIDPNMappings() without calling migrateSession() // mappings via storeLIDPNMappings() without calling migrateSession(). This
// This leaves sessions under PN format while decrypt() expects LID format // leaves sessions under PN format while decrypt() expects LID format.
// Skipping migration based on mapping existence causes "No session record" errors // Skipping migration based on mapping existence causes "No session record" errors.
await signalRepository.migrateSession(primaryJid, alt) await signalRepository.migrateSession(primaryJid, alt)
} else { } else {
// Check if reverse mapping exists // Fire-and-forget: same rationale as above.
const existingMapping = await signalRepository.lidMapping.getLIDForPN(alt) signalRepository.lidMapping
if (!existingMapping) { .storeLIDPNMappings([{ lid: primaryJid, pn: alt }])
// MUST await: normalizeMessageJids() runs after this and needs the mapping .catch(error => logger.warn({ error, alt, primaryJid }, 'background LID mapping store failed'))
// in the LIDMappingStore to resolve LID→PN for events delivered to consumers
await signalRepository.lidMapping
.storeLIDPNMappings([{ lid: primaryJid, pn: alt }])
.catch(error => logger.warn({ error, alt, primaryJid }, 'LID mapping storage failed'))
}
// CRITICAL: ALWAYS migrate session, even if mapping exists // CRITICAL: ALWAYS migrate session SYNC.
// Same reasoning as above - mapping existence doesn't guarantee session migration
await signalRepository.migrateSession(alt, primaryJid) await signalRepository.migrateSession(alt, primaryJid)
} }
} }
@@ -3220,19 +3258,24 @@ export const makeMessagesRecvSocket = (config: SocketConfig) => {
// Await index load first — prevents overwriting a more complete persisted index // Await index load first — prevents overwriting a more complete persisted index
// if the connection closes before the initial load finishes. // if the connection closes before the initial load finishes.
tcTokenIndexLoaded tcTokenIndexLoaded
.then(() => { .then(async () => {
Promise.resolve( try {
authState.keys.set({ // Same merge-with-persisted invariant as scheduleTcTokenIndexSave —
// other layers may have written cross-layer JIDs to the index since
// our in-memory set was last updated.
const merged = await buildMergedTcTokenIndexWrite(authState.keys, tcTokenKnownJids)
await authState.keys.set({
tctoken: { tctoken: {
...merged,
[TC_TOKEN_INDEX_KEY]: { [TC_TOKEN_INDEX_KEY]: {
token: Buffer.from(JSON.stringify([...tcTokenKnownJids]), 'utf8'), ...merged[TC_TOKEN_INDEX_KEY],
timestamp: unixTimestampSeconds().toString() timestamp: unixTimestampSeconds().toString()
} }
} }
}) })
).catch(() => { } catch {
/* non-critical */ /* non-critical */
}) }
}) })
.catch(() => { .catch(() => {
/* non-critical */ /* non-critical */
+59 -6
View File
@@ -46,7 +46,10 @@ import { makeKeyedMutex } from '../Utils/make-mutex'
import { metrics, recordMessageFailure, recordMessageSent } from '../Utils/prometheus-metrics' import { metrics, recordMessageFailure, recordMessageSent } from '../Utils/prometheus-metrics'
import { getMessageReportingToken, shouldIncludeReportingToken } from '../Utils/reporting-utils' import { getMessageReportingToken, shouldIncludeReportingToken } from '../Utils/reporting-utils'
import { import {
buildMergedTcTokenIndexWrite,
isRegularUser,
isTcTokenExpired, isTcTokenExpired,
resolveIssuanceJid,
resolveTcTokenJid, resolveTcTokenJid,
shouldSendNewTcToken, shouldSendNewTcToken,
storeTcTokensFromIqResult storeTcTokensFromIqResult
@@ -125,6 +128,15 @@ export const makeMessagesSocket = (config: SocketConfig) => {
// Tracks JIDs with an in-flight getPrivacyTokens IQ to avoid duplicate concurrent fetches // Tracks JIDs with an in-flight getPrivacyTokens IQ to avoid duplicate concurrent fetches
const tcTokenFetchingJids = new Set<string>() const tcTokenFetchingJids = new Set<string>()
/**
* Set of tctoken storage JIDs with a fire-and-forget `issuePrivacyTokens` IQ in flight.
* Distinct from `tcTokenFetchingJids` (which dedupes inbound *fetches* of the peer's
* token). Prevents duplicate IQs when a caller fires several rapid back-to-back sends
* to the same contact before `senderTimestamp` persists. Entries are always removed
* in `.finally()`, so the set is bounded by current concurrency.
*/
const inFlightTcTokenIssuance = new Set<string>()
let mediaConn: Promise<MediaConnInfo> let mediaConn: Promise<MediaConnInfo>
const refreshMediaConn = async (forceGet = false) => { const refreshMediaConn = async (forceGet = false) => {
const media = await mediaConn const media = await mediaConn
@@ -1715,7 +1727,16 @@ export const makeMessagesSocket = (config: SocketConfig) => {
} }
} }
if (tcTokenBuffer?.length) { // Gate inclusion on AB prop 10518 (privacy_token_sending_on_all_1_on_1_messages).
// WA Web defaults to true; if the server flips it off we mirror that to avoid
// divergence with the spec-compliant client.
//
// CARROUSEL EXCEPTION: Pastorini-validated carousel stanzas REQUIRE tctoken
// (CDP capture confirms it). If the AB prop ever flips off, dropping the
// tctoken from carousel would break rendering on Android. Carousel always
// includes the tctoken when one is available, regardless of the prop —
// matching the fork's existing behaviour pre-PR #2339.
if (tcTokenBuffer?.length && (sock.serverProps.privacyTokenOn1to1 || isCarousel)) {
;(stanza.content as BinaryNode[]).push({ ;(stanza.content as BinaryNode[]).push({
tag: 'tctoken', tag: 'tctoken',
attrs: {}, attrs: {},
@@ -1815,13 +1836,39 @@ export const makeMessagesSocket = (config: SocketConfig) => {
// Gated only by shouldSendNewTcToken — removed tcTokenBuffer?.length guard so // Gated only by shouldSendNewTcToken — removed tcTokenBuffer?.length guard so
// issuance fires even when we don't yet hold a token (bucket boundary crossed). // issuance fires even when we don't yet hold a token (bucket boundary crossed).
// IMPORTANT: must run AFTER sendNode — issuing before the message causes error 463. // IMPORTANT: must run AFTER sendNode — issuing before the message causes error 463.
if (is1on1Send && shouldSendNewTcToken(existingTokenEntry?.senderTimestamp)) { //
// WA Web also skips issuance for:
// - protocol messages (revoke, ephemeral settings, etc — see TcTokenChatAction)
// - PSA, bots, MetaAI (isRegularUser filter)
// and dedupes back-to-back issuances with `inFlightTcTokenIssuance` so a burst of
// rapid sends to the same contact only triggers a single IQ before senderTimestamp
// is persisted.
const isProtocolMsg = !!normalizeMessageContent(message)?.protocolMessage
// Use isRegularUser (the same Wid.isRegularUser() port that gates the store
// path) so we filter PSA/bot/MetaAI consistently regardless of JID server
// (@c.us vs @s.whatsapp.net) and device suffix. The previous PSA_WID/isJidBot
// checks only matched @c.us forms — destinationJid arrives normalized.
const isBotOrPSA = !isRegularUser(destinationJid)
if (
is1on1Send &&
!isProtocolMsg &&
!isBotOrPSA &&
shouldSendNewTcToken(existingTokenEntry?.senderTimestamp) &&
!inFlightTcTokenIssuance.has(tcTokenJid)
) {
inFlightTcTokenIssuance.add(tcTokenJid)
const issueTimestamp = unixTimestampSeconds() const issueTimestamp = unixTimestampSeconds()
logTcToken('reissue', { jid: destinationJid }) logTcToken('reissue', { jid: destinationJid })
getPrivacyTokens([destinationJid], issueTimestamp) const getPNForLID = signalRepository.lidMapping.getPNForLID.bind(signalRepository.lidMapping)
resolveIssuanceJid(
destinationJid,
sock.serverProps.lidTrustedTokenIssueToLid,
getLIDForPN,
getPNForLID
)
.then(issueJid => getPrivacyTokens([issueJid], issueTimestamp))
.then(async result => { .then(async result => {
// Store any tokens received in the IQ response. // Store any tokens received in the IQ response.
// onNewJidStored not passed — pruning index lives in messages-recv (higher layer).
await storeTcTokensFromIqResult({ await storeTcTokensFromIqResult({
result, result,
fallbackJid: tcTokenJid, fallbackJid: tcTokenJid,
@@ -1832,9 +1879,11 @@ export const makeMessagesSocket = (config: SocketConfig) => {
// Persist senderTimestamp unconditionally — WA Web stores it in the chat table // Persist senderTimestamp unconditionally — WA Web stores it in the chat table
// regardless of whether a token exists. Spread preserves token+timestamp if present. // regardless of whether a token exists. Spread preserves token+timestamp if present.
// WABA Android: INSERT INTO wa_trusted_contacts_send (jid, sent_tc_token_timestamp, real_issue_timestamp) // WABA Android: INSERT INTO wa_trusted_contacts_send (jid, sent_tc_token_timestamp, real_issue_timestamp)
// VALUES (?, ?, 0) — realIssueTimestamp=0 means issued but not yet confirmed by server // VALUES (?, ?, 0) — realIssueTimestamp=0 means issued but not yet confirmed by server.
// Also bump the cross-session prune index so this JID is tracked persistently.
const currentData = await authState.keys.get('tctoken', [tcTokenJid]) const currentData = await authState.keys.get('tctoken', [tcTokenJid])
const currentEntry = currentData[tcTokenJid] const currentEntry = currentData[tcTokenJid]
const indexWrite = await buildMergedTcTokenIndexWrite(authState.keys, [tcTokenJid])
await authState.keys.set({ await authState.keys.set({
tctoken: { tctoken: {
[tcTokenJid]: { [tcTokenJid]: {
@@ -1842,7 +1891,8 @@ export const makeMessagesSocket = (config: SocketConfig) => {
token: currentEntry?.token ?? Buffer.alloc(0), token: currentEntry?.token ?? Buffer.alloc(0),
senderTimestamp: issueTimestamp, senderTimestamp: issueTimestamp,
realIssueTimestamp: 0 realIssueTimestamp: 0
} },
...indexWrite
} }
}) })
@@ -1851,6 +1901,9 @@ export const makeMessagesSocket = (config: SocketConfig) => {
.catch(err => { .catch(err => {
logTcToken('reissue_fail', { jid: destinationJid, error: err?.message }) logTcToken('reissue_fail', { jid: destinationJid, error: err?.message })
}) })
.finally(() => {
inFlightTcTokenIssuance.delete(tcTokenJid)
})
} }
// Log with [BAILEYS] prefix // Log with [BAILEYS] prefix
+23
View File
@@ -57,6 +57,16 @@ export type IdentityChangeContext = {
debounceCache: NodeCache<boolean> debounceCache: NodeCache<boolean>
/** Logger instance for debugging and monitoring */ /** Logger instance for debugging and monitoring */
logger: ILogger logger: ILogger
/**
* Invoked right before `assertSessions` is called for an existing-session identity
* change. Used to kick off fire-and-forget side effects (e.g. tctoken re-issuance)
* in the same order WA Web does i.e. before the E2E session is re-established.
* Must not throw; implementations are responsible for their own error handling.
*
* Skipped when the refresh itself is skipped (no_identity_node, invalid_notification,
* skipped_companion_device, skipped_self_primary, debounced, skipped_offline).
*/
onBeforeSessionRefresh?: (jid: string) => void
} }
// ============================================================================ // ============================================================================
@@ -170,6 +180,19 @@ export async function handleIdentityChange(
// This ensures we don't incorrectly debounce when we exit early (offline, etc.) // This ensures we don't incorrectly debounce when we exit early (offline, etc.)
ctx.debounceCache.set(from, true) ctx.debounceCache.set(from, true)
// Fire-and-forget side effects (e.g. tctoken re-issuance) BEFORE the session is
// re-established. WA Web runs these in parallel with the session refresh —
// running afterwards would race with the next outbound send and risk error 463.
//
// Wrapped in try/catch so a misbehaving consumer callback cannot abort identity
// change recovery. We log and continue — assertSessions still runs so the E2E
// session always gets refreshed.
try {
ctx.onBeforeSessionRefresh?.(from)
} catch (error) {
ctx.logger.warn({ error, jid: from }, 'onBeforeSessionRefresh callback threw — continuing with session refresh')
}
// Attempt session refresh/creation // Attempt session refresh/creation
try { try {
await ctx.assertSessions([from], true) await ctx.assertSessions([from], true)
+187
View File
@@ -40,6 +40,7 @@ import { getKeyAuthor, toNumber } from './generics'
import { downloadAndProcessHistorySyncNotification } from './history' import { downloadAndProcessHistorySyncNotification } from './history'
import type { ILogger } from './logger' import type { ILogger } from './logger'
import { metrics, recordHistorySyncMessages } from './prometheus-metrics.js' import { metrics, recordHistorySyncMessages } from './prometheus-metrics.js'
import { buildMergedTcTokenIndexWrite } from './tc-token-utils'
type ProcessMessageContext = { type ProcessMessageContext = {
shouldProcessHistoryMsg: boolean shouldProcessHistoryMsg: boolean
@@ -62,6 +63,163 @@ const REAL_MSG_STUB_TYPES = new Set([
const REAL_MSG_REQ_ME_STUB_TYPES = new Set([WAMessageStubType.GROUP_PARTICIPANT_ADD]) const REAL_MSG_REQ_ME_STUB_TYPES = new Set([WAMessageStubType.GROUP_PARTICIPANT_ADD])
/**
* Extract tctoken / tcTokenTimestamp / tcTokenSenderTimestamp from history-sync chats
* and persist them to the `tctoken` store. Mirrors WA Web's `bulkCreateOrMerge` pass
* over the chat table during history sync.
*
* Why this matters: when a user logs in on a new device, the multi-device history sync
* is the only way that device learns about tctokens issued/received on the original
* device. Without this pass, the new device sends 1:1 messages with no tctoken until
* the contact triggers a fresh notification which surfaces as error 463 in production.
*
* Monotonicity: we only overwrite an existing entry if the incoming timestamp is
* STRICTLY newer (`incoming > existing`). Equal timestamps are skipped to avoid
* reverting senderTimestamp / realIssueTimestamp set by other layers (e.g. a
* reissue that fired between history-sync chunks).
*
* Index hygiene: every JID we write here is added to the persistent prune index
* (TC_TOKEN_INDEX_KEY) via buildMergedTcTokenIndexWrite, so the 24h prune sweep in
* messages-recv picks them up across sessions.
*/
/**
* Single-concurrency queue for `storeTcTokensFromHistorySync` calls.
*
* Why: the function does read-then-write merges (`keyStore.get('tctoken', ...)`
* compute `keyStore.set(...)`) which are NOT atomic at the store level. If two
* history-sync chunks invoke this concurrently (common during reconnect / QR
* scan), an older chunk that started first can `keyStore.set` AFTER a newer
* chunk, overwriting the newer entry and worse, the merged `__index` write
* can drop JIDs the other chunk just added. Result: stale tcTokens / repeat 463
* sends until the next opportunistic refetch.
*
* Serialising via a chained Promise keeps the runs ordered while still freeing
* the calling `processMessage` to emit `messaging-history.set` immediately
* (the chain is fire-and-forget at the call site). Errors don't break the chain
* each `catch` resets it to `Promise.resolve()` so a single failure can't
* stall future runs.
*
* The chain is module-scoped (one per Node process). Multiple Baileys instances
* sharing this module will serialise across instances too, but their writes
* target different keyStores so there's no correctness gain only a tiny loss
* of inter-instance parallelism for tcToken syncs, which is acceptable given
* how rarely this runs vs. how rare cross-instance contention is.
*/
let historyTcTokenChain: Promise<void> = Promise.resolve()
function scheduleHistoryTcTokenSync(
chats: Chat[],
signalRepository: SignalRepositoryWithLIDStore,
keyStore: SignalKeyStoreWithTransaction,
logger?: ILogger
): void {
historyTcTokenChain = historyTcTokenChain
.catch(() => {
/* swallow prior error so chain stays alive */
})
.then(() => storeTcTokensFromHistorySync(chats, signalRepository, keyStore, logger))
.catch(err => {
logger?.warn({ err }, 'background tctoken history-sync persistence failed')
})
}
async function storeTcTokensFromHistorySync(
chats: Chat[],
signalRepository: SignalRepositoryWithLIDStore,
keyStore: SignalKeyStoreWithTransaction,
logger?: ILogger
) {
// Cheap filter first — most chats in a sync chunk don't carry tcToken at all,
// and we want to avoid spinning up promises for them.
const tokenChats = chats.filter(chat => {
const ts = chat.tcTokenTimestamp ? toNumber(chat.tcTokenTimestamp) : 0
return !!chat.tcToken?.length && ts > 0
})
if (!tokenChats.length) {
return
}
// Pre-normalize so the rest of the pipeline is a synchronous join.
const normalized = tokenChats.map(chat => ({
chat,
ts: toNumber(chat.tcTokenTimestamp!),
jid: jidNormalizedUser(chat.id!)
}))
// BATCHED LID resolution. The previous shape called getLIDForPN once per
// chat (sequential await inside a for-of), which became the bottleneck
// during heavy history sync — every cold-cache hit was a DB round-trip,
// stalling messaging-history.set and spilling into the event-buffer.
// `getLIDsForPNs` resolves a deduped list in ONE batched query (and shares
// USync retry across PNs that miss cache), turning O(N) round-trips into 1.
//
// LID inputs (and `@hosted.lid`) skip the lookup entirely — they're already
// the storage form. Failures degrade gracefully: a missing mapping just
// stores under the original jid, matching `resolveTcTokenJid`'s null branch.
const pnsToResolve = [...new Set(normalized.filter(({ jid }) => !isLidUser(jid)).map(({ jid }) => jid))]
const pnToLid = new Map<string, string>()
if (pnsToResolve.length) {
try {
const mappings = await signalRepository.lidMapping.getLIDsForPNs(pnsToResolve)
// Flat loop (continue-on-skip) keeps max nesting depth at 4 for lint.
for (const { pn, lid } of mappings ?? []) {
if (!pn || !lid) continue
pnToLid.set(jidNormalizedUser(pn), lid)
}
} catch (err) {
// Per-chat fallback below (storageJid := jid). Don't abort the chunk —
// CodeRabbit noted that all-or-nothing rejection here would drop every
// tctoken in the batch AND prevent messaging-history.set from firing.
logger?.warn({ err }, 'storeTcTokensFromHistorySync: getLIDsForPNs batch failed; falling back to per-chat jid')
}
}
const candidates = normalized.map(({ chat, ts, jid }) => ({
storageJid: pnToLid.get(jid) ?? jid,
token: Buffer.from(chat.tcToken!),
ts,
senderTs: chat.tcTokenSenderTimestamp ? toNumber(chat.tcTokenSenderTimestamp) : undefined
}))
const jids = candidates.map(c => c.storageJid)
const existing = await keyStore.get('tctoken', jids)
const entries: Record<string, { token: Buffer; timestamp?: string; senderTimestamp?: number }> = {}
for (const c of candidates) {
// Same-batch dedup: when two chats resolve to the same storageJid (e.g. PN+LID
// aliases collapsing through resolveTcTokenJid, or duplicate chunks across
// retries), prefer the value already written by an earlier iteration so a
// lower-ts entry can't overwrite a higher-ts one captured from `existing`.
const existingEntry = entries[c.storageJid] ?? existing[c.storageJid]
const existingTs = existingEntry?.timestamp ? Number(existingEntry.timestamp) : 0
// Strict > guard: equal timestamps are skipped so we never clobber
// senderTimestamp written by other layers (issuance after send, etc).
if (existingTs > 0 && existingTs >= c.ts) {
continue
}
entries[c.storageJid] = {
...existingEntry,
token: c.token,
timestamp: String(c.ts),
...(c.senderTs !== undefined ? { senderTimestamp: c.senderTs } : {})
}
}
if (Object.keys(entries).length) {
logger?.debug({ count: Object.keys(entries).length }, 'storing tctokens from history sync')
try {
// Include updated __index so cross-session pruning picks these JIDs up.
const indexWrite = await buildMergedTcTokenIndexWrite(keyStore, Object.keys(entries))
await keyStore.set({ tctoken: { ...entries, ...indexWrite } })
} catch (err) {
logger?.warn({ err }, 'failed to store tctokens from history sync')
}
}
}
/** Cleans a received message to further processing */ /** Cleans a received message to further processing */
export const cleanMessage = (message: WAMessage, meId: string, meLid: string) => { export const cleanMessage = (message: WAMessage, meId: string, meLid: string) => {
// ensure remoteJid and participant doesn't have device or agent in it // ensure remoteJid and participant doesn't have device or agent in it
@@ -418,6 +576,12 @@ const processMessage = async (
// Emit LID-PN mappings from history sync // Emit LID-PN mappings from history sync
// This is how WhatsApp Web learns mappings for chats with non-contacts // This is how WhatsApp Web learns mappings for chats with non-contacts
//
// MUST run BEFORE storeTcTokensFromHistorySync — otherwise resolveTcTokenJid()
// can't resolve PN→LID for fresh-device chats (mapping cache is empty), tokens
// get persisted under PN keys, and the send path (which resolves to LID first)
// misses them — exactly the error 463 scenario this whole change is meant to
// prevent. Catches Codex P1 review on PR #386.
if (data.lidPnMappings?.length) { if (data.lidPnMappings?.length) {
logger?.debug({ count: data.lidPnMappings.length }, 'processing LID-PN mappings from history sync') logger?.debug({ count: data.lidPnMappings.length }, 'processing LID-PN mappings from history sync')
// eslint-disable-next-line max-depth // eslint-disable-next-line max-depth
@@ -442,6 +606,29 @@ const processMessage = async (
} }
} }
// Persist tctokens carried by history-sync chats in BACKGROUND, serialised.
//
// Originally awaited (PR #386) to avoid 463 on first multi-device send, but in
// production this drained the event buffer per-chunk and added visible delivery
// latency (especially after restart / QR scan when many chunks arrived at once).
//
// `scheduleHistoryTcTokenSync` enqueues onto a single-concurrency promise chain
// (see definition above) — chunks persist sequentially in the order they were
// emitted, preserving timestamp monotonicity AND keeping the `__index` write
// safe from concurrent merge clobbers. The call returns immediately so the
// `messaging-history.set` emit is not blocked.
//
// TRADE-OFF: a listener that fires an outbound send IMMEDIATELY after the emit
// may race the still-pending persistence and get a 463 on that specific send.
// The existing 463 handler in messages-recv.ts triggers a getPrivacyTokens()
// refetch that auto-recovers within seconds. Net result is much better UX than
// per-chunk stalls.
//
// DO NOT add `await` back here without re-evaluating production latency, AND
// DO NOT call storeTcTokensFromHistorySync directly — it must go through the
// chain to preserve write ordering across overlapping chunks.
scheduleHistoryTcTokenSync(data.chats, signalRepository, keyStore, logger)
ev.emit('messaging-history.set', { ev.emit('messaging-history.set', {
...data, ...data,
isLatest: histNotification.syncType !== proto.HistorySync.HistorySyncType.ON_DEMAND ? isLatest : undefined, isLatest: histNotification.syncType !== proto.HistorySync.HistorySyncType.ON_DEMAND ? isLatest : undefined,
+119 -2
View File
@@ -1,12 +1,86 @@
import type { SignalKeyStoreWithTransaction } from '../Types' import type { SignalKeyStoreWithTransaction } from '../Types'
import type { BinaryNode } from '../WABinary' import type { BinaryNode } from '../WABinary'
import { getBinaryNodeChild, getBinaryNodeChildren, isLidUser, jidNormalizedUser } from '../WABinary' import {
getBinaryNodeChild,
getBinaryNodeChildren,
isAnyLidUser,
isAnyPnUser,
isHostedLidUser,
isHostedPnUser,
isJidMetaAI,
isLidUser,
isPnUser,
jidNormalizedUser
} from '../WABinary'
/** 7 days in seconds — matches WA Web AB prop tctoken_duration */ /** 7 days in seconds — matches WA Web AB prop tctoken_duration */
const TC_TOKEN_BUCKET_DURATION = 604800 const TC_TOKEN_BUCKET_DURATION = 604800
/** 4 buckets → ~28-day rolling window — matches WA Web AB prop tctoken_num_buckets */ /** 4 buckets → ~28-day rolling window — matches WA Web AB prop tctoken_num_buckets */
const TC_TOKEN_NUM_BUCKETS = 4 const TC_TOKEN_NUM_BUCKETS = 4
/**
* Sentinel key under the `tctoken` store holding a JSON array of tracked storage JIDs
* for cross-session pruning. Mirrors WA Web's CLEAN_TC_TOKENS index lookup.
*
* Exported so other modules (messages-recv, messages-send, process-message) reference
* the same constant. The fork previously inlined this string in messages-recv.ts;
* keep the value identical (`'__index'`) for backward compatibility with persisted state.
*/
export const TC_TOKEN_INDEX_KEY = '__index'
// Phone-number pattern matching WABinary's isJidBot, applied against the user part so
// the check is invariant to @c.us ↔ @s.whatsapp.net normalization.
const BOT_PHONE_REGEX = /^1313555\d{4}$|^131655500\d{2}$/
/**
* Mirrors WA Web's `Wid.isRegularUser()` (user ¬PSA ¬Bot). Used to gate tctoken
* storage against malformed notifications WA Web filters server-side but we
* defend here for parity with `WAWebSetTcTokenChatAction.handleIncomingTcToken`.
* Works for both pre- and post-normalized JIDs (`@c.us` vs `@s.whatsapp.net`).
*/
export function isRegularUser(jid: string | undefined): boolean {
if (!jid) return false
const user = jid.split('@')[0] ?? ''
if (user === '0') return false // PSA
if (BOT_PHONE_REGEX.test(user)) return false // Bot by phone pattern
if (isJidMetaAI(jid)) return false // MetaAI (@bot server)
return !!(isPnUser(jid) || isLidUser(jid) || isHostedPnUser(jid) || isHostedLidUser(jid) || jid.endsWith('@c.us'))
}
/** Read the persisted tctoken JID index and return its entries (never contains the sentinel key itself). */
export async function readTcTokenIndex(keys: SignalKeyStoreWithTransaction): Promise<string[]> {
const data = await keys.get('tctoken', [TC_TOKEN_INDEX_KEY])
const entry = data[TC_TOKEN_INDEX_KEY]
if (!entry?.token?.length) return []
try {
const parsed = JSON.parse(Buffer.from(entry.token).toString())
if (!Array.isArray(parsed)) return []
return parsed.filter((j): j is string => typeof j === 'string' && j.length > 0 && j !== TC_TOKEN_INDEX_KEY)
} catch {
return []
}
}
/**
* Build a SignalDataSet fragment that writes the merged index (persisted added)
* under the sentinel key. Lets callers update the index without clobbering writes
* made by other layers (history sync, concurrent sessions on the same store).
*/
export async function buildMergedTcTokenIndexWrite(
keys: SignalKeyStoreWithTransaction,
addedJids: Iterable<string>
): Promise<{ [TC_TOKEN_INDEX_KEY]: { token: Buffer } }> {
const persisted = await readTcTokenIndex(keys)
const merged = new Set(persisted)
for (const jid of addedJids) {
if (jid && jid !== TC_TOKEN_INDEX_KEY) merged.add(jid)
}
return {
[TC_TOKEN_INDEX_KEY]: { token: Buffer.from(JSON.stringify([...merged])) }
}
}
/** /**
* Check if a received token is expired using WA Web's rolling bucket algorithm. * Check if a received token is expired using WA Web's rolling bucket algorithm.
* Reference: WAWebTrustedContactsUtils.isTokenExpired * Reference: WAWebTrustedContactsUtils.isTokenExpired
@@ -63,6 +137,43 @@ export async function resolveTcTokenJid(
return lid ?? normalized return lid ?? normalized
} }
/**
* Resolve target JID for issuing privacy token based on AB prop 14303
* (`lid_trusted_token_issue_to_lid`). When the prop is on, issuance goes to
* the LID; when off, it goes to the PN. Returns the original JID if no
* mapping is found in either direction.
*
* Normalizes the JID upfront and uses the `isAny*` helpers so callers can pass
* `@c.us`, `@s.whatsapp.net`, `@hosted`, `@hosted.lid`, `@lid` or device-specific
* forms `LIDMappingStore.getLIDForPN` early-returns unless `isAnyPnUser`,
* so unnormalized inputs would silently bypass routing.
*
* Reference: WAWebTrustedContactsManager.issuePrivacyTokens
*/
export async function resolveIssuanceJid(
jid: string,
issueToLid: boolean,
getLIDForPN: (pn: string) => Promise<string | null>,
getPNForLID?: (lid: string) => Promise<string | null>
): Promise<string> {
const normalized = jidNormalizedUser(jid)
if (issueToLid) {
if (isAnyLidUser(normalized)) return normalized
if (!isAnyPnUser(normalized)) return normalized
const lid = await getLIDForPN(normalized)
return lid ?? normalized
}
if (!isAnyLidUser(normalized)) return normalized
if (getPNForLID) {
const pn = await getPNForLID(normalized)
return pn ?? normalized
}
return normalized
}
type TcTokenParams = { type TcTokenParams = {
jid: string jid: string
baseContent?: BinaryNode[] baseContent?: BinaryNode[]
@@ -135,7 +246,13 @@ export async function storeTcTokensFromIqResult({
continue continue
} }
const rawJid = jidNormalizedUser(tokenNode.attrs.jid || fallbackJid) // In notifications, tokenNode.attrs.jid is OUR own device JID, not the sender's.
// Prefer fallbackJid (resolved from notification's `from` / `sender_lid`) so the
// token is stored under the peer's JID, never under self.
const rawJid = jidNormalizedUser(fallbackJid || tokenNode.attrs.jid)
// Defense against malformed notifications (PSA WID '0', bots, MetaAI). WA Web
// filters these server-side; we mirror Wid.isRegularUser() locally.
if (!isRegularUser(rawJid)) continue
const storageJid = await resolveTcTokenJid(rawJid, getLIDForPN) const storageJid = await resolveTcTokenJid(rawJid, getLIDForPN)
const existingTcData = await keys.get('tctoken', [storageJid]) const existingTcData = await keys.get('tctoken', [storageJid])
const existingEntry = existingTcData[storageJid] const existingEntry = existingTcData[storageJid]
@@ -297,4 +297,52 @@ describe('Identity Change Handling', () => {
expect(result.device).toBe(5) expect(result.device).toBe(5)
}) })
}) })
describe('onBeforeSessionRefresh callback', () => {
it('fires before assertSessions when a session refresh is about to run', async () => {
mockValidateSession.mockResolvedValue({ exists: true })
const callOrder: string[] = []
mockAssertSessions.mockImplementation(async () => {
callOrder.push('assertSessions')
return true
})
const onBeforeSessionRefresh = jest.fn((jid: string) => {
callOrder.push(`before:${jid}`)
})
const node = createIdentityChangeNode('user@s.whatsapp.net')
const ctx = { ...createContext(), onBeforeSessionRefresh }
const result = await handleIdentityChange(node, ctx)
expect(result.action).toBe('session_refreshed')
expect(callOrder).toEqual(['before:user@s.whatsapp.net', 'assertSessions'])
})
it('does not fire when the refresh is skipped (no_identity / offline / self)', async () => {
const onBeforeSessionRefresh = jest.fn()
// no identity node
const noIdentityNode: BinaryNode = {
tag: 'notification',
attrs: { from: 'a@s.whatsapp.net', type: 'encrypt' },
content: []
}
await handleIdentityChange(noIdentityNode, { ...createContext(), onBeforeSessionRefresh })
// offline notification
mockValidateSession.mockResolvedValue({ exists: true })
await handleIdentityChange(createIdentityChangeNode('b@s.whatsapp.net', '0'), {
...createContext(),
onBeforeSessionRefresh
})
// self primary
await handleIdentityChange(createIdentityChangeNode(mockMeId!), {
...createContext(),
onBeforeSessionRefresh
})
expect(onBeforeSessionRefresh).not.toHaveBeenCalled()
})
})
}) })
+348 -1
View File
@@ -1,7 +1,17 @@
import { jest } from '@jest/globals' import { jest } from '@jest/globals'
import { DisconnectReason, type SignalKeyStoreWithTransaction } from '../../Types' import { DisconnectReason, type SignalKeyStoreWithTransaction } from '../../Types'
import { getErrorCodeFromStreamError, SERVER_ERROR_CODES } from '../../Utils' import { getErrorCodeFromStreamError, SERVER_ERROR_CODES } from '../../Utils'
import { buildTcTokenFromJid, isTcTokenExpired, shouldSendNewTcToken } from '../../Utils/tc-token-utils' import {
buildMergedTcTokenIndexWrite,
buildTcTokenFromJid,
isRegularUser,
isTcTokenExpired,
readTcTokenIndex,
resolveIssuanceJid,
shouldSendNewTcToken,
storeTcTokensFromIqResult,
TC_TOKEN_INDEX_KEY
} from '../../Utils/tc-token-utils'
import type { BinaryNode } from '../../WABinary' import type { BinaryNode } from '../../WABinary'
/** 7 days in seconds — matches WA Web tctoken_duration */ /** 7 days in seconds — matches WA Web tctoken_duration */
@@ -848,3 +858,340 @@ describe('tctoken integration scenarios', () => {
}) })
}) })
}) })
// ─── isRegularUser (PSA / bot / MetaAI gating) ─────────────────────────
describe('isRegularUser', () => {
it('rejects undefined / empty', () => {
expect(isRegularUser(undefined)).toBe(false)
expect(isRegularUser('')).toBe(false)
})
it('rejects PSA WID (user "0")', () => {
expect(isRegularUser('0@s.whatsapp.net')).toBe(false)
expect(isRegularUser('0@c.us')).toBe(false)
})
it('rejects bot phone numbers (1313555XXXX, 131655500XX)', () => {
expect(isRegularUser('13135550000@s.whatsapp.net')).toBe(false)
expect(isRegularUser('13135559999@s.whatsapp.net')).toBe(false)
// /^131655500\d{2}$/ — 11 digits: 131655500 + 2 trailing
expect(isRegularUser('13165550000@s.whatsapp.net')).toBe(false)
expect(isRegularUser('13165550099@s.whatsapp.net')).toBe(false)
})
it('rejects MetaAI (@bot server)', () => {
expect(isRegularUser('foo@bot')).toBe(false)
})
it('accepts regular PN users (@s.whatsapp.net, @c.us)', () => {
expect(isRegularUser('5511999999999@s.whatsapp.net')).toBe(true)
expect(isRegularUser('5511999999999@c.us')).toBe(true)
})
it('accepts LID users', () => {
expect(isRegularUser('123456@lid')).toBe(true)
})
it('accepts hosted PN/LID users', () => {
expect(isRegularUser('5511999999999@hosted')).toBe(true)
expect(isRegularUser('123456@hosted.lid')).toBe(true)
})
})
// ─── resolveIssuanceJid (AB prop 14303 routing) ────────────────────────
describe('resolveIssuanceJid', () => {
const PN = '5511999999999@s.whatsapp.net'
const LID = '123456@lid'
const PN_NO_LID = '5511888888888@s.whatsapp.net'
const LID_NO_PN = '999999@lid'
const getLIDForPN = jest.fn<(pn: string) => Promise<string | null>>(async (pn: string) => {
if (pn === PN) return LID
return null
})
const getPNForLID = jest.fn<(lid: string) => Promise<string | null>>(async (lid: string) => {
if (lid === LID) return PN
return null
})
beforeEach(() => {
getLIDForPN.mockClear()
getPNForLID.mockClear()
})
describe('AB prop 14303 ON (issueToLid=true)', () => {
it('returns LID unchanged when input is LID', async () => {
expect(await resolveIssuanceJid(LID, true, getLIDForPN, getPNForLID)).toBe(LID)
expect(getLIDForPN).not.toHaveBeenCalled()
})
it('resolves PN to LID via lidMapping', async () => {
expect(await resolveIssuanceJid(PN, true, getLIDForPN, getPNForLID)).toBe(LID)
expect(getLIDForPN).toHaveBeenCalledWith(PN)
})
it('falls back to original PN when no LID mapping exists', async () => {
expect(await resolveIssuanceJid(PN_NO_LID, true, getLIDForPN, getPNForLID)).toBe(PN_NO_LID)
})
})
describe('AB prop 14303 OFF (issueToLid=false)', () => {
it('returns PN unchanged when input is PN', async () => {
expect(await resolveIssuanceJid(PN, false, getLIDForPN, getPNForLID)).toBe(PN)
expect(getPNForLID).not.toHaveBeenCalled()
})
it('resolves LID to PN via lidMapping', async () => {
expect(await resolveIssuanceJid(LID, false, getLIDForPN, getPNForLID)).toBe(PN)
expect(getPNForLID).toHaveBeenCalledWith(LID)
})
it('falls back to original LID when no PN mapping exists', async () => {
expect(await resolveIssuanceJid(LID_NO_PN, false, getLIDForPN, getPNForLID)).toBe(LID_NO_PN)
})
it('returns LID unchanged when getPNForLID is omitted', async () => {
expect(await resolveIssuanceJid(LID, false, getLIDForPN)).toBe(LID)
})
})
describe('JID normalization (handles @c.us and hosted forms)', () => {
const PN_CUS = '5511999999999@c.us'
const PN_NORMALIZED = '5511999999999@s.whatsapp.net'
const HOSTED_LID = '999999@hosted.lid'
it('normalizes @c.us before resolving (issueToLid=true)', async () => {
const fn = jest.fn<(pn: string) => Promise<string | null>>(async (pn: string) => {
if (pn === PN_NORMALIZED) return LID
return null
})
const result = await resolveIssuanceJid(PN_CUS, true, fn, getPNForLID)
expect(result).toBe(LID)
// Mapping store called with NORMALIZED form
expect(fn).toHaveBeenCalledWith(PN_NORMALIZED)
})
it('treats hosted LID as LID input (issueToLid=true returns it unchanged)', async () => {
expect(await resolveIssuanceJid(HOSTED_LID, true, getLIDForPN, getPNForLID)).toBe(HOSTED_LID)
expect(getLIDForPN).not.toHaveBeenCalled()
})
it('treats hosted LID as LID input (issueToLid=false converts via getPNForLID)', async () => {
const fn = jest.fn<(lid: string) => Promise<string | null>>(async () => null)
await resolveIssuanceJid(HOSTED_LID, false, getLIDForPN, fn)
expect(fn).toHaveBeenCalledWith(HOSTED_LID)
})
})
})
// ─── readTcTokenIndex / buildMergedTcTokenIndexWrite ───────────────────
describe('tctoken cross-session prune index', () => {
it('exports the sentinel key as "__index"', () => {
// Persisted state compatibility — tests freeze the value to catch unintended renames.
expect(TC_TOKEN_INDEX_KEY).toBe('__index')
})
describe('readTcTokenIndex', () => {
it('returns [] when the index entry is absent', async () => {
const keys = createMockKeys()
;(keys.get as any).mockResolvedValue({})
expect(await readTcTokenIndex(keys)).toEqual([])
})
it('returns [] when the index token buffer is empty', async () => {
const keys = createMockKeys()
;(keys.get as any).mockResolvedValue({ [TC_TOKEN_INDEX_KEY]: { token: Buffer.alloc(0) } })
expect(await readTcTokenIndex(keys)).toEqual([])
})
it('returns [] when the JSON payload is corrupted', async () => {
const keys = createMockKeys()
;(keys.get as any).mockResolvedValue({ [TC_TOKEN_INDEX_KEY]: { token: Buffer.from('not-json') } })
expect(await readTcTokenIndex(keys)).toEqual([])
})
it('returns [] when the JSON payload is not an array', async () => {
const keys = createMockKeys()
;(keys.get as any).mockResolvedValue({ [TC_TOKEN_INDEX_KEY]: { token: Buffer.from('{"a":1}') } })
expect(await readTcTokenIndex(keys)).toEqual([])
})
it('returns the persisted JIDs', async () => {
const keys = createMockKeys()
const stored = ['a@lid', 'b@lid', 'c@s.whatsapp.net']
;(keys.get as any).mockResolvedValue({
[TC_TOKEN_INDEX_KEY]: { token: Buffer.from(JSON.stringify(stored)) }
})
expect(await readTcTokenIndex(keys)).toEqual(stored)
})
it('filters out the sentinel key itself, empty strings, and non-strings', async () => {
const keys = createMockKeys()
const stored = ['a@lid', '', TC_TOKEN_INDEX_KEY, 42, null, 'b@lid']
;(keys.get as any).mockResolvedValue({
[TC_TOKEN_INDEX_KEY]: { token: Buffer.from(JSON.stringify(stored)) }
})
expect(await readTcTokenIndex(keys)).toEqual(['a@lid', 'b@lid'])
})
})
describe('buildMergedTcTokenIndexWrite', () => {
it('merges added JIDs with the persisted set (de-duplicated)', async () => {
const keys = createMockKeys()
;(keys.get as any).mockResolvedValue({
[TC_TOKEN_INDEX_KEY]: { token: Buffer.from(JSON.stringify(['a@lid', 'b@lid'])) }
})
const write = await buildMergedTcTokenIndexWrite(keys, ['b@lid', 'c@lid'])
const decoded = JSON.parse(write[TC_TOKEN_INDEX_KEY].token.toString()) as string[]
expect(decoded.sort()).toEqual(['a@lid', 'b@lid', 'c@lid'])
})
it('drops the sentinel key from the added set', async () => {
const keys = createMockKeys()
;(keys.get as any).mockResolvedValue({})
const write = await buildMergedTcTokenIndexWrite(keys, [TC_TOKEN_INDEX_KEY, 'a@lid'])
const decoded = JSON.parse(write[TC_TOKEN_INDEX_KEY].token.toString()) as string[]
expect(decoded).toEqual(['a@lid'])
})
it('handles empty inputs', async () => {
const keys = createMockKeys()
;(keys.get as any).mockResolvedValue({})
const write = await buildMergedTcTokenIndexWrite(keys, [])
const decoded = JSON.parse(write[TC_TOKEN_INDEX_KEY].token.toString()) as string[]
expect(decoded).toEqual([])
})
})
})
// ─── storeTcTokensFromIqResult — isRegularUser gating + monotonicity ───
describe('storeTcTokensFromIqResult — gating and monotonicity', () => {
const PEER_PN = '5511999999999@s.whatsapp.net'
const PEER_LID = '123456@lid'
const getLIDForPN = jest.fn<(pn: string) => Promise<string | null>>(async (pn: string) => {
if (pn === PEER_PN) return PEER_LID
return null
})
beforeEach(() => {
getLIDForPN.mockClear()
})
const buildIqResult = (jid: string, tokenBytes: Uint8Array, t: string): BinaryNode => ({
tag: 'iq',
attrs: {},
content: [
{
tag: 'tokens',
attrs: {},
content: [
{
tag: 'token',
attrs: { jid, type: 'trusted_contact', t },
content: tokenBytes
}
]
}
]
})
it('skips PSA WID (jid="0") even if returned in tokens block', async () => {
const keys = createMockKeys()
;(keys.get as any).mockResolvedValue({})
await storeTcTokensFromIqResult({
result: buildIqResult('myDeviceJid@s.whatsapp.net', Buffer.from([0x01, 0x02]), '1700000000'),
fallbackJid: '0@s.whatsapp.net',
keys,
getLIDForPN
})
expect(keys.set).not.toHaveBeenCalled()
})
it('skips bot phone numbers', async () => {
const keys = createMockKeys()
;(keys.get as any).mockResolvedValue({})
await storeTcTokensFromIqResult({
result: buildIqResult('myDeviceJid@s.whatsapp.net', Buffer.from([0x01, 0x02]), '1700000000'),
fallbackJid: '13135550000@s.whatsapp.net',
keys,
getLIDForPN
})
expect(keys.set).not.toHaveBeenCalled()
})
it('skips MetaAI (@bot server)', async () => {
const keys = createMockKeys()
;(keys.get as any).mockResolvedValue({})
await storeTcTokensFromIqResult({
result: buildIqResult('myDeviceJid@s.whatsapp.net', Buffer.from([0x01, 0x02]), '1700000000'),
fallbackJid: 'meta-ai@bot',
keys,
getLIDForPN
})
expect(keys.set).not.toHaveBeenCalled()
})
it('uses fallbackJid (NOT the token node jid which is own device) as storage source', async () => {
const keys = createMockKeys()
;(keys.get as any).mockResolvedValue({})
const onNewJidStored = jest.fn()
await storeTcTokensFromIqResult({
result: buildIqResult('myDeviceJid:1@s.whatsapp.net', Buffer.from([0xaa, 0xbb]), '1700000000'),
fallbackJid: PEER_PN,
keys,
getLIDForPN,
onNewJidStored
})
expect(keys.set).toHaveBeenCalled()
const setArgs = (keys.set.mock.calls[0]?.[0] as { tctoken: Record<string, unknown> }).tctoken
expect(Object.keys(setArgs)).toContain(PEER_LID)
expect(onNewJidStored).toHaveBeenCalledWith(PEER_LID)
})
it('skips when incoming timestamp is older than existing (monotonicity)', async () => {
const keys = createMockKeys()
;(keys.get as any).mockResolvedValue({
[PEER_LID]: { token: Buffer.from([0x99]), timestamp: '1800000000' }
})
await storeTcTokensFromIqResult({
result: buildIqResult('myDeviceJid@s.whatsapp.net', Buffer.from([0xaa]), '1700000000'),
fallbackJid: PEER_PN,
keys,
getLIDForPN
})
expect(keys.set).not.toHaveBeenCalled()
})
it('skips timestamp-less tokens (would be immediately expired)', async () => {
const keys = createMockKeys()
;(keys.get as any).mockResolvedValue({})
await storeTcTokensFromIqResult({
result: {
tag: 'iq',
attrs: {},
content: [
{
tag: 'tokens',
attrs: {},
content: [
{
tag: 'token',
attrs: { jid: 'd@s.whatsapp.net', type: 'trusted_contact' },
content: Buffer.from([0x01])
}
]
}
]
},
fallbackJid: PEER_PN,
keys,
getLIDForPN
})
expect(keys.set).not.toHaveBeenCalled()
})
})