Compare commits

..

3 Commits

Author SHA1 Message Date
Renato Alcara 170097cf56 perf(logs): suppress libsignal console.warn — the missing piece of stdout flood
Following commits 1734d10209 + 9d79bbe351 the user reported pm2 logs were
STILL flooding and inbound latency persisted. Inspecting libsignal source
reveals 4 console.warn calls that bypass our existing log/info/error
interceptors:

  session_builder.js:74 → "Closing open session in favor of incoming prekey bundle"
  session_record.js:270 → "Session already closed", <session object>
  session_cipher.js:182 → "Decrypted message with closed session."
  queue_job.js:50      → "Unhandled bucket type (for naming):", <bucket>

Plus we were missing 2 console.info paths from session_record.js:
  - "Opening session:" (dumps the full session object)
  - "Migrating session to:" (per migration)

Under WhatsApp's LID/DSM rollout the pkmsg flow fires "Closing open session"
on EVERY incoming pkmsg (which is every recovery message from the user's own
phone after a Bad MAC). The user's log shows hundreds of these per session.
Each is a synchronous stdout.write. pm2's pipe buffer fills, write() blocks
the Node event loop, inbound delivery freezes for tens of seconds. Profile
pictures and IQ queries time out for the same reason — they need the loop
to dispatch them.

This is the same root cause as the historical Feb-27 fix (d233a7856f, "fix:
eliminate 40s message delivery delay caused by libsignal console dumps")
but the warn channel was overlooked.

THIS PATCH:
- Add console.warn interceptor with the same suppression regex.
- Extend the regex to cover the missing patterns (Opening session,
  Migrating session, Closing open session, Session already closed,
  Decrypted message with closed session, Unhandled bucket type).
- Operator-facing signal is unchanged — the console.error path still
  formats Bad MAC / Counter / Decryption Failed as the clean emoji line.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-30 13:00:41 -03:00
Renato Alcara 9d79bbe351 perf(logs): silence MessageCounterError + slim stanza dump on handler errors
Following PR #402 / fix/silence-decrypt-error-spam, prod logs still showed
two heavy stdout-saturation sources that survived the previous pass:

1) auth-utils.ts:353 — `transaction failed, rolling back` fired at ERROR
   level for every `MessageCounterError` ("Key used already or never filled").
   Only `SessionError` was downgraded. Under WhatsApp's LID/DSM rollout,
   own DSM messages flood with MessageCounterError when the auth state has
   a legacy `_1.0` session that no longer matches the LID-addressed
   envelope. ~30+ JSON ERROR lines per minute hit stdout.

2) messages-recv.ts:2698 — `error in handling message` logged the FULL
   stanza XML via `binaryNodeToString(node)`. Each stanza contains the
   inline encrypted ciphertext — kilobytes per log entry. Under load (e.g.
   downstream consumer throws on every Bad MAC), this is THE biggest
   single source of stdout pressure.

Pedro's snapshot doesn't have these specifically — its libsignal failures
fall through to upstream's leaner logging path.

THIS PATCH:
- auth-utils: extend the recoverable-error downgrade to MessageCounterError
  too (same recovery path: retry receipt → pkmsg → new session). Both now
  log at debug; everything else still ERRORs.
- messages-recv: log a SLIM node projection (tag + id + from + type +
  participant) at error level. Full stanza XML moved to debug. Operators
  retain enough to pivot to the offending message; investigations via
  BAILEYS_LOG_LEVEL=debug still get the full payload.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-30 12:40:02 -03:00
Renato Alcara 1734d10209 perf(decrypt): silence per-attempt warn logs that saturate stdout
PRODUCTION BUG (still present after PR #396, #402): inbound 60s latency.
Diff vs Pedro snapshot pinpointed the residual cause: pino warn-level
logs of `errorContext` (key + jid + err + sender + author + decryptionJid
+ flags) on EVERY decrypt failure attempt.

Under WA's LID/DSM rollout the user's own DSM floods the inbound pipeline
with Bad MAC / "Key used already" / "No session record" errors when the
auth state has a legacy `_1.0` session that no longer matches the
LID-addressed envelope. At ~30 failed attempts/min this is ~30 JSON lines/sec
of full key+context dumps. pm2 writes stdout synchronously; once the pipe
buffer fills, `process.stdout.write` BLOCKS the event loop. Inbound
delivery, query responses, profile-pic loads — everything stalls.

Pedro doesn't have this rich logging; libsignal's own console.error is
already intercepted in src/index.ts and reformatted as a single emoji line
("🔐 Bad MAC Error | JID: 1940****_1.0"). The duplicated pino warn was
pure noise.

THIS PATCH:
- Per-attempt warn → debug. With BAILEYS_LOG_LEVEL=warn (production) these
  drop entirely. With LEVEL=debug they still surface for active
  troubleshooting.
- Retry-exhausted warn (rare) keeps a SLIM context: msgId + jid + slimErr
  + attempts. No more full key/sender/author/decryptionJid dump.
- error → warn for retry-exhausted (level 50 is reserved for unknown
  errors that fall through to the else branch — those are real bugs).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-30 12:04:06 -03:00
18 changed files with 197 additions and 25 deletions
+6
View File
@@ -343,11 +343,13 @@ export const extractGroupMetadata = (result: BinaryNode) => {
let descId: string | undefined
let descOwner: string | undefined
let descOwnerPn: string | undefined
let descOwnerUsername: string | undefined
let descTime: number | undefined
if (descChild) {
desc = getBinaryNodeChildString(descChild, 'body')
descOwner = descChild.attrs.participant ? jidNormalizedUser(descChild.attrs.participant) : undefined
descOwnerPn = descChild.attrs.participant_pn ? jidNormalizedUser(descChild.attrs.participant_pn) : undefined
descOwnerUsername = descChild.attrs.participant_username || undefined
descTime = +descChild.attrs.t!
descId = descChild.attrs.id
}
@@ -362,16 +364,19 @@ export const extractGroupMetadata = (result: BinaryNode) => {
subject: group.attrs.subject!,
subjectOwner: group.attrs.s_o,
subjectOwnerPn: group.attrs.s_o_pn,
subjectOwnerUsername: group.attrs.s_o_username,
subjectTime: +(group.attrs.s_t ?? '0'),
size: group.attrs.size ? +group.attrs.size : getBinaryNodeChildren(group, 'participant').length,
creation: +(group.attrs.creation ?? '0'),
owner: group.attrs.creator ? jidNormalizedUser(group.attrs.creator) : undefined,
ownerPn: group.attrs.creator_pn ? jidNormalizedUser(group.attrs.creator_pn) : undefined,
ownerUsername: group.attrs.creator_username || undefined,
owner_country_code: group.attrs.creator_country_code,
desc,
descId,
descOwner,
descOwnerPn,
descOwnerUsername,
descTime,
linkedParent: getBinaryNodeChild(group, 'linked_parent')?.attrs.jid || undefined,
restrict: !!getBinaryNodeChild(group, 'locked'),
@@ -386,6 +391,7 @@ export const extractGroupMetadata = (result: BinaryNode) => {
id: attrs.jid!,
phoneNumber: isLidUser(attrs.jid) && isPnUser(attrs.phone_number) ? attrs.phone_number : undefined,
lid: isPnUser(attrs.jid) && isLidUser(attrs.lid) ? attrs.lid : undefined,
username: attrs.participant_username || attrs.username || undefined,
admin: (attrs.type || null) as GroupParticipant['admin']
}
}),
+19 -1
View File
@@ -2240,6 +2240,7 @@ export const makeMessagesRecvSocket = (config: SocketConfig) => {
fromMe,
participant: node.attrs.participant,
participantAlt,
participantUsername: node.attrs.participant_username,
addressingMode,
id: node.attrs.id,
...(msg.key || {})
@@ -2248,6 +2249,10 @@ export const makeMessagesRecvSocket = (config: SocketConfig) => {
msg.messageTimestamp = +node.attrs.t!
const fullMsg = proto.WebMessageInfo.fromObject(msg) as WAMessage
// Preserve custom WAMessageKey fields (participantAlt, participantUsername,
// addressingMode) that proto.WebMessageInfo.fromObject strips because
// they aren't part of the proto.MessageKey schema.
fullMsg.key = msg.key
await upsertMessage(fullMsg, 'append')
}
})
@@ -2690,7 +2695,20 @@ export const makeMessagesRecvSocket = (config: SocketConfig) => {
}
})
} catch (error) {
logger.error({ error, node: binaryNodeToString(node) }, 'error in handling message')
// Slim log: identify the message that crashed without dumping the full
// stanza XML (which contains kilobytes of inline ciphertext). Under heavy
// load — e.g. a downstream consumer that throws on every Bad MAC — the
// full XML dumps overwhelm stdout and stall the event loop. Operators
// can still pivot on msgId / from / type; full payload at debug level.
const slimNode = {
tag: node.tag,
id: node.attrs?.id,
from: node.attrs?.from,
type: node.attrs?.type,
participant: node.attrs?.participant
}
logger.error({ error, node: slimNode }, 'error in handling message')
logger.debug({ node: binaryNodeToString(node) }, 'error in handling message — full stanza')
}
}
+2
View File
@@ -9,6 +9,8 @@ export interface Contact {
name?: string
/** name of the contact, the contact has set on their own on WA */
notify?: string
/** username associated with this contact, when provided by WA */
username?: string
/** I have no idea */
verifiedName?: string
// Baileys Added
+2
View File
@@ -81,6 +81,7 @@ export type BaileysEventMap = {
id: string
author: string
authorPn?: string
authorUsername?: string
participants: GroupParticipant[]
action: ParticipantAction
}
@@ -88,6 +89,7 @@ export type BaileysEventMap = {
id: string
author: string
authorPn?: string
authorUsername?: string
participant: string
participantPn?: string
action: RequestJoinAction
+4
View File
@@ -20,17 +20,20 @@ export interface GroupMetadata {
addressingMode?: WAMessageAddressingMode
owner: string | undefined
ownerPn?: string | undefined
ownerUsername?: string | undefined
owner_country_code?: string | undefined
subject: string
/** group subject owner */
subjectOwner?: string
subjectOwnerPn?: string
subjectOwnerUsername?: string
/** group subject modification date */
subjectTime?: number
creation?: number
desc?: string
descOwner?: string
descOwnerPn?: string
descOwnerUsername?: string
descId?: string
descTime?: number
/** if this group is part of a community, it returns the jid of the community to which it belongs */
@@ -56,6 +59,7 @@ export interface GroupMetadata {
/** the person who added you to group or changed some setting in group */
author?: string
authorPn?: string
authorUsername?: string
}
export interface WAGroupCreateResponse {
+2
View File
@@ -19,7 +19,9 @@ export type WAContactMessage = proto.Message.IContactMessage
export type WAContactsArrayMessage = proto.Message.IContactsArrayMessage
export type WAMessageKey = proto.IMessageKey & {
remoteJidAlt?: string
remoteJidUsername?: string
participantAlt?: string
participantUsername?: string
server_id?: string
addressingMode?: string
isViewOnce?: boolean // TODO: remove out of the message key, place in WebMessageInfo
+9 -7
View File
@@ -341,14 +341,16 @@ export const addTransactionCapability = (
return result
} catch (error) {
// SessionError is part of the normal Bad MAC recovery flow
// (retry receipt → sender resends as pkmsg → new session within ~1.3s).
// Logging it as ERROR creates 2 noise lines per recoverable Bad MAC cycle.
// Downgrade to debug for SessionError; keep ERROR for everything else.
// The error is still re-thrown — recovery behavior is unchanged.
// SessionError / MessageCounterError are part of the normal Bad MAC
// recovery flow (retry receipt → sender resends as pkmsg → new session
// within ~1.3s). Logging them as ERROR creates 2 noise lines per
// recoverable cycle, and under WhatsApp's LID/DSM rollout these fire
// in dense bursts that saturate stdout (synchronous pm2 writes block
// the event loop). Downgrade to debug for both; keep ERROR for
// everything else. The error is still re-thrown — recovery is unchanged.
const errName = (error as { name?: string })?.name
if (errName === 'SessionError') {
logger.debug({ error }, 'transaction failed (SessionError — recoverable via retry receipt)')
if (errName === 'SessionError' || errName === 'MessageCounterError') {
logger.debug({ error }, `transaction failed (${errName} — recoverable via retry receipt)`)
} else {
logger.error({ error }, 'transaction failed, rolling back')
}
+1
View File
@@ -1083,6 +1083,7 @@ export const processSyncAction = (
action.lidContactAction.firstName ||
action.lidContactAction.username ||
undefined,
username: action.lidContactAction.username || undefined,
lid: id!,
phoneNumber: undefined
}
+33 -8
View File
@@ -246,10 +246,14 @@ export function decodeMessageNode(stanza: BinaryNode, meId: string, meLid: strin
const key: WAMessageKey = {
remoteJid: chatId,
remoteJidAlt: !isJidGroup(chatId) ? addressingContext.senderAlt : undefined,
remoteJidUsername: !isJidGroup(chatId)
? stanza.attrs.peer_recipient_username || stanza.attrs.recipient_username
: undefined,
fromMe,
id: msgId,
participant,
participantAlt: isJidGroup(chatId) ? addressingContext.senderAlt : undefined,
participantUsername: stanza.attrs.participant ? stanza.attrs.participant_username : undefined,
addressingMode: addressingContext.addressingMode,
...(msgType === 'newsletter' && stanza.attrs.server_id ? { server_id: stanza.attrs.server_id } : {})
}
@@ -450,19 +454,40 @@ export const decryptMessageNode = (
...(isRetryExhausted && { retriesExhausted: true, attempts: err.attempts })
}
// Smart logging based on error type and retry status
// Smart logging based on error type and retry status.
//
// PERF NOTE: under WhatsApp's LID/DSM rollout, own DSM messages flood
// the inbound pipeline with Bad MAC / "Key used already" / "No session
// record" errors *every* time the auth state has a session in the
// legacy `_1.0` format that no longer matches the LID-addressed
// envelope. Logging the full errorContext (key + jid + err) per failed
// attempt as warn-level produces tens of JSON lines/sec, which
// saturates stdout in pm2 (synchronous writes to a full pipe block the
// event loop). The clean `🔐 Bad MAC Error | JID: …` line emitted by
// the console.error interceptor in src/index.ts already gives an
// operator-visible signal — the duplicated pino line was pure noise.
//
// We now only emit warn-level when retries are exhausted (rare,
// actionable). Per-attempt detail is still available at debug level
// (BAILEYS_LOG_LEVEL=debug) for active troubleshooting.
const slimErrorContext = {
msgId: fullMessage.key?.id,
jid: fullMessage.key?.remoteJid,
err: slimErr,
attempts: isRetryExhausted ? err.attempts : 1
}
if (isCorrupted) {
// Corrupted session errors are expected and auto-recovered
// Only log as ERROR if retries exhausted, otherwise WARN on first attempt
// Corrupted session errors are expected — Signal Protocol auto-recovers
// via retry receipt → pkmsg → new session.
// eslint-disable-next-line max-depth
if (isRetryExhausted) {
logger.error(
errorContext,
logger.warn(
slimErrorContext,
`⚠️ Session corrupted after ${err.attempts} attempts. Retry+pkmsg flow will recover.`
)
} else {
// First occurrence - log as warning since auto-recovery will attempt
logger.warn(errorContext, '⚠️ Corrupted session detected - attempting auto-recovery')
logger.debug(errorContext, '⚠️ Corrupted session detected - attempting auto-recovery')
}
// Session cleanup is deferred to retry exhaustion (safety net).
@@ -475,7 +500,7 @@ export const decryptMessageNode = (
// Session record errors are transient - retry should handle them
// eslint-disable-next-line max-depth
if (isRetryExhausted) {
logger.error(errorContext, `Failed to decrypt: No session record found after ${err.attempts} attempts`)
logger.warn(slimErrorContext, `Failed to decrypt: No session record found after ${err.attempts} attempts`)
} else {
logger.debug(errorContext, 'No session record - will retry')
}
+1
View File
@@ -315,6 +315,7 @@ export const processHistoryMessage = (item: proto.IHistorySync, logger?: ILogger
contacts.push({
id: chatId,
name: chat.displayName || chat.name || chat.username || undefined,
username: chat.username || undefined,
lid: chat.lidJid || chat.accountLid || undefined,
phoneNumber: chat.pnJid || undefined
})
+9 -1
View File
@@ -950,12 +950,19 @@ const processMessage = async (
id: jid,
author: message.key.participant!,
authorPn: message.key.participantAlt!,
authorUsername: message.key.participantUsername!,
participants,
action
})
const emitGroupUpdate = (update: Partial<GroupMetadata>) => {
ev.emit('groups.update', [
{ id: jid, ...update, author: message.key.participant ?? undefined, authorPn: message.key.participantAlt }
{
id: jid,
...update,
author: message.key.participant ?? undefined,
authorPn: message.key.participantAlt,
authorUsername: message.key.participantUsername
}
])
}
@@ -964,6 +971,7 @@ const processMessage = async (
id: jid,
author: message.key.participant!,
authorPn: message.key.participantAlt!,
authorUsername: message.key.participantUsername!,
participant: participant.lid,
participantPn: participant.pn,
action,
+1
View File
@@ -46,6 +46,7 @@ export const processContactAction = (
{
id,
name: action.fullName || action.firstName || action.username || undefined,
username: action.username || undefined,
lid: lidJid || undefined,
phoneNumber
}
+29 -3
View File
@@ -13,11 +13,37 @@ export class USyncContactProtocol implements USyncQueryProtocol {
}
getUserElement(user: USyncUser): BinaryNode {
//TODO: Implement type / username fields (not yet supported)
if (user.phone) {
return {
tag: 'contact',
attrs: {},
content: user.phone
}
}
if (user.username) {
return {
tag: 'contact',
attrs: {
username: user.username,
...(user.usernameKey ? { pin: user.usernameKey } : {}),
...(user.lid ? { lid: user.lid } : {})
}
}
}
if (user.type) {
return {
tag: 'contact',
attrs: {
type: user.type
}
}
}
return {
tag: 'contact',
attrs: {},
content: user.phone
attrs: {}
}
}
@@ -0,0 +1,28 @@
import type { USyncQueryProtocol } from '../../Types/USync'
import { assertNodeErrorFree, type BinaryNode } from '../../WABinary'
import { USyncUser } from '../USyncUser'
export class USyncUsernameProtocol implements USyncQueryProtocol {
name = 'username'
getQueryElement(): BinaryNode {
return {
tag: 'username',
attrs: {}
}
}
getUserElement(user: USyncUser): BinaryNode | null {
void user
return null
}
parser(node: BinaryNode): string | null {
if (node.tag === 'username') {
assertNodeErrorFree(node)
return typeof node.content === 'string' ? node.content : null
}
return null
}
}
+1
View File
@@ -2,3 +2,4 @@ export * from './USyncDeviceProtocol'
export * from './USyncContactProtocol'
export * from './USyncStatusProtocol'
export * from './USyncDisappearingModeProtocol'
export * from './USyncUsernameProtocol'
+7 -1
View File
@@ -6,7 +6,8 @@ import {
USyncContactProtocol,
USyncDeviceProtocol,
USyncDisappearingModeProtocol,
USyncStatusProtocol
USyncStatusProtocol,
USyncUsernameProtocol
} from './Protocols'
import { USyncUser } from './USyncUser'
@@ -137,4 +138,9 @@ export class USyncQuery {
this.protocols.push(new USyncLIDProtocol())
return this
}
withUsernameProtocol() {
this.protocols.push(new USyncUsernameProtocol())
return this
}
}
+12
View File
@@ -2,6 +2,8 @@ export class USyncUser {
id?: string
lid?: string
phone?: string
username?: string
usernameKey?: string
type?: string
personaId?: string
@@ -20,6 +22,16 @@ export class USyncUser {
return this
}
withUsername(username: string) {
this.username = username
return this
}
withUsernameKey(usernameKey: string) {
this.usernameKey = usernameKey
return this
}
withType(type: string) {
this.type = type
return this
+31 -4
View File
@@ -5,11 +5,30 @@
const _origConsoleError = console.error
const _origConsoleLog = console.log
const _origConsoleInfo = console.info
const _origConsoleWarn = console.warn
// Suppress libsignal session lifecycle dumps from console.log / console.info / console.warn.
// libsignal's session_record.js / session_builder.js / session_cipher.js use:
// console.info("Removing old closed session:", obj) ← ~500ms I/O dump per call
// console.info("Opening session:", obj) ← ~500ms I/O dump per call
// console.info("Migrating session to:", v) ← per migration
// console.log("Closing session:", obj) ← ~500ms I/O dump per call
// console.warn("Closing open session in favor of incoming prekey bundle") ← per pkmsg
// console.warn("Session already closed", obj) ← per stale close
// console.warn("Decrypted message with closed session.") ← per recovered decrypt
// console.warn("Unhandled bucket type (for naming):", ...) ← queue_job edge case
//
// Under WhatsApp's LID/DSM rollout these fire dozens of times per minute when
// the auth state has legacy `_1.0` sessions that don't match LID-addressed
// envelopes. Each dump is a synchronous stdout.write — pm2 buffers fill and
// the event loop blocks. Symptom: 60s inbound delivery latency, profile
// pictures don't load, queries time out.
//
// The clean operator-facing signal lives in the console.error interceptor
// below (formats Bad MAC / Counter / Decryption Failed as one emoji line).
const _SESSION_LIFECYCLE_RE =
/^(Closing session|Removing old closed session|Opening session|Migrating session|Closing open session|Session already closed|Decrypted message with closed session|Unhandled bucket type)/
// Suppress libsignal session lifecycle dumps from console.log / console.info.
// libsignal's session_record.js uses console.info("Removing old closed session:", obj)
// and console.log("Closing session:", obj) which dump full session objects (~500ms I/O each).
const _SESSION_LIFECYCLE_RE = /^(Closing session|Removing old closed session)/
console.log = function (...args: unknown[]) {
if (args.length > 0 && typeof args[0] === 'string' && _SESSION_LIFECYCLE_RE.test(args[0])) {
return
@@ -26,6 +45,14 @@ console.info = function (...args: unknown[]) {
_origConsoleInfo.apply(console, args)
}
console.warn = function (...args: unknown[]) {
if (args.length > 0 && typeof args[0] === 'string' && _SESSION_LIFECYCLE_RE.test(args[0])) {
return
}
_origConsoleWarn.apply(console, args)
}
// Track errors by type + JID to avoid duplicates (using Map for better performance)
const _errorTimestamps = new Map<string, number>()
// Dedup window for repeated decrypt-error console lines (Bad MAC / Counter / etc).