Compare commits

...

2 Commits

Author SHA1 Message Date
Renato Alcara f0ee205f96 fix(waproto): normalize high to int32 + tighten roundtrip test
Addresses 2 review comments on PR #385:

1. CodeRabbit Major — longToString/longToNumber checked
   `value.high < 0` on the raw input. When a Long-like object carries
   high as an unsigned 32-bit value (e.g. raw {low, high} from JSON
   deserialization, not constructed via Long.fromBits/fromValue which
   normalize via `| 0`), the sign-bit detection fails and the value is
   interpreted as unsigned. Example: {low: 0, high: 0xFFFFFFFF} signed
   should be -4294967296 but returned 18446744069414584320.

   Fix: `const high = value.high | 0` before the sign check. Applied
   to both helpers in WAProto/index.js (generated artifact) and
   WAProto/fix-imports.js (template). Diverges from upstream Baileys
   PR #2333 which has the same latent bug.

   Added 2 test cases in bigint-validation.test.ts for the raw
   {low, high} scenario (Long.fromBits would mask the bug because it
   normalizes internally).

2. CodeRabbit Nitpick — the encode/decode roundtrip test in
   proto-tojson-long.test.ts used safe integers (1700000000 and
   123456789), which would silently pass even if longToString fell
   back to Number(value). Switched to uint64 values above
   MAX_SAFE_INTEGER (max uint64 and 9999999999999999999) so the
   roundtrip actually exercises the BigInt fast path.

Test suite: 789/789 passing (+2 cases for the unsigned-high scenario).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-25 15:06:16 -03:00
Renato Alcara 5d7840a66c perf: optimize history sync memory and CPU usage
Aligns with Baileys upstream PR #2333 (jlucaso1). Surgical port of
4 micro-optimizations + 2 test files. No collision with InfiniteAPI's
LID/PN, carousel, buttons, Bad MAC or app state sync resilience code.

Changes:

1. WAProto longToString/longToNumber: use native BigInt instead of
   Long library division loops. ~4.6x faster per call (237ns -> 52ns).
   BigInt.toString() delegates to V8 native C++. Same change applied
   to fix-imports.js (template) and index.js (generated artifact) so
   future regenerations stay in sync.

2. downloadHistory: stream-pipe createInflate instead of buffering
   the full compressed payload then inflating. Cuts RSS peak ~50%
   on 50MB history payloads.

3. processHistoryMessage: drop the { ...chat } shallow copy before
   pushing to chats[]. The decoded protobuf object is not referenced
   after the push.

4. downloadEncryptedContent transform: skip Buffer.concat when
   remainingBytes is empty (the common case — chunks usually arrive
   AES-aligned). ~19x faster per chunk (78ms -> 4ms over 25k chunks).

Tests:
- bigint-validation.test.ts (NEW): 32 parameterized cases covering
  zero, MAX_SAFE_INTEGER boundary, max int64/uint64, negative values,
  tricky bit patterns and powers of 2; plus 200-iter fuzz comparing
  old Long vs new BigInt implementations for equivalence.
- proto-tojson-long.test.ts: kept the existing fork-only test for
  string-in-Long-field graceful handling, appended 4 upstream tests
  covering Long->string fast path, large unsigned > MAX_SAFE_INTEGER,
  zero/small values and encode/decode roundtrip.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-25 14:27:10 -03:00
6 changed files with 361 additions and 48 deletions
+26 -18
View File
@@ -18,14 +18,21 @@ try {
'\tif (typeof value === "number") {\n' +
'\t\treturn String(value);\n' +
'\t}\n' +
'\tif (!$util.Long) {\n' +
'\t\treturn String(value);\n' +
'\t// Fast path: convert Long {low, high} directly via native BigInt\n' +
'\t// BigInt.toString() is a native C++ operation, much faster than Long\'s pure JS division loops\n' +
'\tif (value && typeof value.low === "number" && typeof value.high === "number") {\n' +
'\t\t// Normalize high to signed int32 so the sign-bit check works for inputs\n' +
'\t\t// where high is stored unsigned (e.g. raw {low, high} JSON).\n' +
'\t\tconst high = value.high | 0;\n' +
'\t\tconst lo = BigInt(value.low >>> 0);\n' +
'\t\tconst hi = BigInt(value.high >>> 0);\n' +
'\t\tconst combined = (hi << 32n) | lo;\n' +
'\t\tif (!unsigned && high < 0) {\n' +
'\t\t\treturn (combined - (1n << 64n)).toString();\n' +
'\t\t}\n' +
'\t\treturn combined.toString();\n' +
'\t}\n' +
'\tconst normalized = $util.Long.fromValue(value);\n' +
'\tconst prepared = unsigned && normalized && typeof normalized.toUnsigned === "function"\n' +
'\t\t? normalized.toUnsigned()\n' +
'\t\t: normalized;\n' +
'\treturn prepared.toString();\n' +
'\treturn String(value);\n' +
'}\n\n'
const longToNumberHelper =
'function longToNumber(value, unsigned) {\n' +
@@ -33,19 +40,20 @@ try {
'\t\treturn value;\n' +
'\t}\n' +
'\tif (typeof value === "string") {\n' +
'\t\tconst numeric = Number(value);\n' +
'\t\treturn numeric;\n' +
'\t}\n' +
'\tif (!$util.Long) {\n' +
'\t\treturn Number(value);\n' +
'\t}\n' +
'\tconst normalized = $util.Long.fromValue(value);\n' +
'\tconst prepared = unsigned && normalized && typeof normalized.toUnsigned === "function"\n' +
'\t\t? normalized.toUnsigned()\n' +
'\t\t: typeof normalized.toSigned === "function"\n' +
'\t\t\t? normalized.toSigned()\n' +
'\t\t\t: normalized;\n' +
'\treturn prepared.toNumber();\n' +
'\t// Fast path: convert Long {low, high} directly via native BigInt\n' +
'\tif (value && typeof value.low === "number" && typeof value.high === "number") {\n' +
'\t\tconst high = value.high | 0;\n' +
'\t\tconst lo = BigInt(value.low >>> 0);\n' +
'\t\tconst hi = BigInt(value.high >>> 0);\n' +
'\t\tconst combined = (hi << 32n) | lo;\n' +
'\t\tif (!unsigned && high < 0) {\n' +
'\t\t\treturn Number(combined - (1n << 64n));\n' +
'\t\t}\n' +
'\t\treturn Number(combined);\n' +
'\t}\n' +
'\treturn Number(value);\n' +
'}\n\n'
if (!content.includes('function longToString(')) {
+26 -18
View File
@@ -12,14 +12,21 @@ function longToString(value, unsigned) {
if (typeof value === "number") {
return String(value);
}
if (!$util.Long) {
return String(value);
// Fast path: convert Long {low, high} directly via native BigInt
// BigInt.toString() is a native C++ operation, much faster than Long's pure JS division loops
if (value && typeof value.low === "number" && typeof value.high === "number") {
// Normalize high to signed int32 so the sign-bit check works for inputs
// where high is stored unsigned (e.g. raw {low, high} JSON).
const high = value.high | 0;
const lo = BigInt(value.low >>> 0);
const hi = BigInt(value.high >>> 0);
const combined = (hi << 32n) | lo;
if (!unsigned && high < 0) {
return (combined - (1n << 64n)).toString();
}
const normalized = $util.Long.fromValue(value);
const prepared = unsigned && normalized && typeof normalized.toUnsigned === "function"
? normalized.toUnsigned()
: normalized;
return prepared.toString();
return combined.toString();
}
return String(value);
}
function longToNumber(value, unsigned) {
@@ -27,19 +34,20 @@ function longToNumber(value, unsigned) {
return value;
}
if (typeof value === "string") {
const numeric = Number(value);
return numeric;
}
if (!$util.Long) {
return Number(value);
}
const normalized = $util.Long.fromValue(value);
const prepared = unsigned && normalized && typeof normalized.toUnsigned === "function"
? normalized.toUnsigned()
: typeof normalized.toSigned === "function"
? normalized.toSigned()
: normalized;
return prepared.toNumber();
// Fast path: convert Long {low, high} directly via native BigInt
if (value && typeof value.low === "number" && typeof value.high === "number") {
const high = value.high | 0;
const lo = BigInt(value.low >>> 0);
const hi = BigInt(value.high >>> 0);
const combined = (hi << 32n) | lo;
if (!unsigned && high < 0) {
return Number(combined - (1n << 64n));
}
return Number(combined);
}
return Number(value);
}
export const proto = $root.proto = (() => {
+11 -11
View File
@@ -1,5 +1,6 @@
import { pipeline } from 'stream/promises'
import { promisify } from 'util'
import { inflate } from 'zlib'
import { createInflate, inflate } from 'zlib'
import { proto } from '../../WAProto/index.js'
import type { Chat, Contact, LIDMapping, WAMessage } from '../Types'
import { WAMessageStubType } from '../Types'
@@ -29,16 +30,15 @@ const inflatePromise = promisify(inflate)
*/
export const downloadHistory = async (msg: proto.Message.IHistorySyncNotification, options: RequestInit) => {
const stream = await downloadContentFromMessage(msg, 'md-msg-hist', { options })
const bufferArray: Buffer[] = []
for await (const chunk of stream) {
bufferArray.push(chunk)
}
let buffer: Buffer = Buffer.concat(bufferArray)
// decompress buffer
buffer = await inflatePromise(buffer)
// Pipe decrypted stream directly through zlib inflate.
// Avoids allocating an intermediate buffer for the compressed payload —
// memory peaks during 50MB history syncs drop ~50% (PR upstream #2333).
const inflater = createInflate()
const chunks: Buffer[] = []
inflater.on('data', (chunk: Buffer) => chunks.push(chunk))
await pipeline(stream, inflater)
const buffer = Buffer.concat(chunks)
const syncData = proto.HistorySync.decode(buffer)
return syncData
}
@@ -361,7 +361,7 @@ export const processHistoryMessage = (item: proto.IHistorySync, logger?: ILogger
}
}
chats.push({ ...chat })
chats.push(chat)
}
break
+1 -1
View File
@@ -618,7 +618,7 @@ export const downloadEncryptedContent = async (
const output = new Transform({
transform(chunk, _, callback) {
let data = Buffer.concat([remainingBytes, chunk])
let data = remainingBytes.length ? Buffer.concat([remainingBytes, chunk]) : chunk
const decryptLength = toSmallestChunkSize(data.length)
remainingBytes = data.slice(decryptLength)
+209
View File
@@ -0,0 +1,209 @@
import Long from 'long'
import $protobuf from 'protobufjs/minimal.js'
const $util = $protobuf.util
// proto implementation
function longToStringOld(value: any, unsigned?: boolean): string {
if (typeof value === 'string') return value
if (typeof value === 'number') return String(value)
if (!$util.Long) return String(value)
const normalized = ($util.Long as any).fromValue(value)
const prepared =
unsigned && normalized && typeof normalized.toUnsigned === 'function' ? normalized.toUnsigned() : normalized
return prepared.toString()
}
// bigint implementation
function longToStringNew(value: any, unsigned?: boolean): string {
if (typeof value === 'string') return value
if (typeof value === 'number') return String(value)
if (value && typeof value.low === 'number' && typeof value.high === 'number') {
const high = value.high | 0
const lo = BigInt(value.low >>> 0)
const hi = BigInt(value.high >>> 0)
const combined = (hi << 32n) | lo
if (!unsigned && high < 0) {
return (combined - (1n << 64n)).toString()
}
return combined.toString()
}
return String(value)
}
function longToNumberOld(value: any, unsigned?: boolean): number {
if (typeof value === 'number') return value
if (typeof value === 'string') return Number(value)
if (!$util.Long) return Number(value)
const normalized = ($util.Long as any).fromValue(value)
const prepared =
unsigned && normalized && typeof normalized.toUnsigned === 'function'
? normalized.toUnsigned()
: typeof normalized.toSigned === 'function'
? normalized.toSigned()
: normalized
return prepared.toNumber()
}
function longToNumberNew(value: any, unsigned?: boolean): number {
if (typeof value === 'number') return value
if (typeof value === 'string') return Number(value)
if (value && typeof value.low === 'number' && typeof value.high === 'number') {
const high = value.high | 0
const lo = BigInt(value.low >>> 0)
const hi = BigInt(value.high >>> 0)
const combined = (hi << 32n) | lo
if (!unsigned && high < 0) {
return Number(combined - (1n << 64n))
}
return Number(combined)
}
return Number(value)
}
describe('BigInt vs Long equivalence validation', () => {
// Test cases: [description, Long value, unsigned flag]
const testCases: [string, Long, boolean][] = [
// Basic values
['zero unsigned', Long.fromNumber(0, true), true],
['zero signed', Long.fromNumber(0, false), false],
['one unsigned', Long.fromNumber(1, true), true],
['one signed', Long.fromNumber(1, false), false],
// Typical WhatsApp timestamps (seconds since epoch)
['timestamp 2023', Long.fromNumber(1700000000, true), true],
['timestamp 2025', Long.fromNumber(1750000000, true), true],
['timestamp as signed', Long.fromNumber(1700000000, false), false],
// File sizes
['small file', Long.fromNumber(1024, true), true],
['medium file 10MB', Long.fromNumber(10485760, true), true],
['large file 2GB', Long.fromNumber(2147483648, true), true],
// Boundary values - 32-bit
['max int32', Long.fromNumber(2147483647, false), false],
['min int32', Long.fromNumber(-2147483648, false), false],
['max uint32', Long.fromNumber(4294967295, true), true],
// Around MAX_SAFE_INTEGER
['MAX_SAFE_INTEGER', Long.fromString('9007199254740991', false), false],
['MAX_SAFE_INTEGER + 1', Long.fromString('9007199254740992', false), false],
['MAX_SAFE_INTEGER unsigned', Long.fromString('9007199254740991', true), true],
// Large unsigned values
['large unsigned', Long.fromString('9999999999999999999', true), true],
['max uint64', Long.fromString('18446744073709551615', true), true],
['max uint64 - 1', Long.fromString('18446744073709551614', true), true],
// Signed negative values
['negative one', Long.fromNumber(-1, false), false],
['negative small', Long.fromNumber(-100, false), false],
['negative large', Long.fromNumber(-2147483648, false), false],
['min int64', Long.fromString('-9223372036854775808', false), false],
['max int64', Long.fromString('9223372036854775807', false), false],
['-1 as signed Long', Long.fromBits(-1, -1, false), false],
// Tricky bit patterns
['high=1, low=0', Long.fromBits(0, 1, true), true],
['high=0, low=-1 (0xFFFFFFFF)', Long.fromBits(-1, 0, true), true],
['high=-1, low=-1 unsigned (max uint64)', Long.fromBits(-1, -1, true), true],
['high=0x7FFFFFFF, low=0xFFFFFFFF (max int64)', Long.fromBits(-1, 0x7fffffff, false), false],
['high=0x80000000, low=0 (min int64)', Long.fromBits(0, -2147483648, false), false],
// Powers of 2
['2^32', Long.fromString('4294967296', true), true],
['2^32 signed', Long.fromString('4294967296', false), false],
['2^48', Long.fromString('281474976710656', true), true],
['2^63', Long.fromString('9223372036854775808', true), true]
]
describe('longToString equivalence', () => {
for (const [desc, longVal, unsigned] of testCases) {
it(`${desc}: Long(${longVal.low}, ${longVal.high}, ${unsigned})`, () => {
const oldResult = longToStringOld(longVal, unsigned)
const newResult = longToStringNew(longVal, unsigned)
expect(newResult).toBe(oldResult)
})
}
})
describe('longToNumber equivalence', () => {
for (const [desc, longVal, unsigned] of testCases) {
it(`${desc}: Long(${longVal.low}, ${longVal.high}, ${unsigned})`, () => {
const oldResult = longToNumberOld(longVal, unsigned)
const newResult = longToNumberNew(longVal, unsigned)
// For large values, both may lose precision equally (Number limitation)
// So we compare string representations
expect(newResult.toString()).toBe(oldResult.toString())
})
}
})
describe('non-Long inputs (fallback paths)', () => {
it('string passthrough', () => {
expect(longToStringNew('12345')).toBe('12345')
expect(longToStringNew('0')).toBe('0')
})
it('number passthrough', () => {
expect(longToStringNew(42)).toBe('42')
expect(longToStringNew(0)).toBe('0')
expect(longToStringNew(-1)).toBe('-1')
})
it('null/undefined fallback', () => {
expect(longToStringNew(null)).toBe('null')
expect(longToStringNew(undefined)).toBe('undefined')
})
it('object without low/high', () => {
expect(longToStringNew({ foo: 'bar' })).toBe('[object Object]')
})
// Defensive: a raw {low, high} JSON object can carry `high` as an
// unsigned 32-bit value (Long.fromBits would normalize via `| 0`,
// but plain JSON deserialization does not). Without `value.high | 0`
// the sign-bit check fails and the result is interpreted as unsigned.
// Upstream Baileys PR #2333 has this latent bug; InfiniteAPI fixes it.
it('signed sign-bit detection on raw unsigned high (-1 as {low: -1, high: 4294967295})', () => {
const raw = { low: -1, high: 0xffffffff }
// signed: -1
expect(longToStringNew(raw, false)).toBe('-1')
// unsigned: max uint64
expect(longToStringNew(raw, true)).toBe('18446744073709551615')
})
it('signed sign-bit detection on raw unsigned high (-4294967296 as {low: 0, high: 4294967295})', () => {
const raw = { low: 0, high: 0xffffffff }
// signed: -4294967296 (high word = -1 after normalization)
expect(longToStringNew(raw, false)).toBe('-4294967296')
// unsigned: 0xFFFFFFFF00000000 = 18446744069414584320
expect(longToStringNew(raw, true)).toBe('18446744069414584320')
})
})
describe('fuzz: random Long values', () => {
it('100 random unsigned values match', () => {
for (let i = 0; i < 100; i++) {
const low = (Math.random() * 0xffffffff) | 0
const high = (Math.random() * 0xffffffff) | 0
const longVal = Long.fromBits(low, high, true)
const oldResult = longToStringOld(longVal, true)
const newResult = longToStringNew(longVal, true)
expect(newResult).toBe(oldResult)
}
})
it('100 random signed values match', () => {
for (let i = 0; i < 100; i++) {
const low = (Math.random() * 0xffffffff) | 0
const high = (Math.random() * 0xffffffff) | 0
const longVal = Long.fromBits(low, high, false)
const oldResult = longToStringOld(longVal, false)
const newResult = longToStringNew(longVal, false)
expect(newResult).toBe(oldResult)
}
})
})
})
+88
View File
@@ -1,3 +1,4 @@
import Long from 'long'
import '../index.js'
import { proto } from '../../WAProto/index.js'
@@ -28,4 +29,91 @@ describe('proto serialization', () => {
const json = message.toJSON()
expect(json.message?.imageMessage?.fileLength).toBe('1234567890123456789')
})
it('converts Long objects to strings correctly via BigInt fast path', () => {
const message = proto.WebMessageInfo.fromObject({
key: {
remoteJid: '123@s.whatsapp.net',
id: 'ABC123',
fromMe: false
},
messageTimestamp: Long.fromNumber(1700000000, true),
message: {
imageMessage: {
fileLength: Long.fromString('9876543210', true)
}
}
})
const json = message.toJSON()
expect(json.messageTimestamp).toBe('1700000000')
expect(json.message?.imageMessage?.fileLength).toBe('9876543210')
})
it('handles large unsigned Long values (> MAX_SAFE_INTEGER)', () => {
const message = proto.WebMessageInfo.fromObject({
key: {
remoteJid: '123@s.whatsapp.net',
id: 'ABC123',
fromMe: false
},
messageTimestamp: Long.fromString('18446744073709551615', true), // max uint64
message: {
imageMessage: {
fileLength: Long.fromString('9999999999999999999', true)
}
}
})
const json = message.toJSON()
expect(json.messageTimestamp).toBe('18446744073709551615')
expect(json.message?.imageMessage?.fileLength).toBe('9999999999999999999')
})
it('handles zero and small Long values', () => {
const message = proto.WebMessageInfo.fromObject({
key: {
remoteJid: '123@s.whatsapp.net',
id: 'ABC123',
fromMe: false
},
messageTimestamp: Long.fromNumber(0, true),
message: {
imageMessage: {
fileLength: Long.fromNumber(1, true)
}
}
})
const json = message.toJSON()
expect(json.messageTimestamp).toBe('0')
expect(json.message?.imageMessage?.fileLength).toBe('1')
})
it('roundtrips encode/decode with Long fields preserving values (> MAX_SAFE_INTEGER)', () => {
// Use uint64 values above Number.MAX_SAFE_INTEGER (2^53 - 1) so the
// roundtrip actually exercises the BigInt fast path on decode->toJSON.
// Safe integers would silently work even if longToString fell back to
// Number(value).
const original = proto.WebMessageInfo.fromObject({
key: {
remoteJid: '123@s.whatsapp.net',
id: 'ABC123',
fromMe: false
},
messageTimestamp: Long.fromString('18446744073709551615', true),
message: {
imageMessage: {
fileLength: Long.fromString('9999999999999999999', true)
}
}
})
const encoded = proto.WebMessageInfo.encode(original).finish()
const decoded = proto.WebMessageInfo.decode(encoded)
const json = decoded.toJSON()
expect(json.messageTimestamp).toBe('18446744073709551615')
expect(json.message?.imageMessage?.fileLength).toBe('9999999999999999999')
})
})