Commit Graph

2283 Commits

Author SHA1 Message Date
Claude de5988ba8f feat: implement WhatsApp connection and message metrics
Added tracking for all WhatsApp-related metrics:

Connection metrics:
- active_connections: Gauge tracking active connections (inc on open, dec on close)
- connection_attempts_total: Counter for connection attempts (success/failure)

Message metrics:
- messages_sent_total: Counter with type label (text, image, video, audio, etc)
- messages_received_total: Counter with type label
- history_sync_messages_total: Counter for history sync messages

Added helper functions in prometheus-metrics.ts:
- incrementActiveConnections(), decrementActiveConnections(), setActiveConnections()
- recordConnectionAttempt(status)
- recordMessageSent(type), recordMessageReceived(type)
- recordMessageRetry(type), recordMessageFailure(type, reason)
- setMessagesQueued(count, priority)
- recordHistorySyncMessages(count)
2026-01-22 14:20:17 +00:00
Claude f9ed1dc16f feat: add adaptive metrics and circuit breaker trips tracking
Added metrics tracking for:
- circuit_breaker_trips_total: Incremented when circuit opens
- adaptive_health_status: 1 if healthy (not in aggressive mode), 0 otherwise
- adaptive_event_rate: Current events per second

Added methods to AdaptiveTimeoutCalculator:
- getEventRate(): Returns current event rate in events/second
- isHealthy(): Returns true if not in aggressive mode

Metrics are updated on each buffer flush when adaptive timeout is enabled.
2026-01-22 13:57:34 +00:00
Claude 5b8ea3bb2b fix: add buffer_destroyed_total and buffer_final_flush_total metrics
Added recordBufferDestroyed() and recordBufferFinalFlush() functions
and integrated into destroy() function in event-buffer.ts.

Tracks:
- Buffer destruction with reason and whether there was pending flush
- Final flushes that occur during buffer destruction
2026-01-22 13:47:48 +00:00
Claude 8d0c03cc2a feat: add connection_errors_total metric tracking
Added recordConnectionError() function and integrated it into socket.ts
to track connection errors by type:
- connection_closed
- connection_lost
- connection_replaced
- timed_out
- logged_out
- bad_session
- restart_required
- multidevice_mismatch
- error_{code} for other status codes
2026-01-22 13:43:31 +00:00
Claude 6412286c73 fix: add buffer_overflows_total metric increment
The metric was defined but never incremented when buffer overflow occurred.

Added recordBufferOverflow() function and call it from checkBufferOverflow()
when buffer exceeds maxBufferSize (default 5000 events).
2026-01-22 13:40:31 +00:00
Claude 6570a1fc6e fix: add buffer_cache_cleanup_total metric increment
The metric was defined but never incremented when LRU cleanup happened.

Added recordCacheCleanup() function and call it from cleanupHistoryCache()
when cache entries are removed due to exceeding maxHistoryCacheSize.
2026-01-22 13:31:32 +00:00
Claude 68f6a2ae88 fix: update buffer_cache_size metric with actual history cache size
The buffer_cache_size metric was defined but never updated, always showing 0.

Fix:
- Add historyCacheSize parameter to recordBufferFlush function
- Pass historyCache.size from event-buffer.ts when recording flush metrics
- Update both bufferCacheSize and bufferHistoryCacheSize metrics
2026-01-22 13:28:25 +00:00
Claude b1687389cd fix: resolve buffer_flushes_total metric label mismatch
The buffer_flushes_total metric was registered twice with different labels:
- In main metrics object: ['type', 'reason']
- In getEventBufferMetrics(): ['forced']

This caused recordBufferFlush() to silently fail because it tried to
increment with { forced } labels but the registered metric expected
{ type, reason } labels.

Fix:
- Remove duplicate bufferFlushes from getEventBufferMetrics()
- Update recordBufferFlush() to use metrics.bufferFlushes with correct labels
- Update recordEventBuffered() to use main metrics.eventsBuffered
- Rename local variable to avoid shadowing global metrics object
2026-01-22 12:57:44 +00:00
Claude e4ffbb4b5c fix: enable buffer metrics automatically when Prometheus is enabled
Previously, buffer metrics required a separate BAILEYS_BUFFER_METRICS=true
env variable even when BAILEYS_PROMETHEUS_ENABLED=true was set. This caused
the buffer_flushes_total metric to have no data even though flushes were
occurring.

Now buffer metrics are enabled automatically when either BAILEYS_BUFFER_METRICS
or BAILEYS_PROMETHEUS_ENABLED is set to true.
2026-01-22 12:50:02 +00:00
Claude ee80d2cdf8 feat: add missing metrics for dashboard compatibility
Added new metrics:
- active_connections: Number of active WhatsApp connections
- connection_errors_total: Total connection errors by type
- buffer_destroyed_total: Total buffers destroyed
- buffer_final_flush_total: Final flushes during destruction
- buffer_cache_cleanup_total: Cache cleanup operations
- buffer_cache_size: Current buffer cache size
- adaptive_health_status: System health (0=unhealthy, 1=healthy)
- adaptive_event_rate: Current event rate per second

These metrics are required for the Grafana dashboard to display all panels correctly.
2026-01-22 11:57:23 +00:00
Claude b5b43b5fd6 fix: use different metric name to avoid conflict with collectDefaultMetrics 2026-01-22 04:13:42 +00:00
Claude 931ef71250 fix: prevent duplicate metric registration error
Check if nodejs_eventloop_lag_seconds metric already exists before creating
it in SystemMetricsCollector. This prevents conflicts when collectDefaultMetrics
from prom-client has already registered metrics with the same name.

Added helper functions metricExists() and getExistingMetric() to safely
check and retrieve existing metrics from the custom registry.
2026-01-22 04:06:58 +00:00
Claude 4dc4f20550 fix: auto-initialize Prometheus metrics server when enabled
The Prometheus metrics server was implemented but never started because
initializeMetrics() was never called. This fix adds automatic initialization
when the module is loaded and BAILEYS_PROMETHEUS_ENABLED=true.

This ensures the /metrics endpoint is available without requiring manual
initialization in the application code.
2026-01-22 03:58:54 +00:00
Renato Alcara 77c05b04b8 fix: add missing await for mutationKeys in decodeSyncdMutations
fix: add missing await for mutationKeys in decodeSyncdMutations
2026-01-21 16:52:09 -03:00
Claude a319a63ff3 fix: add missing await for mutationKeys in decodeSyncdMutations
When hkdf became async, mutationKeys also became async but the call
site wasn't updated. This caused Promise<Promise<Key>> instead of
Promise<Key>, breaking contact sync events (contacts.upsert).

Ref: WhiskeySockets/Baileys#2288
2026-01-21 19:36:00 +00:00
Renato Alcara 2a28b766a6 feat: add persistent shared version cache for multiple connections
feat: add persistent shared version cache for multiple connections
2026-01-21 16:33:42 -03:00
Claude 29475018a3 fix: address PR review comments for version cache
- Replace Function type with proper VersionCacheLogger interface
- Add documentation for cacheFilePath about container/serverless limitations
- Handle fetchInProgress in clearVersionCache to prevent cache restoration
- Add deduplication check in refreshVersionCache
- Remove unused fetchLatestWaWebVersion import
- Return success status from refreshVersionCache to detect fallback
- Fix inefficient getCachedVersion call - use cacheStatus.version directly
- Remove as any casts by using logger adapter pattern
- Don't downgrade version on transient network errors
2026-01-21 19:14:32 +00:00
Claude 185632c053 feat: add persistent shared version cache for multiple connections
- Add version-cache.ts with file-based persistence
- 150 connections = 1 request (not 150) via request deduplication
- Cache survives server restarts via .baileys-version-cache.json
- Export cache utilities: getCachedVersion, refreshVersionCache, clearVersionCache
- Update makeWASocketAutoVersion to use shared cache
2026-01-21 18:47:12 +00:00
Renato Alcara 8d528058a7 fix: add import assertion for JSON imports in ESM
fix: add import assertion for JSON imports in ESM
2026-01-21 14:51:02 -03:00
Claude 8ef129bd3d fix: add import assertion for JSON imports in ESM
Adds 'with { type: "json" }' to JSON imports to fix Node.js ESM runtime error:
ERR_IMPORT_ATTRIBUTE_MISSING
2026-01-21 17:46:54 +00:00
Renato Alcara b3e3b48e51 feat: centralize WhatsApp version + improve update robustness
feat: centralize WhatsApp version + improve update robustness
2026-01-21 14:41:05 -03:00
Claude 46ffaf027c fix: address PR review comments
Fixes from code review:

1. Fix #1,6: Use connection.update event instead of overriding sock.end()
   - Listens for 'close' event to cleanup interval
   - Handles both explicit close and internal disconnections

2. Fix #3: Exit code 2 when fetch fails (not 0)
   - Allows CI to distinguish success/error/fetch-failed
   - Properly signals fetch failures to workflows

3. Fix #4: Document revision bounds + add env vars
   - Added detailed comments explaining min/max revision values
   - Made configurable via WA_MIN_REVISION/WA_MAX_REVISION env vars

4. Fix #5,9: Remove unused fetchLatestVersion option
   - Removed from SocketConfig and defaults
   - Updated versionCheckIntervalMs docs to clarify it's only for makeWASocketAutoVersion

5. Fix #7: Use separate variable for version tracking
   - trackedVersion instead of mutating mergedConfig
   - Prevents unexpected side effects

6. Fix #8: Check socket state before emitting events
   - isSocketClosed flag to prevent race conditions
   - Double-check after async operations

7. Fix #10: Implement force parameter in workflow
   - Creates PR even without version changes when force=true
   - Useful for re-triggering updates manually

Note: Test coverage (Fix #2) deferred to separate PR due to
ESM mocking complexity with Jest.
2026-01-21 17:21:49 +00:00
Claude 805fdd3525 feat: add periodic version check with soft reconnection
Implements automatic version updates that are transparent to users:

- Checks for new WhatsApp Web version every 6 hours (configurable)
- When new version detected, saves it for next natural reconnection
- Emits 'version.update' event so users can track updates
- No disconnection required - WhatsApp naturally reconnects every 30min-2h
- Cleans up interval when socket closes

Configuration:
```typescript
const sock = await makeWASocketAutoVersion({
    auth: state,
    versionCheckIntervalMs: 6 * 60 * 60 * 1000 // 6 hours (default)
})

sock.ev.on('version.update', ({ currentVersion, newVersion, isCritical }) => {
    console.log(`New version: ${newVersion.join('.')}`)
})
```
2026-01-21 16:51:39 +00:00
Claude 76e080daec feat: add makeWASocketAutoVersion for automatic version fetching
Adds a new async function that automatically fetches the latest
WhatsApp Web version from web.whatsapp.com before connecting.

Usage:
```typescript
// Option 1: Auto-fetch version (recommended)
const sock = await makeWASocketAutoVersion({ auth: state })

// Option 2: Manual version (existing behavior)
const sock = makeWASocket({ auth: state })
```

Benefits:
- No need to update library for version changes
- Automatic fallback to bundled version if fetch fails
- Logged warnings when using fallback
2026-01-21 16:38:46 +00:00
Claude b9a7c2cabc feat: improve version update robustness and frequency
Changes:
- Update frequency: weekly → daily (06:00 UTC)
- Add retry with exponential backoff (3 attempts per source)
- Add multiple source endpoints (sw.js + bootstrap page)
- Add version validation (sanity checks on revision numbers)
- Add fetch timeout to prevent hanging
- Add detailed logging for debugging
- Simplify workflow to only update baileys-version.json

This makes the version update more reliable and responsive
to WhatsApp Web changes.
2026-01-21 16:22:46 +00:00
Claude 8a134d2420 refactor: centralize WhatsApp version to single source of truth
Instead of maintaining the version number in 3 separate files
(baileys-version.json, index.ts, generics.ts), this change makes
baileys-version.json the single source of truth. Other files now
import the version from this JSON file.

This follows the DRY principle and reduces:
- Risk of version inconsistency across files
- Maintenance burden (only 1 file to update)
- PR diff size (3 files → 1 file for version updates)
- Merge conflict probability

Addresses inefficiency found in Baileys PR #2269.
2026-01-21 16:17:14 +00:00
Renato Alcara f59e8cfe8b feat: add automatic CTWA (Click-to-WhatsApp) ads message recovery
feat: add automatic CTWA (Click-to-WhatsApp) ads message recovery
2026-01-21 12:22:41 -03:00
Claude 5a36ae20d4 feat: add automatic CTWA (Click-to-WhatsApp) ads message recovery
Messages from Facebook/Instagram ads (Click-to-WhatsApp) don't arrive on
linked devices because Meta's ads endpoint doesn't encrypt for multi-device.
They arrive as "Message absent from node" placeholders.

This change automatically requests the message from the primary phone via
PDO (Peer Data Operation) when a CTWA placeholder is detected.

Changes:
- Add enableCTWARecovery config option (default: true)
- Trigger requestPlaceholderResend() for "Message absent from node" errors
- Add Prometheus metrics for CTWA recovery tracking
- Add comprehensive unit tests for CTWA recovery functionality

Resolves: https://github.com/WhiskeySockets/Baileys/issues/1723
Resolves: https://github.com/WhiskeySockets/Baileys/issues/1034
2026-01-21 14:56:39 +00:00
Renato Alcara af6f2eede7 feat: normalize inbound JIDs to PN + batch LID/PN lookups + mapping logs
feat: normalize inbound JIDs to PN + batch LID/PN lookups + mapping logs
2026-01-21 01:34:59 -03:00
Claude d968205376 fix: resolve TypeScript errors in merged code
- Add null check for decoded in getLIDsForPNs loop
- Fix jest mock typing in process-message.test.ts
2026-01-21 04:24:09 +00:00
Claude be1f00c5d5 Merge remote-tracking branch 'origin/codex/analyze-pr-2274-for-repository-contribution' into claude/review-pr-undefined-returns-725K7 2026-01-21 04:16:55 +00:00
Renato Alcara 8ef29c7d94 feat: normalize inbound JIDs to PN + batch LID/PN lookups + mapping logs
Motivation
Avoid duplicate contacts when the same user arrives sometimes as a PN and sometimes as a LID.
Ensure that LID↔PN mappings acquired via history and event synchronization are used immediately.
Improved performance and scalability with batch searches (inspired by PR #2274).

Description
Incoming JID normalization: added normalizeMessageJids to convert remoteJid/participant LID → PN when available, keeping LID as fallback if PN does not already exist.
Batch lookups: getLIDsForPNs and getPNsForLIDs now use batch DB, bidirectional caching, and fallback to USync, reducing individual calls.
Automatic hydration: lidPnMappings from history sync are persisted in the store so that future LID messages are resolved immediately.
Fallback logs: logs detail when LID→PN resolves and when new mappings are stored (history sync and lid-mapping.update).

Tests
Added tests for batch PN→LID, batch LID→PN, and LID→PN normalization fallback.

 Changes applied
Summary
Added batch tests for PN→LID and LID→PN resolution 
Added tests for normalizeMessageJids cobrindo sucesso e fallback (LID permanece quando PN não existe).
2026-01-21 01:12:50 -03:00
Renato Alcara 1c17ecf2ec test: cover lid batch and jid normalization 2026-01-21 01:07:03 -03:00
Renato Alcara 31fdfa8b7c chore: log fallback mapping availability 2026-01-21 00:52:17 -03:00
Claude 9454a824d9 feat(upstream): add comprehensive LID-PN mapping extraction for Baileys PR
This folder contains the files ready for upstream WhiskeySockets/Baileys PR:

Changes to src/Utils/history.ts:
- Add isPersonJid() helper to validate JID types for mapping eligibility
- Add extractLidPnFromConversation() for conversation-level mapping extraction
- Add extractLidPnFromMessage() for message-level mapping extraction
- Modify processHistoryMessage() to extract from 3 sources with deduplication
- Fix validation bug: use || (OR) instead of && (AND) to prevent poisoned mappings

Changes to src/__tests__/Utils/history.test.ts:
- 55 comprehensive tests covering all new functions and edge cases
- Tests for all JID formats (@lid, @hosted.lid, @s.whatsapp.net, @hosted)
- Tests for skipped JIDs (@g.us, @broadcast, @newsletter)
- Tests for bug fix validation (person + group, person + broadcast, etc.)
- Tests for deduplication across all three sources

Related issues:
- Closes WhiskeySockets/Baileys#2282
- Partially addresses WhiskeySockets/Baileys#2281
2026-01-21 01:35:20 +00:00
Renato Alcara 34ab791dc0 fix(history): use OR instead of AND for JID validation in extractLidP…
fix(history): use OR instead of AND for JID validation in extractLidP…
2026-01-20 20:27:35 -03:00
Claude 3e8c89d7d0 fix: resolve merge conflicts from master
Merge master branch and resolve conflicts:
- Use import from '../WABinary/index.js' (master convention)
- Keep the bug fix: || instead of && in extractLidPnFromMessage
- Combine comprehensive tests from both branches
- Use Map for deduplication (master implementation)

All 55 tests passing.
2026-01-20 23:22:45 +00:00
Renato Alcara 02033e7a72 feat(history): comprehensive LID-PN mapping extraction with full JID format support
feat(history): comprehensive LID-PN mapping extraction with full JID format support
2026-01-20 20:17:38 -03:00
Claude 8f3660a099 fix(history): use OR instead of AND for JID validation in extractLidPnFromMessage
Address PR #20 code review feedback:

- Fix logical bug: changed `&&` to `||` in isPersonJid validation
  This ensures BOTH JIDs must be person JIDs before extracting
  a LID-PN mapping, preventing "poisoned" mappings where one side
  is a group/newsletter/broadcast

- Add isPersonJid() helper to validate JID types

- Add extractLidPnFromConversation() to extract mappings from
  conversation lidJid/pnJid properties

- Add extractLidPnFromMessage() to extract mappings from message
  remoteJidAlt/participantAlt fields

- Update processHistoryMessage() to use new extraction functions

- Add comprehensive tests covering edge cases for mixed JID types
  (person + group, person + broadcast, person + newsletter)

Note: Returning `undefined` is the correct TypeScript convention
for indicating absence of value (as per TypeScript team guidelines)
2026-01-20 23:04:49 +00:00
Claude 154dd772dd feat(history): comprehensive LID-PN mapping with full JID format support
Enhances LID-PN mapping extraction to handle all WhatsApp JID formats
and extract mappings from multiple sources within history sync data.

New JID validation:
- Add `isPersonJid()` helper to identify JIDs that can have LID-PN mappings
- Skip non-person JIDs: @newsletter (channels), @broadcast, @g.us (groups)

Triple-source extraction:
1. Top-level `phoneNumberToLidMappings` array (existing)
2. Conversation objects via `lidJid`/`pnJid` properties (existing)
3. Message objects via `remoteJidAlt`/`participantAlt` fields (NEW)

New `extractLidPnFromMessage()` function:
- Extracts mappings from message key alternative JID fields
- Handles both direct messages (remoteJidAlt) and group messages (participantAlt)
- Properly identifies LID vs PN based on server type

JID formats handled:
- @s.whatsapp.net - Standard phone number
- @lid - Logical ID (privacy)
- @hosted - Hosted phone number
- @hosted.lid - Hosted LID

46 comprehensive tests covering all scenarios.

Related: WhiskeySockets/Baileys#2263
2026-01-20 22:29:36 +00:00
Claude 86c1cf439a feat(history): extract LID-PN mappings from conversation objects
WhatsApp provides LID-PN mappings in two locations within history sync:
1. Top-level `phoneNumberToLidMappings` array (already processed)
2. Individual conversation objects with `lidJid`/`pnJid` properties (new)

This change adds extraction from conversation objects, ensuring maximum
mapping coverage regardless of sync type or payload structure.

Key improvements:
- Add `extractLidPnFromConversation()` function with comprehensive JSDoc
- Use Map for O(1) deduplication of mappings across both sources
- Handle all JID formats: @lid, @hosted.lid, @s.whatsapp.net, @hosted
- Normalize JIDs using `jidNormalizedUser()` for consistency
- Skip group chats (@g.us) that don't have LID-PN mappings

Includes 22 comprehensive tests covering:
- LID chat with pnJid extraction
- PN chat with lidJid extraction
- Hosted format handling
- Deduplication between sources
- Edge cases (nulls, groups, missing data)
- All conversation sync types

Related: WhiskeySockets/Baileys#2263
See-also: WhiskeySockets/Baileys#2282
2026-01-20 22:17:09 +00:00
Renato Alcara cc4c918e1a feat(logger): add [BAILEYS] console-friendly logging functions
feat(logger): add [BAILEYS] console-friendly logging functions
2026-01-20 18:47:08 -03:00
Claude bb73662e8f fix(logger): address PR #19 code review feedback
Fixes:
- Remove unused import logBufferMetrics from event-buffer.ts
- Add safeStringify() for robust data formatting:
  - Handles circular references gracefully
  - Formats Error objects properly
  - Truncates large arrays [Array(n)]
  - Handles undefined/null/Date/BigInt values
- Add getMode() method to AdaptiveTimeoutCalculator to avoid
  duplicating mode calculation logic in flush()
- Fix empty data object handling (no trailing space)

Tests:
- Add comprehensive test suite (39 tests) covering:
  - Environment variable check (BAILEYS_LOG=false)
  - All logging functions with emoji verification
  - Session name handling
  - formatLogData edge cases (circular, Error, arrays, etc.)
2026-01-20 21:44:08 +00:00
Claude 3f2ced4501 feat(logger): add [BAILEYS] console-friendly logging functions
Add formatted logging functions with [BAILEYS] prefix and emojis:
- logEventBuffer: buffer events (📦, 🔄, ⚠️, , 🧹, 🧠)
- logBufferMetrics: metrics display (📊)
- logMessageSent/Received: message events (📤, 📥)
- logConnection: connection events (🔌, , 🔴, 🔄, )
- logAuth: authentication events (📱, 🔑, 🚪, 🔐)
- logCircuitBreaker: circuit breaker state (, , 🔶)
- logRetry: retry attempts (🔁)
- logInfo/Warn/Error: generic messages (ℹ️, ⚠️, )
- logLidMapping: LID store events (🗂️, 🔍, 💾, 📦)

All functions respect BAILEYS_LOG=false environment variable.
Integrated into event-buffer.ts and messages-send/recv.ts.
2026-01-20 20:48:46 +00:00
Renato Alcara 88b965c603 feat(logger): add comprehensive environment variable configuration
feat(logger): add comprehensive environment variable configuration
2026-01-20 17:32:36 -03:00
Renato Alcara 44fb5997a1 fix(logger): avoid pino-pretty dependency 2026-01-20 17:30:55 -03:00
Claude a462440c5c feat(logger): add comprehensive environment variable configuration
Support all logging-related environment variables:
- BAILEYS_LOG: Enable/disable logging (default: true)
- BAILEYS_LOG_LEVEL: Set log level (default: info)
- USE_STRUCTURED_LOGS: Enable structured logging (default: false)
- LOG_FORMAT: Output format - 'json' or 'pretty' (default: auto)
- LOGGER_INFO/WARN/ERROR: Enable specific log levels (default: true)

Added createFilteredLogger for level-specific filtering and
createSilentLogger for disabled logging mode.
2026-01-20 20:16:58 +00:00
Renato Alcara 68fd02b924 fix(metrics,retry): gold standard code review corrections
fix(metrics,retry): gold standard code review corrections
2026-01-20 17:06:50 -03:00
Claude 5449e3f70a fix(metrics): handle malformed URLs gracefully in metrics endpoint
Wrap URL parsing in try/catch to prevent server crashes from malformed
requests (invalid percent-encoding, problematic Host headers, etc.).
Returns 400 Bad Request instead of crashing the process.
2026-01-20 19:48:28 +00:00
Claude 6f057b8895 fix(metrics,retry): address gold standard code review
Prometheus Metrics:
- Add METRICS_LABELS fallback for defaultLabels env config
- Separate includeSystem from collectDefaultMetrics with own env var
- Parse URL in /metrics endpoint to support querystrings and trailing slashes
- Sort keys in labelsToKey for stable/deterministic lookups
- Update documentation with all environment variables

Retry Utils:
- Remove abort listener on sleep completion to prevent memory leak
- Apply maxDelay cap AFTER jitter to guarantee max delay limit
- Validate/normalize attempt parameter in calculateDelay
- Count retries only when attempt > 1 (actual retry, not first try)
- Use metrics.retryExhausted instead of generic errors for exhausted retries

Tests:
- Add consistency tests to verify retryConfigs.rsocket matches constants
2026-01-20 19:37:19 +00:00