stephen/apartment-dashboard-api
1
0
Fork 0
Code Issues 1 Pull Requests Actions Releases Wiki Activity

Add scraperLogger.js with credential redaction #6

Merged
stephen merged 2 commits from scraper/logger into dev 2026-01-28 10:52:01 -07:00
Conversation 1 Commits 2 Files Changed 4 +993 -7
stephen commented 2026-01-28 08:06:59 -07:00
Owner
Copy Link

Summary

  • Implement structured JSON logging module for the scraper service (T-01)
  • Add createLogger(jobId) factory function for job-scoped logging with info/warn/error methods
  • Automatic redaction of MongoDB connection string credentials and sensitive keys (password, secret, token, apikey, authorization)
  • Message truncation, circular reference handling, and Buffer representation

Test Plan

  • All 35 unit tests pass (npm test -- --testPathPatterns="scraperLogger")
  • Verified MongoDB URI redaction for both mongodb:// and mongodb+srv:// formats
  • Verified sensitive key redaction in nested objects
  • Verified circular reference handling
  • Verified Buffer objects display as [Buffer: N bytes]

Files Changed

  • services/scraperLogger.js (new)
  • __tests__/scraper/scraperLogger.test.js (new)
## Summary - Implement structured JSON logging module for the scraper service (T-01) - Add `createLogger(jobId)` factory function for job-scoped logging with info/warn/error methods - Automatic redaction of MongoDB connection string credentials and sensitive keys (password, secret, token, apikey, authorization) - Message truncation, circular reference handling, and Buffer representation ## Test Plan - [x] All 35 unit tests pass (`npm test -- --testPathPatterns="scraperLogger"`) - [x] Verified MongoDB URI redaction for both `mongodb://` and `mongodb+srv://` formats - [x] Verified sensitive key redaction in nested objects - [x] Verified circular reference handling - [x] Verified Buffer objects display as `[Buffer: N bytes]` ## Files Changed - `services/scraperLogger.js` (new) - `__tests__/scraper/scraperLogger.test.js` (new)
stephen added 2 commits 2026-01-28 08:07:00 -07:00
T-33: Add npm dependencies 59fb6da5fb
Add scraperLogger.js with credential redaction
All checks were successful
CI/CD Pipeline - Apartment API / Scan Dependencies (pull_request) Successful in 14s
Details
CI/CD Pipeline - Apartment API / Run Tests (pull_request) Successful in 9m45s
Details
CI/CD Pipeline - Apartment API / Build & Push Image (pull_request) Has been skipped
Details
CI/CD Pipeline - Apartment API / Deploy to Production (pull_request) Has been skipped
Details
d14bd724ed 
Implement structured JSON logging module for the scraper service:
- createLogger(jobId) factory function for job-scoped logging
- info(), warn(), error() methods with consistent JSON output
- Automatic redaction of MongoDB connection string credentials
- Redaction of sensitive keys (password, secret, token, apikey, authorization)
- Message truncation to prevent log bloat (1000 char limit)
- Circular reference handling
- Buffer object representation as [Buffer: N bytes]

Includes comprehensive test suite with 35 test cases covering:
- Log entry format validation (timestamp, level, message, jobId, context)
- MongoDB URI redaction for both mongodb:// and mongodb+srv://
- Sensitive key redaction
- Edge cases (null, undefined, nested objects, arrays)
- Buffer and circular reference handling
stephen changed target branch from main to dev 2026-01-28 08:26:14 -07:00
stephen merged commit 9dba679221 into dev 2026-01-28 10:52:01 -07:00
stephen commented 2026-01-28 13:15:00 -07:00
Author
Owner
Copy Link

This is a test comment

This is a test comment
Sign in to join this conversation.
Reviewers
No Reviewers
Labels
No items
No Label
Milestone
No items
No Milestone
Assignees
Clear assignees
stephen
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: stephen/apartment-dashboard-api#6
No description provided.
Delete Branch "scraper/logger"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?