feat: add graceful shutdown handling to scraper job scheduler
All checks were successful
CI/CD Pipeline - Apartment API / Scan Dependencies (pull_request) Successful in 13s
CI/CD Pipeline - Apartment API / Lint & Test (pull_request) Successful in 39s
CI/CD Pipeline - Apartment API / Send Webhook Notification (pull_request) Successful in 2s
CI/CD Pipeline - Apartment API / Build & Push Image (pull_request) Has been skipped
CI/CD Pipeline - Apartment API / Deploy to Production (pull_request) Has been skipped

Add gracefulShutdown(), registerSignalHandlers(), and isShuttingDown()
to enable clean process termination during scraper operations.

Key changes:
- Register SIGTERM and SIGINT signal handlers for process shutdown
- Wait for in-progress scraper jobs to complete before exiting, with
  a configurable timeout (SCRAPER_SHUTDOWN_TIMEOUT, default 30s)
- Release mutex lock on shutdown to prevent stale lock state
- Track shutdown state to reject new job acquisitions during shutdown
- Stop cron scheduler as part of the shutdown sequence
- Add SHUTDOWN_TIMEOUT to scraper config module

Adds 14 new tests covering shutdown lifecycle, timeout enforcement,
idempotent shutdown calls, and lock rejection during shutdown (56 total).
This commit is contained in:
2026-02-06 18:03:06 -07:00
parent 9d789d38fd
commit 48a4db45ba
3 changed files with 636 additions and 4 deletions

View File

@ -27,6 +27,9 @@ module.exports = {
timeout: parseInt(process.env.SCRAPER_TIMEOUT, 10) || 30000
},
// Graceful shutdown timeout (how long to wait for running job before force-stopping)
SHUTDOWN_TIMEOUT: parseInt(process.env.SCRAPER_SHUTDOWN_TIMEOUT, 10) || 30000,
// MongoDB collection names (environment variable overrides for development isolation)
COLLECTIONS: {
UNITS: process.env.SCRAPER_UNITS_COLLECTION || 'units_migration_test',