feat: add graceful shutdown handling to scraper job scheduler
All checks were successful
CI/CD Pipeline - Apartment API / Scan Dependencies (pull_request) Successful in 13s
CI/CD Pipeline - Apartment API / Lint & Test (pull_request) Successful in 39s
CI/CD Pipeline - Apartment API / Send Webhook Notification (pull_request) Successful in 2s
CI/CD Pipeline - Apartment API / Build & Push Image (pull_request) Has been skipped
CI/CD Pipeline - Apartment API / Deploy to Production (pull_request) Has been skipped
All checks were successful
CI/CD Pipeline - Apartment API / Scan Dependencies (pull_request) Successful in 13s
CI/CD Pipeline - Apartment API / Lint & Test (pull_request) Successful in 39s
CI/CD Pipeline - Apartment API / Send Webhook Notification (pull_request) Successful in 2s
CI/CD Pipeline - Apartment API / Build & Push Image (pull_request) Has been skipped
CI/CD Pipeline - Apartment API / Deploy to Production (pull_request) Has been skipped
Add gracefulShutdown(), registerSignalHandlers(), and isShuttingDown() to enable clean process termination during scraper operations. Key changes: - Register SIGTERM and SIGINT signal handlers for process shutdown - Wait for in-progress scraper jobs to complete before exiting, with a configurable timeout (SCRAPER_SHUTDOWN_TIMEOUT, default 30s) - Release mutex lock on shutdown to prevent stale lock state - Track shutdown state to reject new job acquisitions during shutdown - Stop cron scheduler as part of the shutdown sequence - Add SHUTDOWN_TIMEOUT to scraper config module Adds 14 new tests covering shutdown lifecycle, timeout enforcement, idempotent shutdown calls, and lock rejection during shutdown (56 total).
This commit is contained in:
@ -27,6 +27,9 @@ module.exports = {
|
||||
timeout: parseInt(process.env.SCRAPER_TIMEOUT, 10) || 30000
|
||||
},
|
||||
|
||||
// Graceful shutdown timeout (how long to wait for running job before force-stopping)
|
||||
SHUTDOWN_TIMEOUT: parseInt(process.env.SCRAPER_SHUTDOWN_TIMEOUT, 10) || 30000,
|
||||
|
||||
// MongoDB collection names (environment variable overrides for development isolation)
|
||||
COLLECTIONS: {
|
||||
UNITS: process.env.SCRAPER_UNITS_COLLECTION || 'units_migration_test',
|
||||
|
||||
Reference in New Issue
Block a user