SCRAPE-16: Implement POST /admin/scraper/run endpoint #21

Merged
stephen merged 2 commits from scraper/api-run into dev 2026-02-06 22:07:11 -07:00

2 Commits

Author SHA1 Message Date
7e796295cb fix: replace ESM-only uuid package with built-in crypto.randomUUID
All checks were successful
CI/CD Pipeline - Apartment API / Scan Dependencies (pull_request) Successful in 13s
CI/CD Pipeline - Apartment API / Lint & Test (pull_request) Successful in 41s
CI/CD Pipeline - Apartment API / Send Webhook Notification (pull_request) Successful in 2s
CI/CD Pipeline - Apartment API / Build & Push Image (pull_request) Has been skipped
CI/CD Pipeline - Apartment API / Deploy to Production (pull_request) Has been skipped
The uuid v13 package is ESM-only, which causes Jest to fail when
loading routes/admin.js -> jobs/scraperJob.js -> uuid without mocks.
This broke all admin dashboard tests (phase1-4) with silent 404s.

Switched to Node.js built-in crypto.randomUUID() and updated all
test mocks from uuid to crypto accordingly.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-06 22:05:31 -07:00
c30b02681e feat: add POST /api/admin/scraper/run endpoint
Some checks failed
CI/CD Pipeline - Apartment API / Scan Dependencies (pull_request) Successful in 13s
CI/CD Pipeline - Apartment API / Lint & Test (pull_request) Successful in 41s
CI/CD Pipeline - Apartment API / Send Webhook Notification (pull_request) Failing after 1s
CI/CD Pipeline - Apartment API / Build & Push Image (pull_request) Has been skipped
CI/CD Pipeline - Apartment API / Deploy to Production (pull_request) Has been skipped
Implement the manual scraper trigger endpoint with the following:

- Protected by requireAuth and requireAdmin middleware (401/403)
- Mutex lock check via isScraperRunning to prevent concurrent runs (409)
- Generates UUID jobId for tracking async scrape execution
- Returns 202 Accepted immediately without blocking on scrape completion
- Releases mutex lock in .finally() to ensure cleanup on success or failure
- Logs ADMIN_TRIGGER_SCRAPE activity with jobId, dryRun, and
  usingProvidedHtml metadata via activityLogger
- Supports dryRun option (defaults to false) and htmlContent for
  testing with pre-fetched HTML
- Passes trigger: 'manual', jobId, dryRun, and htmlContent to runScrape

Includes 20 tests covering auth, mutex, async execution, activity
logging, dryRun/htmlContent options, and response structure.
2026-02-06 21:20:42 -07:00