Commit Graph

5 Commits

Author SHA1 Message Date
d09aa1d179 Add fetchPage() with axios, timeout, and retry logic
Some checks failed
CI/CD Pipeline - Apartment API / Run Linting (pull_request) Successful in 9m39s
CI/CD Pipeline - Apartment API / Run Tests (pull_request) Successful in 9m50s
CI/CD Pipeline - Apartment API / Scan Dependencies (pull_request) Successful in 15s
CI/CD Pipeline - Apartment API / Send Webhook Notification (pull_request) Failing after 2s
CI/CD Pipeline - Apartment API / Build & Push Image (pull_request) Has been skipped
CI/CD Pipeline - Apartment API / Deploy to Production (pull_request) Has been skipped
Implement fetchPage() function for HTTP scraping:
- Uses axios for HTTP GET requests
- Configurable timeout (default 30s) and User-Agent header
- Retry with exponential backoff (1s, 2s, 4s) on 5xx and network errors
- Does not retry on 4xx client errors
- Logs each attempt with attempt number and error details
- Returns HTML string on success, throws after retries exhausted
2026-02-05 15:43:20 -07:00
2993d019c5 SCRAPE-2: Add scraper config constants (#7) 2026-01-31 20:54:43 -07:00
9dba679221 Add scraperLogger.js with credential redaction (#6)
Co-authored-by: Stephen Minakian <stephenminakian@gmail.com>
Co-committed-by: Stephen Minakian <stephenminakian@gmail.com>
2026-01-28 10:52:00 -07:00
ccda2be551 Update tests to expect wrapped response format
- Update all test files to use response.body.data.* instead of
  response.body.* to match the API response convention
- Skip SEC-4.3 rate limiting tests (moved to Phase 5)
- All 202 tests now pass
2026-01-22 14:33:01 -07:00
81e5682ab1 Admin Dashboard Backend (Phases 1-4) (#5)
Some checks failed
Deploy Apartment API / deploy (push) Failing after 5m1s
Co-authored-by: Stephen Minakian <stephenminakian@gmail.com>
Co-committed-by: Stephen Minakian <stephenminakian@gmail.com>
2026-01-22 14:11:43 -07:00