6eaf79f662
SCRAPE-8: Implement markStaleUnits() to flag unavailable units
...
CI/CD Pipeline - Apartment API / Scan Dependencies (pull_request) Successful in 13s
CI/CD Pipeline - Apartment API / Run Linting (pull_request) Successful in 9m37s
CI/CD Pipeline - Apartment API / Run Tests (pull_request) Successful in 9m47s
CI/CD Pipeline - Apartment API / Send Webhook Notification (pull_request) Successful in 2s
CI/CD Pipeline - Apartment API / Build & Push Image (pull_request) Has been skipped
CI/CD Pipeline - Apartment API / Deploy to Production (pull_request) Has been skipped
Add markStaleUnits() that marks units as unavailable when they are not
present in the current scrape results. Uses MongoDB updateMany with a
$nin filter on unit_code to efficiently update only units that were
previously available but no longer appear in the scraped data. Sets
available to false and records marked_stale_date.
Includes 18 tests covering filter criteria, $set fields, already
unavailable units, empty currentUnitCodes, error handling, logging,
and return value verification.
2026-02-05 23:37:41 -07:00
2b53288fbe
SCRAPE-7: Implement insertPrices() bulk operation ( #12 )
...
Co-authored-by: Stephen Minakian <stephenminakian@gmail.com >
Co-committed-by: Stephen Minakian <stephenminakian@gmail.com >
2026-02-05 22:32:02 -07:00
4863dc1824
SCRAPE-6: Implement upsertUnits() bulk operation ( #11 )
...
Co-authored-by: Stephen Minakian <stephenminakian@gmail.com >
Co-committed-by: Stephen Minakian <stephenminakian@gmail.com >
2026-02-05 22:10:54 -07:00
0cbeaaf9d5
SCRAPE-5: Implement convertDataTypes() with type parsing helpers ( #10 )
...
Co-authored-by: Stephen Minakian <stephenminakian@gmail.com >
Co-committed-by: Stephen Minakian <stephenminakian@gmail.com >
2026-02-05 21:33:58 -07:00
072e80d069
SCRAPE-4: Implement parseUnits() with cheerio selectors ( #9 )
...
Co-authored-by: Stephen Minakian <stephenminakian@gmail.com >
Co-committed-by: Stephen Minakian <stephenminakian@gmail.com >
2026-02-05 17:14:01 -07:00
c6beaf8333
SCRAPE-3: Implement fetchPage() with axios, timeout, retry ( #8 )
...
Co-authored-by: Stephen Minakian <stephenminakian@gmail.com >
Co-committed-by: Stephen Minakian <stephenminakian@gmail.com >
2026-02-05 15:58:07 -07:00
2993d019c5
SCRAPE-2: Add scraper config constants ( #7 )
2026-01-31 20:54:43 -07:00
9dba679221
Add scraperLogger.js with credential redaction ( #6 )
...
Co-authored-by: Stephen Minakian <stephenminakian@gmail.com >
Co-committed-by: Stephen Minakian <stephenminakian@gmail.com >
2026-01-28 10:52:00 -07:00
ccda2be551
Update tests to expect wrapped response format
...
- Update all test files to use response.body.data.* instead of
response.body.* to match the API response convention
- Skip SEC-4.3 rate limiting tests (moved to Phase 5)
- All 202 tests now pass
2026-01-22 14:33:01 -07:00
81e5682ab1
Admin Dashboard Backend (Phases 1-4) ( #5 )
...
Deploy Apartment API / deploy (push) Failing after 5m1s
Co-authored-by: Stephen Minakian <stephenminakian@gmail.com >
Co-committed-by: Stephen Minakian <stephenminakian@gmail.com >
2026-01-22 14:11:43 -07:00
