stephen/apartment-dashboard-api
1
0
Fork 0
Code Issues 1 Pull Requests Actions Releases Wiki Activity

SCRAPE-2: Add scraper config constants #7

Merged
stephen merged 3 commits from scraper/config into dev 2026-01-31 20:54:44 -07:00
Conversation 1 Commits 3 Files Changed 3 +230 -2
stephen commented 2026-01-29 16:19:50 -07:00
Owner
Copy Link

Create configuration module for scraper constants and settings with environment variable support. Includes 25 test cases.

Create configuration module for scraper constants and settings with environment variable support. Includes 25 test cases.
stephen added 4 commits 2026-01-29 16:19:50 -07:00
Add ESLint and n8n webhook notifications to CI/CD 6daacf4d12 
- Add ESLint 9 with flat config for Node.js linting
- Add lint and lint:fix npm scripts
- Add lint job to CI/CD pipeline
- Add notify job to send test/lint results to n8n webhook
- Webhook reports pass/fail status with failure details

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Add scraperLogger.js with credential redaction (#6) 9dba679221 
Co-authored-by: Stephen Minakian <stephenminakian@gmail.com>
Co-committed-by: Stephen Minakian <stephenminakian@gmail.com>
Add PR number to webhook payload 3af5a90c93 
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Add scraper configuration module with environment variable support
Some checks failed
CI/CD Pipeline - Apartment API / Scan Dependencies (pull_request) Failing after 0s
Details
CI/CD Pipeline - Apartment API / Run Linting (pull_request) Successful in 9m35s
Details
CI/CD Pipeline - Apartment API / Run Tests (pull_request) Successful in 9m45s
Details
CI/CD Pipeline - Apartment API / Send Webhook Notification (pull_request) Successful in 2s
Details
CI/CD Pipeline - Apartment API / Build & Push Image (pull_request) Has been skipped
Details
CI/CD Pipeline - Apartment API / Deploy to Production (pull_request) Has been skipped
Details
7114aee239 
- Create config/scraper.js with all scraper constants
- Add TARGET_URL for apartment listing page
- Add SCRAPER_SCHEDULE, SCRAPER_TIMEZONE, SCRAPER_ENABLED settings
- Add SCRAPER_TIMEOUT and USER_AGENT settings
- Add RETRY_CONFIG with maxRetries, baseDelay, timeout
- Add COLLECTIONS object with env var overrides for all 4 collections
- Create comprehensive test suite with 25 test cases
stephen changed target branch from main to dev 2026-01-29 16:24:55 -07:00
stephen added 1 commit 2026-01-31 20:10:16 -07:00
Update .github/workflows/deploy.yml 30aea78d8c
stephen added 1 commit 2026-01-31 20:13:08 -07:00
Update .github/workflows/deploy.yml
All checks were successful
CI/CD Pipeline - Apartment API / Scan Dependencies (pull_request) Successful in 15s
Details
CI/CD Pipeline - Apartment API / Run Linting (pull_request) Successful in 9m35s
Details
CI/CD Pipeline - Apartment API / Run Tests (pull_request) Successful in 9m46s
Details
CI/CD Pipeline - Apartment API / Send Webhook Notification (pull_request) Successful in 2s
Details
CI/CD Pipeline - Apartment API / Build & Push Image (pull_request) Has been skipped
Details
CI/CD Pipeline - Apartment API / Deploy to Production (pull_request) Has been skipped
Details
7a10aa3036
stephen merged commit 2993d019c5 into dev 2026-01-31 20:54:44 -07:00
stephen deleted branch scraper/config 2026-01-31 20:54:44 -07:00
stephen commented 2026-02-05 14:20:13 -07:00
Author
Owner
Copy Link

❌ Pipeline Failed

Lint: ❌ Failed

This is a test lint failure

Test: ❌ Failed

This is a test failure.
## ❌ Pipeline Failed ### Lint: ❌ Failed ``` This is a test lint failure ``` ### Test: ❌ Failed ``` This is a test failure. ```
Sign in to join this conversation.
Reviewers
No Reviewers
Labels
No items
No Label
Milestone
No items
No Milestone
Assignees
Clear assignees
stephen
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: stephen/apartment-dashboard-api#7
No description provided.
Delete Branch "scraper/config"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?