stephen/apartment-dashboard-api
1
0
Fork 0
Code Issues 1 Pull Requests Actions Releases Wiki Activity

Compare commits

base: stephen:10801213864ff7a169095de58be8004f8319e804
Branches Tags
stephen:main stephen:dev stephen:add-soonest-field
...
compare: stephen:fcfd35aa3f7ec108cc882b9a33e27e0f12be1901
Branches Tags
stephen:main stephen:dev stephen:add-soonest-field

3 Commits
1080121386 ... fcfd35aa3f

Author SHA1 Message Date
Stephen Minakian
fcfd35aa3f Update parseUnits tests to match real fixture data
All checks were successful
CI/CD Pipeline - Apartment API / Scan Dependencies (pull_request) Successful in 13s
Details
CI/CD Pipeline - Apartment API / Lint & Test (pull_request) Successful in 43s
Details
CI/CD Pipeline - Apartment API / Send Webhook Notification (pull_request) Successful in 2s
Details
CI/CD Pipeline - Apartment API / Build & Push Image (pull_request) Has been skipped
Details
CI/CD Pipeline - Apartment API / Deploy to Production (pull_request) Has been skipped
Details 
The fixture file was updated with real website data (10 units) but
the tests still referenced old synthetic data (5 units with CCT-*
codes). Updated tests to use real unit codes from the fixture (W2707,
E3205, W2603) and moved edge case tests (image extraction, missing
attributes, unavailable units) to use inline HTML for isolation.
 2026-02-07 10:56:37 -07:00
Stephen Minakian
724115f2b3 Add HTML fixture files for scraper unit tests 
Create three HTML fixture files in __tests__/scraper/fixtures/ to support
deterministic testing of the scraper's HTML parsing logic:

- sample-listing.html: Contains 10 real unit articles extracted from the
  live listings page, covering studios, 1BR, and 2BR floor plans with
  varied pricing and availability dates.

- sample-listing-empty.html: Minimal page structure with an empty units
  section, for testing graceful handling of pages with no listings.

- sample-listing-call.html: Contains units with "Call for pricing" instead
  of numeric rent values, for testing the parser's handling of non-numeric
  price fields.
 2026-02-07 10:53:04 -07:00
Stephen Minakian
6f1651436d Fix webhook notification failure when CI tests produce large output 
Truncate lint/test output to 10000 chars at the source (CI job) before
writing to GITHUB_OUTPUT, instead of in the downstream notify job.
Previously the full output was passed as an env var between jobs, which
could exceed Linux's ARG_MAX limit and prevent bash from launching.
 2026-02-07 10:51:28 -07:00
5 changed files with 503 additions and 211 deletions
Expand all files Collapse all files

14
.github/workflows/deploy.yml vendored
View File

@ -44,8 +44,10 @@ jobs:
set +e set +e
OUTPUT=$(npm run lint 2>&1) OUTPUT=$(npm run lint 2>&1)
EXIT_CODE=$? EXIT_CODE=$?
# Truncate before writing to GITHUB_OUTPUT to prevent
# "argument list too long" in downstream jobs
echo "lint_output<<EOF" >> $GITHUB_OUTPUT echo "lint_output<<EOF" >> $GITHUB_OUTPUT
echo "$OUTPUT" >> $GITHUB_OUTPUT echo "${OUTPUT:0:10000}" >> $GITHUB_OUTPUT
echo "EOF" >> $GITHUB_OUTPUT echo "EOF" >> $GITHUB_OUTPUT
exit $EXIT_CODE exit $EXIT_CODE
@ -56,8 +58,10 @@ jobs:
set +e set +e
OUTPUT=$(npm test -- --runInBand 2>&1) OUTPUT=$(npm test -- --runInBand 2>&1)
EXIT_CODE=$? EXIT_CODE=$?
# Truncate before writing to GITHUB_OUTPUT to prevent
# "argument list too long" in downstream jobs
echo "test_output<<EOF" >> $GITHUB_OUTPUT echo "test_output<<EOF" >> $GITHUB_OUTPUT
echo "$OUTPUT" >> $GITHUB_OUTPUT echo "${OUTPUT:0:10000}" >> $GITHUB_OUTPUT
echo "EOF" >> $GITHUB_OUTPUT echo "EOF" >> $GITHUB_OUTPUT
exit $EXIT_CODE exit $EXIT_CODE
env: env:
@ -98,9 +102,9 @@ jobs:
OVERALL_STATUS="failure" OVERALL_STATUS="failure"
fi fi
# Truncate outputs if too long (max 10000 chars each) # Outputs are already truncated at the source (ci job)
LINT_OUTPUT="${RAW_LINT_OUTPUT:0:10000}" LINT_OUTPUT="$RAW_LINT_OUTPUT"
TEST_OUTPUT="${RAW_TEST_OUTPUT:0:10000}" TEST_OUTPUT="$RAW_TEST_OUTPUT"
# Build JSON payload # Build JSON payload
PAYLOAD=$(jq -n \ PAYLOAD=$(jq -n \

64
__tests__/scraper/fixtures/sample-listing-call.html Normal file
View File

@ -0,0 +1,64 @@
<!DOCTYPE html>
<html>
<head><title>Country Club Towers - Call for Pricing</title></head>
<body>
<section class="spaces__tab-unit">
<article
class="spaces-unit floor_2760 spaces-community-country-club-towers 1bed 1bath spaces-market-country-club-towers-gardens penthouse"
data-spaces-id="9001"
data-spaces-soonest="Now"
data-spaces-sort-date="1706745600"
data-spaces-sort-price="Call for pricing"
data-spaces-sort-area="750"
data-spaces-sort-bed="1"
data-spaces-unit="P-101"
data-spaces-unit-id="9001"
data-spaces-unit-floor="1"
data-spaces-obj="unit"
data-spaces-unavailable="false"
data-spaces-available="true"
data-spaces-community="Country Club Towers"
data-spaces-asset="420"
data-spaces-specials-content=""
data-spaces-plan-id="500"
data-spaces-sort-plan-name="Penthouse A"
data-spaces-sort-bath="1"
data-spaces-href="?spaces_tab=unit-detail&detail=9001"
data-spaces-inventory-href="?spaces_tab=unit-detail&detail=9001"
aria-label="Unit P-101">
<div class="spaces-unit__inner">
<span class="spaces-unit__price">Call for pricing</span>
<span class="spaces-unit__unit">P-101</span>
</div>
</article>
<article
class="spaces-unit floor_2760 spaces-community-country-club-towers 2bed 2bath spaces-market-country-club-towers-gardens penthouse"
data-spaces-id="9002"
data-spaces-soonest="Now"
data-spaces-sort-date="1706745600"
data-spaces-sort-price="Call for pricing"
data-spaces-sort-area="900"
data-spaces-sort-bed="2"
data-spaces-unit="P-102"
data-spaces-unit-id="9002"
data-spaces-unit-floor="2"
data-spaces-obj="unit"
data-spaces-unavailable="false"
data-spaces-available="true"
data-spaces-community="Country Club Towers"
data-spaces-asset="420"
data-spaces-specials-content=""
data-spaces-plan-id="501"
data-spaces-sort-plan-name="Penthouse B"
data-spaces-sort-bath="2"
data-spaces-href="?spaces_tab=unit-detail&detail=9002"
data-spaces-inventory-href="?spaces_tab=unit-detail&detail=9002"
aria-label="Unit P-102">
<div class="spaces-unit__inner">
<span class="spaces-unit__price">Call for pricing</span>
<span class="spaces-unit__unit">P-102</span>
</div>
</article>
</section>
</body>
</html>

9
__tests__/scraper/fixtures/sample-listing-empty.html Normal file
View File

@ -0,0 +1,9 @@
<!DOCTYPE html>
<html>
<head><title>Country Club Towers - No Units</title></head>
<body>
<section class="spaces__tab-unit">
<!-- Empty - no units available -->
</section>
</body>
</html>

426
__tests__/scraper/fixtures/sample-listing.html
View File

@ -1,144 +1,288 @@
<!DOCTYPE html> <!DOCTYPE html>
<html lang="en"> <html>
<head> <head><title>Country Club Towers - Apartments</title></head>
<title>Country Club Towers - Available Units</title>
</head>
<body> <body>
<div class="spaces-container"> <section class="spaces__tab-unit">
<section class="spaces__tab-unit"> <article
<!-- Unit 1: Fully populated with all attributes and an image --> class="spaces-unit price_3500plus floor_2755 spaces-community-country-club-towers 2bed 2bath spaces-market-country-club-towers-gardens"
<article data-spaces-id="154842"
data-spaces-id="1001" data-spaces-soonest="2025-10-11"
data-spaces-unit="CCT-101" data-spaces-sort-date="1760140800"
data-spaces-unit-id="5001" data-spaces-sort-price="5230"
data-spaces-unit-floor="1" data-spaces-sort-area="1210"
data-spaces-sort-area="750" data-spaces-sort-bed="2"
data-spaces-sort-bed="1" data-spaces-unit="W2707"
data-spaces-sort-bath="1" data-spaces-unit-id="154842"
data-spaces-sort-price="1450" data-spaces-unit-floor="2755"
data-spaces-available="true" data-spaces-obj="unit"
data-spaces-unavailable="false" data-spaces-unavailable="false"
data-spaces-soonest="2026-03-01" data-spaces-available="true"
data-spaces-sort-date="20260301" data-spaces-community="Country Club Towers"
data-spaces-plan-id="201" data-spaces-asset="420"
data-spaces-sort-plan-name="Alpine" data-spaces-specials-content=""
data-spaces-obj="unit" data-spaces-plan-id="10270"
data-spaces-community="Country Club Towers" data-spaces-sort-plan-name="Pyramid Peak - Terrace"
data-spaces-asset="100" data-spaces-sort-bath="2"
data-spaces-href="/units/CCT-101" data-spaces-href="?spaces_tab=unit-detail&detail=154842"
data-spaces-inventory-href="/inventory/CCT-101" data-spaces-inventory-href="?spaces_tab=unit-detail&detail=154842"
data-spaces-specials-content="First month free!" aria-label="Unit W2707">
> <div class="spaces-unit__inner">
<div class="unit-card"> <span class="spaces-unit__price">5230</span>
<img src="https://example.com/images/unit-101.jpg" alt="Unit 101" /> <span class="spaces-unit__unit">W2707</span>
<h3>Unit CCT-101</h3> </div>
<p>1 Bed / 1 Bath - $1,450/mo</p> </article>
</div> <article
</article> class="spaces-unit price_2500-3000 floor_2738 spaces-community-country-club-towers 1bed 1bath spaces-market-country-club-towers-gardens"
data-spaces-id="154451"
<!-- Unit 2: Different floor plan, no specials, with data-src image --> data-spaces-soonest="2026-03-08"
<article data-spaces-sort-date="1772928000"
data-spaces-id="1002" data-spaces-sort-price="2767"
data-spaces-unit="CCT-205" data-spaces-sort-area="806"
data-spaces-unit-id="5002" data-spaces-sort-bed="1"
data-spaces-unit-floor="2" data-spaces-unit="E0901"
data-spaces-sort-area="950" data-spaces-unit-id="154451"
data-spaces-sort-bed="2" data-spaces-unit-floor="2738"
data-spaces-sort-bath="1.5" data-spaces-obj="unit"
data-spaces-sort-price="1850" data-spaces-unavailable="false"
data-spaces-available="true" data-spaces-available="true"
data-spaces-unavailable="false" data-spaces-community="Country Club Towers"
data-spaces-soonest="2026-02-15" data-spaces-asset="420"
data-spaces-sort-date="20260215" data-spaces-specials-content=""
data-spaces-plan-id="202" data-spaces-plan-id="8016"
data-spaces-sort-plan-name="Birchwood" data-spaces-sort-plan-name="Sunshine Peak"
data-spaces-obj="unit" data-spaces-sort-bath="1"
data-spaces-community="Country Club Towers" data-spaces-href="?spaces_tab=unit-detail&detail=154451"
data-spaces-asset="100" data-spaces-inventory-href="?spaces_tab=unit-detail&detail=154451"
data-spaces-href="/units/CCT-205" aria-label="Unit E0901">
data-spaces-inventory-href="/inventory/CCT-205" <div class="spaces-unit__inner">
data-spaces-specials-content="" <span class="spaces-unit__price">2767</span>
> <span class="spaces-unit__unit">E0901</span>
<div class="unit-card"> </div>
<img data-src="https://example.com/images/unit-205.jpg" alt="Unit 205" /> </article>
<h3>Unit CCT-205</h3> <article
</div> class="spaces-unit price_2000-2500 floor_2754 spaces-community-country-club-towers 0bed 1bath spaces-market-country-club-towers-gardens"
</article> data-spaces-id="154814"
data-spaces-soonest="2025-12-18"
<!-- Unit 3: Studio with some missing attributes --> data-spaces-sort-date="1766016000"
<article data-spaces-sort-price="2255"
data-spaces-id="1003" data-spaces-sort-area="623"
data-spaces-unit="CCT-310" data-spaces-sort-bed="0"
data-spaces-unit-id="5003" data-spaces-unit="W2603"
data-spaces-unit-floor="3" data-spaces-unit-id="154814"
data-spaces-sort-area="500" data-spaces-unit-floor="2754"
data-spaces-sort-bed="0" data-spaces-obj="unit"
data-spaces-sort-bath="1" data-spaces-unavailable="false"
data-spaces-sort-price="1100" data-spaces-available="true"
data-spaces-available="true" data-spaces-community="Country Club Towers"
data-spaces-unavailable="false" data-spaces-asset="420"
data-spaces-soonest="Available Now" data-spaces-specials-content=""
data-spaces-sort-date="20260201" data-spaces-plan-id="8018"
data-spaces-plan-id="203" data-spaces-sort-plan-name="Little Bear Peak"
data-spaces-sort-plan-name="Cedar" data-spaces-sort-bath="1"
data-spaces-obj="unit" data-spaces-href="?spaces_tab=unit-detail&detail=154814"
data-spaces-community="Country Club Towers" data-spaces-inventory-href="?spaces_tab=unit-detail&detail=154814"
data-spaces-asset="100" aria-label="Unit W2603">
data-spaces-href="/units/CCT-310" <div class="spaces-unit__inner">
data-spaces-inventory-href="/inventory/CCT-310" <span class="spaces-unit__price">2255</span>
> <span class="spaces-unit__unit">W2603</span>
<div class="unit-card"> </div>
<h3>Unit CCT-310</h3> </article>
<p>Studio - $1,100/mo</p> <article
</div> class="spaces-unit price_3500plus floor_2737 spaces-community-country-club-towers 2bed 2bath spaces-market-country-club-towers-gardens"
</article> data-spaces-id="154428"
data-spaces-soonest="2026-02-22"
<!-- Unit 4: Unavailable unit --> data-spaces-sort-date="1771718400"
<article data-spaces-sort-price="4250"
data-spaces-id="1004" data-spaces-sort-area="1152"
data-spaces-unit="CCT-415" data-spaces-sort-bed="2"
data-spaces-unit-id="5004" data-spaces-unit="W0802"
data-spaces-unit-floor="4" data-spaces-unit-id="154428"
data-spaces-sort-area="1200" data-spaces-unit-floor="2737"
data-spaces-sort-bed="3" data-spaces-obj="unit"
data-spaces-sort-bath="2" data-spaces-unavailable="false"
data-spaces-sort-price="2500" data-spaces-available="true"
data-spaces-available="false" data-spaces-community="Country Club Towers"
data-spaces-unavailable="true" data-spaces-asset="420"
data-spaces-soonest="" data-spaces-specials-content=""
data-spaces-sort-date="" data-spaces-plan-id="8017"
data-spaces-plan-id="204" data-spaces-sort-plan-name="Mt. Princeton"
data-spaces-sort-plan-name="Dogwood" data-spaces-sort-bath="2"
data-spaces-obj="unit" data-spaces-href="?spaces_tab=unit-detail&detail=154428"
data-spaces-community="Country Club Towers" data-spaces-inventory-href="?spaces_tab=unit-detail&detail=154428"
data-spaces-asset="100" aria-label="Unit W0802">
data-spaces-href="/units/CCT-415" <div class="spaces-unit__inner">
data-spaces-inventory-href="/inventory/CCT-415" <span class="spaces-unit__price">4250</span>
data-spaces-specials-content="" <span class="spaces-unit__unit">W0802</span>
> </div>
<div class="unit-card"> </article>
<img src="https://example.com/images/unit-415.jpg" alt="Unit 415" /> <article
<h3>Unit CCT-415</h3> class="spaces-unit price_2000-2500 floor_2752 spaces-community-country-club-towers 1bed 1bath spaces-market-country-club-towers-gardens"
</div> data-spaces-id="154778"
</article> data-spaces-soonest="2025-12-31"
data-spaces-sort-date="1767139200"
<!-- Unit 5: Minimal attributes - edge case --> data-spaces-sort-price="2495"
<article data-spaces-sort-area="764"
data-spaces-id="1005" data-spaces-sort-bed="1"
data-spaces-unit="CCT-520" data-spaces-unit="W2405"
data-spaces-unit-id="5005" data-spaces-unit-id="154778"
data-spaces-sort-price="1300" data-spaces-unit-floor="2752"
data-spaces-available="true" data-spaces-obj="unit"
data-spaces-unavailable="false" data-spaces-unavailable="false"
data-spaces-obj="unit" data-spaces-available="true"
data-spaces-community="Country Club Towers" data-spaces-community="Country Club Towers"
data-spaces-asset="100" data-spaces-asset="420"
> data-spaces-specials-content=""
<div class="unit-card"> data-spaces-plan-id="8020"
<h3>Unit CCT-520</h3> data-spaces-sort-plan-name="Grays Peak"
</div> data-spaces-sort-bath="1"
</article> data-spaces-href="?spaces_tab=unit-detail&detail=154778"
</section> data-spaces-inventory-href="?spaces_tab=unit-detail&detail=154778"
</div> aria-label="Unit W2405">
<div class="spaces-unit__inner">
<span class="spaces-unit__price">2495</span>
<span class="spaces-unit__unit">W2405</span>
</div>
</article>
<article
class="spaces-unit price_2000-2500 floor_2737 spaces-community-country-club-towers 1bed 1bath spaces-market-country-club-towers-gardens"
data-spaces-id="154442"
data-spaces-soonest="2025-02-01"
data-spaces-sort-date="1738368000"
data-spaces-sort-price="2495"
data-spaces-sort-area="802"
data-spaces-sort-bed="1"
data-spaces-unit="W0809"
data-spaces-unit-id="154442"
data-spaces-unit-floor="2737"
data-spaces-obj="unit"
data-spaces-unavailable="false"
data-spaces-available="true"
data-spaces-community="Country Club Towers"
data-spaces-asset="420"
data-spaces-specials-content=""
data-spaces-plan-id="8022"
data-spaces-sort-plan-name="Pikes Peak"
data-spaces-sort-bath="1"
data-spaces-href="?spaces_tab=unit-detail&detail=154442"
data-spaces-inventory-href="?spaces_tab=unit-detail&detail=154442"
aria-label="Unit W0809">
<div class="spaces-unit__inner">
<span class="spaces-unit__price">2495</span>
<span class="spaces-unit__unit">W0809</span>
</div>
</article>
<article
class="spaces-unit price_2000-2500 floor_2751 spaces-community-country-club-towers 1bed 1bath spaces-market-country-club-towers-gardens"
data-spaces-id="154764"
data-spaces-soonest="2026-02-17"
data-spaces-sort-date="1771286400"
data-spaces-sort-price="2435"
data-spaces-sort-area="715"
data-spaces-sort-bed="1"
data-spaces-unit="W2308"
data-spaces-unit-id="154764"
data-spaces-unit-floor="2751"
data-spaces-obj="unit"
data-spaces-unavailable="false"
data-spaces-available="true"
data-spaces-community="Country Club Towers"
data-spaces-asset="420"
data-spaces-specials-content=""
data-spaces-plan-id="8023"
data-spaces-sort-plan-name="Longs Peak"
data-spaces-sort-bath="1"
data-spaces-href="?spaces_tab=unit-detail&detail=154764"
data-spaces-inventory-href="?spaces_tab=unit-detail&detail=154764"
aria-label="Unit W2308">
<div class="spaces-unit__inner">
<span class="spaces-unit__price">2435</span>
<span class="spaces-unit__unit">W2308</span>
</div>
</article>
<article
class="spaces-unit price_2500-3000 floor_2744 spaces-community-country-club-towers 1bed 1bath spaces-market-country-club-towers-gardens"
data-spaces-id="154612"
data-spaces-soonest="2026-03-24"
data-spaces-sort-date="1774310400"
data-spaces-sort-price="2683"
data-spaces-sort-area="797"
data-spaces-sort-bed="1"
data-spaces-unit="W1610"
data-spaces-unit-id="154612"
data-spaces-unit-floor="2744"
data-spaces-obj="unit"
data-spaces-unavailable="false"
data-spaces-available="true"
data-spaces-community="Country Club Towers"
data-spaces-asset="420"
data-spaces-specials-content=""
data-spaces-plan-id="8024"
data-spaces-sort-plan-name="Torreys Peak"
data-spaces-sort-bath="1"
data-spaces-href="?spaces_tab=unit-detail&detail=154612"
data-spaces-inventory-href="?spaces_tab=unit-detail&detail=154612"
aria-label="Unit W1610">
<div class="spaces-unit__inner">
<span class="spaces-unit__price">2683</span>
<span class="spaces-unit__unit">W1610</span>
</div>
</article>
<article
class="spaces-unit price_3500plus floor_2739 spaces-community-country-club-towers 2bed 2bath spaces-market-country-club-towers-gardens"
data-spaces-id="154495"
data-spaces-soonest="2025-11-07"
data-spaces-sort-date="1762473600"
data-spaces-sort-price="4470"
data-spaces-sort-area="1230"
data-spaces-sort-bed="2"
data-spaces-unit="E1011"
data-spaces-unit-id="154495"
data-spaces-unit-floor="2739"
data-spaces-obj="unit"
data-spaces-unavailable="false"
data-spaces-available="true"
data-spaces-community="Country Club Towers"
data-spaces-asset="420"
data-spaces-specials-content=""
data-spaces-plan-id="8025"
data-spaces-sort-plan-name="Maroon Peak"
data-spaces-sort-bath="2"
data-spaces-href="?spaces_tab=unit-detail&detail=154495"
data-spaces-inventory-href="?spaces_tab=unit-detail&detail=154495"
aria-label="Unit E1011">
<div class="spaces-unit__inner">
<span class="spaces-unit__price">4470</span>
<span class="spaces-unit__unit">E1011</span>
</div>
</article>
<article
class="spaces-unit price_3500plus floor_2760 spaces-community-country-club-towers 2bed 2.5bath has_tour spaces-market-country-club-towers-gardens penthouse"
data-spaces-id="154919"
data-spaces-soonest="2025-10-27"
data-spaces-sort-date="1761523200"
data-spaces-sort-price="8581"
data-spaces-sort-area="1532"
data-spaces-sort-bed="2"
data-spaces-unit="E3205"
data-spaces-unit-id="154919"
data-spaces-unit-floor="2760"
data-spaces-obj="unit"
data-spaces-unavailable="false"
data-spaces-available="true"
data-spaces-community="Country Club Towers"
data-spaces-asset="420"
data-spaces-specials-content=""
data-spaces-plan-id="8037"
data-spaces-sort-plan-name="Mt. Antero"
data-spaces-sort-bath="2.5"
data-spaces-href="?spaces_tab=unit-detail&detail=154919"
data-spaces-inventory-href="?spaces_tab=unit-detail&detail=154919"
aria-label="Unit E3205">
<div class="spaces-unit__inner">
<span class="spaces-unit__price">8581</span>
<span class="spaces-unit__unit">E3205</span>
</div>
</article>
</section>
</body> </body>
</html> </html>

201
__tests__/scraper/parseUnits.test.js
View File

@ -87,55 +87,55 @@ describe('parseUnits', () => {
describe('attribute extraction', () => { describe('attribute extraction', () => {
it('should extract all expected attributes from a fully populated article', () => { it('should extract all expected attributes from a fully populated article', () => {
const units = parseUnits(sampleHtml, mockLogger); const units = parseUnits(sampleHtml, mockLogger);
const unit1 = units.find(u => u.unit_code === 'CCT-101'); const unit1 = units.find(u => u.unit_code === 'W2707');
expect(unit1).toBeDefined(); expect(unit1).toBeDefined();
// Core identifiers // Core identifiers
expect(unit1.id).toBe('1001'); expect(unit1.id).toBe('154842');
expect(unit1.unit_code).toBe('CCT-101'); expect(unit1.unit_code).toBe('W2707');
expect(unit1.unit_id).toBe('5001'); expect(unit1.unit_id).toBe('154842');
// Physical attributes // Physical attributes
expect(unit1.floor).toBe('1'); expect(unit1.floor).toBe('2755');
expect(unit1.area).toBe('750'); expect(unit1.area).toBe('1210');
expect(unit1.bed_count).toBe('1'); expect(unit1.bed_count).toBe('2');
expect(unit1.bath_count).toBe('1'); expect(unit1.bath_count).toBe('2');
// Pricing // Pricing
expect(unit1.price).toBe('1450'); expect(unit1.price).toBe('5230');
// Availability // Availability
expect(unit1.available).toBe('true'); expect(unit1.available).toBe('true');
expect(unit1.unavailable).toBe('false'); expect(unit1.unavailable).toBe('false');
expect(unit1.soonest).toBe('2026-03-01'); expect(unit1.soonest).toBe('2025-10-11');
expect(unit1.date_available).toBe('20260301'); expect(unit1.date_available).toBe('1760140800');
// Plan information // Plan information
expect(unit1.plan_id).toBe('201'); expect(unit1.plan_id).toBe('10270');
expect(unit1.plan_name).toBe('Alpine'); expect(unit1.plan_name).toBe('Pyramid Peak - Terrace');
// Property information // Property information
expect(unit1.obj_type).toBe('unit'); expect(unit1.obj_type).toBe('unit');
expect(unit1.community).toBe('Country Club Towers'); expect(unit1.community).toBe('Country Club Towers');
expect(unit1.asset).toBe('100'); expect(unit1.asset).toBe('420');
// URLs // URLs
expect(unit1.href).toBe('/units/CCT-101'); expect(unit1.href).toBe('?spaces_tab=unit-detail&detail=154842');
expect(unit1.inventory_href).toBe('/inventory/CCT-101'); expect(unit1.inventory_href).toBe('?spaces_tab=unit-detail&detail=154842');
// Specials // Specials
expect(unit1.specials_content).toBe('First month free!'); expect(unit1.specials_content).toBe('');
}); });
it('should extract correct count of units from the fixture', () => { it('should extract correct count of units from the fixture', () => {
const units = parseUnits(sampleHtml, mockLogger); const units = parseUnits(sampleHtml, mockLogger);
expect(units).toHaveLength(5); expect(units).toHaveLength(10);
}); });
it('should return raw string values without type conversion', () => { it('should return raw string values without type conversion', () => {
const units = parseUnits(sampleHtml, mockLogger); const units = parseUnits(sampleHtml, mockLogger);
const unit = units.find(u => u.unit_code === 'CCT-101'); const unit = units.find(u => u.unit_code === 'W2707');
// All values should be strings (raw extraction, no conversion) // All values should be strings (raw extraction, no conversion)
expect(typeof unit.id).toBe('string'); expect(typeof unit.id).toBe('string');
@ -152,24 +152,48 @@ describe('parseUnits', () => {
// --------------------------------------------------------------- // ---------------------------------------------------------------
describe('image URL extraction', () => { describe('image URL extraction', () => {
it('should extract image_url from img src attribute', () => { it('should extract image_url from img src attribute', () => {
const units = parseUnits(sampleHtml, mockLogger); const html = `
const unit1 = units.find(u => u.unit_code === 'CCT-101'); <html><body>
<section class="spaces__tab-unit">
<article data-spaces-id="1" data-spaces-unit="IMG-1" data-spaces-sort-price="1000">
<img src="https://example.com/images/unit-1.jpg" />
</article>
</section>
</body></html>
`;
const units = parseUnits(html, mockLogger);
expect(unit1.image_url).toBe('https://example.com/images/unit-101.jpg'); expect(units[0].image_url).toBe('https://example.com/images/unit-1.jpg');
}); });
it('should fall back to data-src when src is not present', () => { it('should fall back to data-src when src is not present', () => {
const units = parseUnits(sampleHtml, mockLogger); const html = `
const unit2 = units.find(u => u.unit_code === 'CCT-205'); <html><body>
<section class="spaces__tab-unit">
<article data-spaces-id="1" data-spaces-unit="IMG-2" data-spaces-sort-price="1000">
<img data-src="https://example.com/images/unit-2.jpg" />
</article>
</section>
</body></html>
`;
const units = parseUnits(html, mockLogger);
expect(unit2.image_url).toBe('https://example.com/images/unit-205.jpg'); expect(units[0].image_url).toBe('https://example.com/images/unit-2.jpg');
}); });
it('should not set image_url when no img element exists', () => { it('should not set image_url when no img element exists', () => {
const units = parseUnits(sampleHtml, mockLogger); const html = `
const unit3 = units.find(u => u.unit_code === 'CCT-310'); <html><body>
<section class="spaces__tab-unit">
<article data-spaces-id="1" data-spaces-unit="IMG-3" data-spaces-sort-price="1000">
<div>No image here</div>
</article>
</section>
</body></html>
`;
const units = parseUnits(html, mockLogger);
expect(unit3.image_url).toBeUndefined(); expect(units[0].image_url).toBeUndefined();
}); });
}); });
@ -178,37 +202,67 @@ describe('parseUnits', () => {
// --------------------------------------------------------------- // ---------------------------------------------------------------
describe('missing attributes', () => { describe('missing attributes', () => {
it('should return undefined for attributes not present on the article', () => { it('should return undefined for attributes not present on the article', () => {
const units = parseUnits(sampleHtml, mockLogger); const html = `
// Unit 5 (CCT-520) is missing floor, area, bed_count, bath_count, <html><body>
// soonest, date_available, plan_id, plan_name, href, inventory_href, specials_content <section class="spaces__tab-unit">
const unit5 = units.find(u => u.unit_code === 'CCT-520'); <article
data-spaces-id="1005"
data-spaces-unit="SPARSE-1"
data-spaces-unit-id="5005"
data-spaces-sort-price="1300"
data-spaces-available="true"
data-spaces-obj="unit"
data-spaces-community="Country Club Towers"
data-spaces-asset="100">
</article>
</section>
</body></html>
`;
const units = parseUnits(html, mockLogger);
const unit = units[0];
expect(unit5).toBeDefined(); expect(unit).toBeDefined();
expect(unit5.floor).toBeUndefined(); expect(unit.floor).toBeUndefined();
expect(unit5.area).toBeUndefined(); expect(unit.area).toBeUndefined();
expect(unit5.bed_count).toBeUndefined(); expect(unit.bed_count).toBeUndefined();
expect(unit5.bath_count).toBeUndefined(); expect(unit.bath_count).toBeUndefined();
expect(unit5.soonest).toBeUndefined(); expect(unit.soonest).toBeUndefined();
expect(unit5.date_available).toBeUndefined(); expect(unit.date_available).toBeUndefined();
expect(unit5.plan_id).toBeUndefined(); expect(unit.plan_id).toBeUndefined();
expect(unit5.plan_name).toBeUndefined(); expect(unit.plan_name).toBeUndefined();
expect(unit5.href).toBeUndefined(); expect(unit.href).toBeUndefined();
expect(unit5.inventory_href).toBeUndefined(); expect(unit.inventory_href).toBeUndefined();
expect(unit5.specials_content).toBeUndefined(); expect(unit.specials_content).toBeUndefined();
}); });
it('should still extract the attributes that are present on a sparse article', () => { it('should still extract the attributes that are present on a sparse article', () => {
const units = parseUnits(sampleHtml, mockLogger); const html = `
const unit5 = units.find(u => u.unit_code === 'CCT-520'); <html><body>
<section class="spaces__tab-unit">
<article
data-spaces-id="1005"
data-spaces-unit="SPARSE-1"
data-spaces-unit-id="5005"
data-spaces-sort-price="1300"
data-spaces-available="true"
data-spaces-obj="unit"
data-spaces-community="Country Club Towers"
data-spaces-asset="100">
</article>
</section>
</body></html>
`;
const units = parseUnits(html, mockLogger);
const unit = units[0];
expect(unit5.id).toBe('1005'); expect(unit.id).toBe('1005');
expect(unit5.unit_code).toBe('CCT-520'); expect(unit.unit_code).toBe('SPARSE-1');
expect(unit5.unit_id).toBe('5005'); expect(unit.unit_id).toBe('5005');
expect(unit5.price).toBe('1300'); expect(unit.price).toBe('1300');
expect(unit5.available).toBe('true'); expect(unit.available).toBe('true');
expect(unit5.obj_type).toBe('unit'); expect(unit.obj_type).toBe('unit');
expect(unit5.community).toBe('Country Club Towers'); expect(unit.community).toBe('Country Club Towers');
expect(unit5.asset).toBe('100'); expect(unit.asset).toBe('100');
}); });
}); });
@ -313,7 +367,7 @@ describe('parseUnits', () => {
expect(mockLogger.info).toHaveBeenCalledWith( expect(mockLogger.info).toHaveBeenCalledWith(
'Units parsed', 'Units parsed',
expect.objectContaining({ count: 5 }) expect.objectContaining({ count: 10 })
); );
}); });
@ -334,27 +388,44 @@ describe('parseUnits', () => {
// --------------------------------------------------------------- // ---------------------------------------------------------------
describe('varied unit data', () => { describe('varied unit data', () => {
it('should handle unavailable units correctly', () => { it('should handle unavailable units correctly', () => {
const units = parseUnits(sampleHtml, mockLogger); const html = `
const unit4 = units.find(u => u.unit_code === 'CCT-415'); <html><body>
<section class="spaces__tab-unit">
<article
data-spaces-id="4001"
data-spaces-unit="UNAVAIL-1"
data-spaces-sort-price="2000"
data-spaces-available="false"
data-spaces-unavailable="true"
data-spaces-soonest=""
data-spaces-sort-date="">
</article>
</section>
</body></html>
`;
const units = parseUnits(html, mockLogger);
const unit = units[0];
expect(unit4.available).toBe('false'); expect(unit.available).toBe('false');
expect(unit4.unavailable).toBe('true'); expect(unit.unavailable).toBe('true');
expect(unit4.soonest).toBe(''); expect(unit.soonest).toBe('');
expect(unit4.date_available).toBe(''); expect(unit.date_available).toBe('');
}); });
it('should handle half bath counts', () => { it('should handle half bath counts', () => {
const units = parseUnits(sampleHtml, mockLogger); const units = parseUnits(sampleHtml, mockLogger);
const unit2 = units.find(u => u.unit_code === 'CCT-205'); // E3205 is the penthouse with 2.5 baths
const unit = units.find(u => u.unit_code === 'E3205');
expect(unit2.bath_count).toBe('1.5'); expect(unit.bath_count).toBe('2.5');
}); });
it('should handle studio units (bed_count = 0)', () => { it('should handle studio units (bed_count = 0)', () => {
const units = parseUnits(sampleHtml, mockLogger); const units = parseUnits(sampleHtml, mockLogger);
const unit3 = units.find(u => u.unit_code === 'CCT-310'); // W2603 is the studio with bed_count 0
const unit = units.find(u => u.unit_code === 'W2603');
expect(unit3.bed_count).toBe('0'); expect(unit.bed_count).toBe('0');
}); });
}); });
}); });