StorageSecurity/Particle/docs/troubleshooting.md

# Particle Boron Security Device Troubleshooting Guide

Comprehensive troubleshooting guide for diagnosing and resolving common issues with the ultra-low-power security device system.

## 🚨 Quick Diagnostic Checklist

### Emergency Troubleshooting (Device Not Responding)
```
□ Check power LED on Boron (should be solid/breathing)
□ Verify battery voltage (>3.0V for operation)
□ Test microswitch manually (should trigger alarm)
□ Check cellular signal strength (>-85dBm preferred)
□ Verify SIM card seating and activation
□ Try safe mode recovery (SETUP+RESET procedure)
```

### System Health Indicators
```
✅ HEALTHY SYSTEM:
- Status LED: Occasional brief flashes
- Battery reports: Daily via webhook
- Alarm response: <1 second trigger time
- Cellular connection: <60 seconds to connect
- Sleep current: 130μA ± 20μA

⚠️ WARNING SIGNS:
- No daily battery reports for >48 hours
- Alarm delay >5 seconds
- High power consumption (battery drain)
- Frequent connection failures
- Unexpected device resets

🚨 CRITICAL ISSUES:
- No response to manual testing
- Device constantly resetting
- Power consumption >1mA in sleep
- Complete loss of cellular connectivity
- Mechanical failure of sensors/alarm
```

## 🔋 Power and Battery Issues

### Problem: High Power Consumption

**Symptoms:**
- Battery draining faster than expected
- Device feels warm to touch
- Sleep current >200μA

**Diagnostic Steps:**
```bash
# 1. Measure sleep current
# Expected: 130μA ± 20μA
# Use μCurrent Gold or similar precision meter

# 2. Check for software issues
particle serial monitor
# Look for continuous output (should be silent in sleep)

# 3. Verify sleep mode entry
# Device should enter STOP mode within 10 seconds of last activity
```

**Common Causes and Solutions:**

**Serial Interface Left Open:**
```cpp
// ❌ WRONG - Prevents sleep
void setup() {
    Serial.begin(9600);  // Remove this line!
    // rest of setup
}

// ✅ CORRECT - No serial in production
void setup() {
    // Serial.begin(9600);  // Commented out
    // rest of setup
}
```

**Incorrect Pin Configuration:**
```cpp
// ✅ Add to setup() function
void configurePowerOptimalPins() {
    // Configure unused pins to prevent floating
    pinMode(A0, INPUT_PULLDOWN);
    pinMode(A1, INPUT_PULLDOWN);
    pinMode(A2, INPUT_PULLDOWN);
    pinMode(A3, INPUT_PULLDOWN);
    pinMode(A4, INPUT_PULLDOWN);
    pinMode(A5, INPUT_PULLDOWN);
    pinMode(D0, INPUT_PULLDOWN);
    pinMode(D1, INPUT_PULLDOWN);
    pinMode(D4, INPUT_PULLDOWN);
    pinMode(D5, INPUT_PULLDOWN);
    pinMode(D6, INPUT_PULLDOWN);
    pinMode(D8, INPUT_PULLDOWN);
    
    // Keep D2 (sensor), D3 (alarm), D7 (status) as configured
}
```

**Stuck in Active Loop:**
```cpp
// ❌ WRONG - Prevents sleep
void loop() {
    while (someCondition) {
        // Infinite loop prevents sleep
    }
}

// ✅ CORRECT - Always allow sleep
void loop() {
    if (someCondition) {
        // Handle condition
    }
    // Loop exits, device can sleep
}
```

### Problem: Battery Not Charging (Solar Setup)

**Symptoms:**
- Battery level decreasing despite solar panel
- Charge controller not showing charging status

**Diagnostic Steps:**
```bash
# 1. Check solar panel voltage
# Expected: 6V+ in direct sunlight
multimeter across panel terminals

# 2. Check charge controller input/output
# Input: Should match panel voltage
# Output: Should be 4.1-4.2V when charging

# 3. Check battery connection
# Ensure JST connector properly seated
# Verify polarity (red=+, black=-)
```

**Solutions:**
- Clean solar panel surface
- Check all connections for corrosion
- Verify charge controller wiring
- Test battery capacity with external charger

### Problem: Inaccurate Battery Readings

**Symptoms:**
- Battery percentage jumps unexpectedly
- Reports 0% or >100% charge

**Solution:**
```cpp
// Add battery reading validation
float getValidBatteryLevel() {
    float reading = System.batteryCharge();
    
    // Filter obviously invalid readings
    if (reading < 0 || reading > 100) {
        return lastValidReading;  // Use previous value
    }
    
    // Filter large jumps (>20% change)
    if (abs(reading - lastValidReading) > 20) {
        return lastValidReading;  // Reject sudden changes
    }
    
    lastValidReading = reading;
    return reading;
}
```

## 📡 Cellular and Connectivity Issues

### Problem: Device Won't Connect to Cellular

**Symptoms:**
- No daily battery reports received
- Events show "Connection timeout" in logs
- Cellular LED patterns indicate no connection

**Diagnostic Steps:**

**Check Signal Strength:**
```bash
# Use Particle CLI to check signal
particle get YOUR_DEVICE_ID signal

# Expected values:
# >-70dBm: Excellent
# -70 to -85dBm: Good  
# -85 to -100dBm: Fair
# <-100dBm: Poor (may not connect)
```

**Check SIM Card Status:**
```bash
# Verify SIM is active
particle sim list

# Check data usage
particle sim usage YOUR_SIM_ID
```

**Test Manual Connection:**
```cpp
// Add to device code for testing
void testCellularConnection() {
    Serial.begin(9600);
    Serial.println("Testing cellular connection...");
    
    Cellular.on();
    Serial.println("Cellular radio on");
    
    waitFor(Cellular.ready, 60000);
    if (Cellular.ready()) {
        Serial.println("Cellular ready");
        
        CellularSignal signal = Cellular.RSSI();
        Serial.printf("Signal: %d dBm\n", signal.getStrength());
        Serial.printf("Quality: %d\n", signal.getQuality());
    } else {
        Serial.println("Cellular failed to connect");
    }
}
```

**Common Solutions:**

**Poor Signal Strength:**
- Relocate device to higher position
- Install external cellular antenna
- Use cellular signal booster
- Switch to different carrier if available

**SIM Card Issues:**
- Reseat SIM card in holder
- Check SIM activation status
- Verify data plan is active
- Replace SIM if damaged

**Network Configuration:**
```cpp
// Force specific network settings if needed
void setNetworkParameters() {
    // Set APN if required by carrier
    Cellular.setActiveSim(EXTERNAL_SIM);
    
    // Set specific bands if needed
    // Cellular.setBandSelect("...");
}
```

### Problem: Slow Cellular Connections

**Symptoms:**
- Connection takes >2 minutes
- High power consumption during connection
- Timeouts during data transfer

**Solutions:**

**Optimize Connection Sequence:**
```cpp
bool fastCellularConnect() {
    unsigned long start = millis();
    
    // Enable cellular with timeout
    Cellular.on();
    if (!waitFor(Cellular.ready, 30000)) {
        Serial.println("Cellular radio timeout");
        return false;
    }
    
    // Connect to Particle cloud with timeout
    Particle.connect();
    if (!waitFor(Particle.connected, 60000)) {
        Serial.println("Cloud connection timeout");
        return false;
    }
    
    Serial.printf("Connected in %lu seconds\n", (millis() - start) / 1000);
    return true;
}
```

**Use Connection Keep-Alive:**
```cpp
void setup() {
    // Maintain network registration between sleeps
    Particle.keepAlive(120);  // 2 minutes
}
```

### Problem: Data Usage Higher Than Expected

**Symptoms:**
- Monthly data usage >10MB
- Carrier data overage charges
- Frequent connection events

**Analysis:**
```bash
# Check Particle data usage
particle sim usage YOUR_SIM_ID

# Review webhook frequency
# Should be 1-2 events per day normally
```

**Solutions:**

**Reduce Message Size:**
```cpp
// Compact data format
String createCompactAlert() {
    // Before: "Type:SECURITY_BREACH|Description:Motion detected|Battery:85.2%|Signal:-45dBm"
    // After: "T:BREACH|D:Motion|B:85|S:-45"
    
    return String::format("T:%s|D:%s|B:%d|S:%d",
        getAlertTypeCode(),
        getShortDescription(), 
        (int)System.batteryCharge(),
        Cellular.RSSI().getStrength()
    );
}
```

**Batch Multiple Events:**
```cpp
void batchEventReporting() {
    static String eventBuffer = "";
    
    eventBuffer += createEventData() + ";";
    
    // Send when buffer reaches size limit or time limit
    if (eventBuffer.length() > 200 || shouldSendBatch()) {
        Particle.publish("Batch_Events", eventBuffer, PRIVATE);
        eventBuffer = "";
    }
}
```

## 🔧 Hardware Issues

### Problem: Microswitch Not Triggering

**Symptoms:**
- No alarm when switch is manually opened
- Security breaches not detected
- Switch reads same value regardless of position

**Diagnostic Steps:**

**Test Switch Continuity:**
```bash
# Use multimeter to test switch
# Closed position: 0Ω resistance
# Open position: ∞Ω (no continuity)
```

**Test Boron Input:**
```cpp
// Add diagnostic code
void testMicroswitchInput() {
    Serial.begin(9600);
    
    while (true) {
        bool switchState = digitalRead(MICROSWITCH_PIN);
        Serial.printf("Switch state: %s\n", switchState ? "CLOSED" : "OPEN");
        delay(500);
    }
}
```

**Common Issues and Solutions:**

**Wiring Error:**
```
✅ CORRECT NC WIRING:
Microswitch Common → D2 (Boron)
Microswitch NC → GND (Boron)

❌ INCORRECT (NO/Normal Open):
Microswitch Common → D2 (Boron)  
Microswitch NO → GND (Boron)
```

**Mechanical Issues:**
- Check switch mounting alignment
- Verify actuator travel distance
- Lubricate switch if sticky
- Replace switch if worn out

**Electrical Issues:**
- Check for loose connections
- Verify wire gauge (22-18 AWG recommended)
- Test for electrical noise/interference
- Add hardware debouncing if needed

### Problem: Alarm Not Activating

**Symptoms:**
- Switch triggers detected but no alarm sound
- Alarm device doesn't activate
- D3 pin voltage incorrect

**Diagnostic Steps:**

**Test GPIO Output:**
```cpp
void testAlarmOutput() {
    Serial.begin(9600);
    
    Serial.println("Testing alarm output...");
    
    // Test D3 voltage
    digitalWrite(ALARM_PIN, HIGH);
    Serial.println("D3 should be 3.3V - measure with multimeter");
    delay(5000);
    
    digitalWrite(ALARM_PIN, LOW);
    Serial.println("D3 should be 0V - measure with multimeter");
    delay(5000);
}
```

**Check Driver Circuit:**
```
For transistor driver circuit:
1. Measure base voltage (should be 3.3V when D3=HIGH)
2. Check transistor collector voltage (should be ~0V when ON)
3. Verify alarm device voltage across terminals
4. Test transistor with known good component
```

**Common Solutions:**

**Driver Circuit Issues:**
```
Component Checklist:
□ 2N2222 transistor (or equivalent NPN)
□ 1kΩ base resistor
□ Proper ground connections
□ Adequate current rating for alarm device

Voltage Tests:
□ D3 output: 0V (LOW) / 3.3V (HIGH)
□ Base voltage: 0V (OFF) / 3.3V (ON)
□ Collector: VIN (OFF) / ~0V (ON)
□ Emitter: Always 0V (GND)
```

**Alarm Device Issues:**
- Check power requirements vs available current
- Verify voltage compatibility
- Test alarm device independently
- Check for blown fuse in alarm circuit

### Problem: Status LED Behavior Issues

**Symptoms:**
- LED constantly on/off
- Unexpected flash patterns
- LED not responding to events

**Normal LED Patterns:**
```
Startup:           3 slow blinks (500ms each)
Security Breach:   5 rapid blinks (100ms each)  
Daily Report:      2 medium blinks (200ms each)
Cellular Connect:  Alternating during connection
Success:           2 quick blinks (100ms each)
Error:             10 rapid blinks (50ms each)
```

**Diagnostic Code:**
```cpp
void testStatusLED() {
    Serial.begin(9600);
    Serial.println("Testing status LED patterns...");
    
    // Test each pattern
    flashStatusLED(3, 500);   // Startup pattern
    delay(2000);
    flashStatusLED(5, 100);   // Breach pattern  
    delay(2000);
    flashStatusLED(2, 200);   // Report pattern
}
```

## 🌐 Webhook and Server Issues

### Problem: Webhook Not Receiving Events

**Symptoms:**
- Particle Console shows events published
- No notifications received
- Webhook service logs show no activity

**Diagnostic Steps:**

**Test Particle Event Publishing:**
```bash
# Check if device is publishing events
particle subscribe YOUR_DEVICE_ID

# Manually trigger test event
particle publish "Security Alert" "Type:TEST|Desc:Manual test" --private
```

**Test Webhook Endpoint:**
```bash
# Test webhook health
curl https://webhook.yourdomain.com/health

# Test webhook with manual POST
curl -X POST https://webhook.yourdomain.com/webhook \
  -H "Content-Type: application/json" \
  -H "User-Agent: ParticleBot/1.0" \
  -H "Authorization: Bearer YOUR_PARTICLE_SECRET" \
  -d '{"event":"Security Alert","data":"Type:TEST|Desc:Manual webhook test","published_at":"2024-01-07T12:00:00Z","coreid":"test","device_name":"Test Device"}'
```

**Check Webhook Configuration:**

**Particle Console Webhook Settings:**
```json
{
  "event": "Security Alert",
  "url": "https://webhook.yourdomain.com/webhook",
  "headers": {
    "Authorization": "Bearer YOUR_SECRET_HERE",
    "Content-Type": "application/json",
    "User-Agent": "ParticleBot/1.0"
  }
}
```

**Common Solutions:**

**Authentication Issues:**
```bash
# Verify webhook secret matches
docker exec webhook-service env | grep PARTICLE_WEBHOOK_SECRET

# Check webhook service logs
docker compose logs webhook-service | grep -i auth
```

**SSL/TLS Issues:**
```bash
# Test SSL certificate
openssl s_client -connect webhook.yourdomain.com:443

# Check certificate expiration
curl -vI https://webhook.yourdomain.com/health
```

**Network Connectivity:**
```bash
# Test from external network
curl -I https://webhook.yourdomain.com/health

# Check DNS resolution
nslookup webhook.yourdomain.com
```

### Problem: Notifications Not Being Sent

**Symptoms:**
- Webhook receives events successfully
- No email/SMS notifications received
- Webhook logs show email errors

**Diagnostic Steps:**

**Test SMTP Configuration:**
```bash
# Test Gmail SMTP from webhook container
docker exec webhook-service python3 -c "
import smtplib
import os
try:
    with smtplib.SMTP_SSL('smtp.gmail.com', 465) as server:
        server.login(os.environ['SMTP_EMAIL'], os.environ['SMTP_PASSWORD'])
    print('SMTP connection successful')
except Exception as e:
    print(f'SMTP error: {e}')
"
```

**Common Solutions:**

**Gmail Authentication Issues:**
```
Common Problems:
□ Using regular password instead of app password
□ App password contains spaces (remove them)
□ 2FA not enabled on Google account
□ App password expired or revoked

Solution:
1. Enable 2FA on Google account
2. Generate new app password at https://myaccount.google.com/apppasswords
3. Use 16-character app password (no spaces)
4. Update .env file with new password
```

**SMS Gateway Issues:**
```
Common Carrier Gateways:
□ Verizon: number@vtext.com
□ AT&T: number@txt.att.net  
□ T-Mobile: number@tmomail.net
□ Sprint: number@messaging.sprintpcs.com

Test SMS Gateway:
1. Send test email to your SMS gateway address
2. Verify format: 10-digit number (no dashes/spaces)
3. Check carrier-specific limitations
```

## 🔄 Device Reset and Recovery

### Safe Mode Recovery

**When to Use:**
- Device constantly resetting
- Firmware appears corrupted
- Device not responding to OTA updates

**Procedure:**
```bash
# Enter safe mode
1. Hold SETUP button
2. Press and release RESET button (while holding SETUP)
3. Continue holding SETUP until LED breathes magenta
4. Release SETUP button

# Device is now in safe mode (breathing magenta)
# Flash firmware via CLI
particle flash YOUR_DEVICE_ID security_device.ino --target 5.8.0
```

### DFU Mode Recovery

**When to Use:**
- Safe mode doesn't work
- Device completely unresponsive
- Need to flash system firmware

**Procedure:**
```bash
# Enter DFU mode
1. Hold SETUP and RESET buttons
2. Release RESET while continuing to hold SETUP
3. Wait for LED to blink yellow rapidly
4. Release SETUP button

# Device is now in DFU mode (yellow blinking)
# Flash tinker firmware to reset
particle flash --usb tinker

# Then flash your application
particle flash YOUR_DEVICE_ID security_device.ino
```

### Factory Reset

**Complete Device Reset:**
```bash
# Clear all user firmware and data
particle device doctor YOUR_DEVICE_ID

# This will:
# - Reset device keys
# - Clear user firmware  
# - Reset to factory state
# - Require device re-claiming
```

## 🔍 Advanced Diagnostics

### Real-Time Monitoring Setup

**Hardware Debug Setup:**
```
Equipment Needed:
□ Logic analyzer or oscilloscope
□ μCurrent Gold (current measurement)
□ Multimeter
□ USB-to-serial adapter
□ Jumper wires and test probes

Monitoring Points:
□ VIN: Power supply voltage
□ D2: Microswitch state changes  
□ D3: Alarm output signal
□ D7: Status LED activity
□ TX/RX: Serial debug output (if enabled)
```

**Software Debug Mode:**
```cpp
// Enable comprehensive debugging (REMOVE in production)
#define DEBUG_MODE
#ifdef DEBUG_MODE

void debugLog(String component, String message) {
    String timestamp = String(millis());
    Serial.println(timestamp + " [" + component + "] " + message);
}

void debugSystemState() {
    debugLog("POWER", "Battery: " + String(System.batteryCharge()) + "%");
    debugLog("NETWORK", "Signal: " + String(Cellular.RSSI().getStrength()) + "dBm");
    debugLog("GPIO", "D2: " + String(digitalRead(D2)) + ", D3: " + String(digitalRead(D3)));
    debugLog("MEMORY", "Free RAM: " + String(System.freeMemory()) + " bytes");
    debugLog("SYSTEM", "Uptime: " + String(millis() / 1000) + " seconds");
}

#else
#define debugLog(component, message)
#define debugSystemState()
#endif
```

### Performance Profiling

**Power Consumption Profiling:**
```cpp
void profilePowerConsumption() {
    unsigned long start, duration;
    
    // Profile cellular connection
    start = millis();
    Cellular.on();
    waitFor(Cellular.ready, 60000);
    duration = millis() - start;
    debugLog("PROFILE", "Cellular ready: " + String(duration) + "ms");
    
    // Profile cloud connection
    start = millis();
    Particle.connect();
    waitFor(Particle.connected, 60000);
    duration = millis() - start;
    debugLog("PROFILE", "Cloud connected: " + String(duration) + "ms");
    
    // Profile event publishing
    start = millis();
    Particle.publish("Test", "data", PRIVATE);
    duration = millis() - start;
    debugLog("PROFILE", "Event published: " + String(duration) + "ms");
}
```

**Memory Usage Monitoring:**
```cpp
void monitorMemoryUsage() {
    static unsigned long lastCheck = 0;
    static int minFreeMemory = 999999;
    
    if (millis() - lastCheck > 10000) {  // Check every 10 seconds
        int freeMemory = System.freeMemory();
        if (freeMemory < minFreeMemory) {
            minFreeMemory = freeMemory;
        }
        
        debugLog("MEMORY", "Free: " + String(freeMemory) + 
                          ", Min: " + String(minFreeMemory) + " bytes");
        lastCheck = millis();
    }
}
```

### Log Analysis and Pattern Detection

**Event Pattern Analysis:**
```python
# Python script to analyze webhook logs
import re
import json
from datetime import datetime
from collections import defaultdict

def analyze_webhook_logs(log_file):
    events = defaultdict(list)
    power_data = []
    
    with open(log_file, 'r') as f:
        for line in f:
            # Parse timestamp and event type
            if 'Received webhook:' in line:
                timestamp = extract_timestamp(line)
                event_data = extract_event_data(line)
                events[event_data['type']].append(timestamp)
                
                if 'Batt:' in line:
                    battery = extract_battery_level(line)
                    power_data.append((timestamp, battery))
    
    # Analyze patterns
    print(f"Security breaches: {len(events['SECURITY_BREACH'])}")
    print(f"Daily reports: {len(events['DAILY_BATTERY_REPORT'])}")
    print(f"Average time between breaches: {calculate_average_interval(events['SECURITY_BREACH'])}")
    
    # Battery trend analysis
    if len(power_data) > 10:
        battery_trend = calculate_battery_trend(power_data)
        print(f"Battery drain rate: {battery_trend:.2f}% per day")

def extract_battery_level(line):
    match = re.search(r'Batt:(\d+\.?\d*)%', line)
    return float(match.group(1)) if match else None

# Usage: python analyze_logs.py webhook.log
```

### Network Diagnostics

**Cellular Network Analysis:**
```cpp
void analyzeCellularNetwork() {
    CellularSignal signal = Cellular.RSSI();
    CellularData data = Cellular.getDataUsage();
    
    debugLog("CELLULAR", "Provider: " + Cellular.getOperator());
    debugLog("CELLULAR", "Technology: " + String(Cellular.getTechnology()));
    debugLog("CELLULAR", "Signal Strength: " + String(signal.getStrength()) + " dBm");
    debugLog("CELLULAR", "Signal Quality: " + String(signal.getQuality()));
    debugLog("CELLULAR", "Data Used: " + String(data.tx_session + data.rx_session) + " bytes");
    
    // Test connection speed
    unsigned long start = millis();
    if (Particle.publish("Speed_Test", "test", PRIVATE)) {
        unsigned long latency = millis() - start;
        debugLog("CELLULAR", "Publish latency: " + String(latency) + "ms");
    }
}
```

**DNS and Connectivity Testing:**
```cpp
void testConnectivity() {
    // Test different endpoints
    String endpoints[] = {
        "api.particle.io",
        "device.spark.io", 
        "google.com",
        "webhook.yourdomain.com"
    };
    
    for (String endpoint : endpoints) {
        unsigned long start = millis();
        TCPClient client;
        
        if (client.connect(endpoint, 80)) {
            unsigned long connectionTime = millis() - start;
            debugLog("CONNECTIVITY", endpoint + " connected in " + String(connectionTime) + "ms");
            client.stop();
        } else {
            debugLog("CONNECTIVITY", endpoint + " connection failed");
        }
    }
}
```

## 📊 Preventive Maintenance

### Weekly Automated Checks

**Device Self-Test Routine:**
```cpp
bool performWeeklyHealthCheck() {
    bool allTestsPassed = true;
    
    // Test 1: Battery level check
    float battery = System.batteryCharge();
    if (battery < 20.0) {
        publishAlert("MAINTENANCE", "Low battery: " + String(battery) + "%");
        allTestsPassed = false;
    }
    
    // Test 2: Cellular signal strength
    int signal = Cellular.RSSI().getStrength();
    if (signal < -90) {
        publishAlert("MAINTENANCE", "Weak signal: " + String(signal) + "dBm");
        allTestsPassed = false;
    }
    
    // Test 3: Memory usage check
    int freeMemory = System.freeMemory();
    if (freeMemory < 10000) {
        publishAlert("MAINTENANCE", "Low memory: " + String(freeMemory) + " bytes");
        allTestsPassed = false;
    }
    
    // Test 4: Switch operation test
    bool switchState = digitalRead(MICROSWITCH_PIN);
    // Record for trend analysis
    
    // Test 5: Alarm circuit test (brief pulse)
    digitalWrite(ALARM_PIN, HIGH);
    delay(100);
    digitalWrite(ALARM_PIN, LOW);
    
    if (allTestsPassed) {
        publishAlert("MAINTENANCE", "Weekly health check: PASS");
    }
    
    return allTestsPassed;
}
```

### Monthly Maintenance Tasks

**Automated Maintenance Reports:**
```python
# Server-side maintenance analysis
def generate_monthly_report(device_id, start_date, end_date):
    """Generate comprehensive device health report"""
    
    events = fetch_device_events(device_id, start_date, end_date)
    
    report = {
        'device_id': device_id,
        'period': f"{start_date} to {end_date}",
        'statistics': analyze_device_statistics(events),
        'battery_health': analyze_battery_trend(events),
        'connectivity': analyze_connectivity_stats(events),
        'security_events': count_security_events(events),
        'maintenance_needed': identify_maintenance_needs(events),
        'recommendations': generate_recommendations(events)
    }
    
    return report

def identify_maintenance_needs(events):
    """Identify potential maintenance requirements"""
    needs = []
    
    # Check battery degradation
    battery_trend = calculate_battery_trend(events)
    if battery_trend > 2.0:  # >2% per day indicates issues
        needs.append("Battery replacement recommended")
    
    # Check false alarm rate
    false_alarm_rate = calculate_false_alarm_rate(events)
    if false_alarm_rate > 0.1:  # >10% false alarms
        needs.append("Sensor calibration needed")
    
    # Check connectivity issues
    connection_success_rate = calculate_connection_success_rate(events)
    if connection_success_rate < 0.95:  # <95% success
        needs.append("Cellular antenna check recommended")
    
    return needs
```

### Predictive Maintenance

**Early Warning System:**
```cpp
void predictiveMaintenance() {
    static float batteryHistory[30];  // 30-day history
    static int historyIndex = 0;
    static bool historyFull = false;
    
    // Record daily battery level
    float currentBattery = System.batteryCharge();
    batteryHistory[historyIndex] = currentBattery;
    historyIndex = (historyIndex + 1) % 30;
    if (historyIndex == 0) historyFull = true;
    
    if (historyFull) {
        // Calculate battery drain trend
        float trend = calculateLinearTrend(batteryHistory, 30);
        
        // Predict battery life remaining
        float daysRemaining = currentBattery / abs(trend);
        
        if (daysRemaining < 90) {  // Less than 3 months
            publishAlert("PREDICTIVE", "Battery replacement needed in " + 
                        String((int)daysRemaining) + " days");
        }
    }
}

float calculateLinearTrend(float data[], int length) {
    // Simple linear regression to find trend
    float sumX = 0, sumY = 0, sumXY = 0, sumX2 = 0;
    
    for (int i = 0; i < length; i++) {
        sumX += i;
        sumY += data[i];
        sumXY += i * data[i];
        sumX2 += i * i;
    }
    
    float slope = (length * sumXY - sumX * sumY) / (length * sumX2 - sumX * sumX);
    return slope;  // % per day
}
```

## 🔧 Field Repair Procedures

### Emergency Field Diagnostics

**Minimal Equipment Diagnostics:**
```
Required Tools:
□ Multimeter
□ Small screwdriver set
□ Spare battery (charged)
□ Mobile phone with Particle app

Quick Tests (5 minutes):
1. Check battery voltage (>3.0V)
2. Test microswitch continuity  
3. Verify alarm device operation
4. Check cellular signal strength via Particle app
5. Attempt manual event trigger
```

**Field Diagnostic Checklist:**
```
□ Visual inspection for physical damage
□ Check all wire connections for looseness
□ Verify enclosure seal integrity
□ Test battery voltage under load
□ Check for signs of water ingress
□ Verify antenna connections (if external)
□ Test switch mechanical operation
□ Check for insect/rodent interference
```

### Remote Diagnostic Commands

**Emergency Remote Access:**
```bash
# Check device vitals
particle get YOUR_DEVICE_ID vitals

# Force connection test
particle function call YOUR_DEVICE_ID testConnection

# Get detailed diagnostics
particle function call YOUR_DEVICE_ID systemDiagnostics

# Force immediate battery report
particle function call YOUR_DEVICE_ID forceBatteryReport
```

**Remote Recovery Commands:**
```bash
# Safe restart
particle function call YOUR_DEVICE_ID safeRestart

# Force sleep mode
particle function call YOUR_DEVICE_ID forceSleep

# Emergency OTA update
particle flash YOUR_DEVICE_ID firmware_recovery.ino

# Factory reset (last resort)
particle device doctor YOUR_DEVICE_ID
```

## 📋 Troubleshooting Decision Tree

```
Device Issue Reported
        │
        ▼
   Is device responsive?
    ┌─NO─┐        ┌─YES─┐
    │    │        │     │
    ▼    │        ▼     │
Power Check    Check specific symptom
    │    │        │     │
    │    │        ▼     │
    │    │   Network Issue?
    │    │    ┌─YES─┐   ┌─NO─┐
    │    │    │     │   │    │
    │    │    ▼     │   ▼    │
    │    │ Cellular │Hardware│
    │    │Diagnostic│ Check  │
    │    │    │     │   │    │
    │    │    ▼     │   ▼    │
    │    │ Signal   │Sensor  │
    │    │  Test    │ Test   │
    │    │    │     │   │    │
    ▼    ▼    ▼     ▼   ▼    ▼
  Replace  Check   Fix  Test  Replace
  Battery  Power   Network Switch Component
     │    Supply     │    │      │
     ▼       │       ▼    ▼      ▼
   Test      │    Restart Reset  Test
   Device    ▼     Device System Function
     │    Replace      │    │      │
     ▼    Hardware     ▼    ▼      ▼
  Functional?       Works? Works? Works?
     │                │     │      │
     ▼                ▼     ▼      ▼
   SUCCESS         SUCCESS SUCCESS SUCCESS
```

## 📞 Support Escalation

### When to Escalate

**Level 1: Self-Service (User)**
- Basic connectivity issues
- Simple configuration changes
- Battery replacement
- Environmental factors

**Level 2: Technical Support**
- Persistent connectivity issues
- Hardware component failures
- Firmware corruption
- Complex network problems

**Level 3: Engineering Support**
- Design modifications needed
- Systematic reliability issues
- Performance optimization
- Custom firmware requirements

### Information to Gather Before Escalation

**Device Information:**
```
□ Device ID and name
□ Firmware version
□ Hardware revision
□ SIM card ID
□ Installation date
□ Last known working state
□ Environmental conditions
```

**Issue Documentation:**
```
□ Detailed symptom description
□ Frequency of occurrence
□ Steps already attempted
□ Error messages observed
□ Timeline of events
□ Impact on operations
```

**Diagnostic Data:**
```
□ Power consumption measurements
□ Signal strength readings
□ Event logs from webhook
□ Photos of hardware setup
□ Particle Console screenshots
□ Network configuration details
```

---

**🔍 Quick Reference Summary:**

**Emergency Contacts:**
- Device Issues: Check Particle Community Forum
- Server Issues: Check webhook service logs
- Hardware Issues: Consult wiring diagrams
- Network Issues: Test with Particle CLI

**Most Common Issues:**
1. High power consumption → Check for Serial.begin() in code
2. No cellular connection → Check signal strength and SIM status
3. Switch not triggering → Verify NC wiring configuration
4. No notifications → Check webhook authentication
5. Device not sleeping → Remove debug code and configure pins

**Recovery Procedures:**
1. Safe mode: SETUP+RESET → breathing magenta
2. DFU mode: SETUP+RESET → yellow blinking
3. Factory reset: particle device doctor
4. OTA recovery: particle flash via CLI