Hamza/maps-saas
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat: sync source-backed OSM facility entrances

Browse Source
This commit is contained in:
Hamza-Ayed
2026-09-29 15:25:38 +03:00
parent 58310330b0
commit f4e24b0725
17 changed files with 772 additions and 391 deletions
Download Patch File Download Diff File Expand all files Collapse all files
crontab/README.md
+7 -7
View File
@@ -9,9 +9,9 @@ This directory contains the central crontab configuration, installer, and docume
| # | Task Name | Schedule | Script / Command | Log File | Description |
|---|-----------|----------|-------------------|----------|-------------|
| 1 | **Bi-Monthly Geospatial Sync** | Every 1st & 15th at 03:00 | `scripts/master_cron_sync.sh` | `logs/cron_places_sync.log` | Syncs OSM & Overture places for JO, IQ, EG, SY. Generates and classifies gates & entrances for complexes. Refreshes `unified_search_index` concurrently and flushes Redis cache. |
| 2 | **Road Network & Routing Sync** | Every 1st, 10th, 20th at 01:00 | `scripts/update-data.sh` | `logs/cron_osm_update.log` | Downloads fresh OSM PBF for Jordan & Syria, updates PostGIS road tables, and rebuilds GraphHopper routing index. |
| 3 | **Daily Database Backup** | Daily at 02:00 | `scripts/backup_db.sh` | `logs/cron_backup.log` | Backs up custom tables (`places_*`, `place_gates`, `users`, `api_keys`, `billing_*`, `telemetry_*`) with 7-day retention. |
| 1 | **Map, Place & OSM Gate Sync** | Every 10 days at 03:00 | `infrastructure/scripts/update-data.sh` | `infrastructure/logs/update-data.log` | Updates the configured regional OSM PBF, rebuilds routing data, refreshes regional place names, imports only explicitly tagged OSM entrance/gate nodes, then clears search cache. |
| 2 | **Supplemental Place Enrichment** | Every 1st & 15th at 07:00 | `scripts/master_cron_sync.sh` | `logs/cron_places_sync.log` | Enriches Iraq and Egypt place tables. Gate coordinates are no longer generated by this job. |
| 3 | **Daily Database Backup** | Daily at 02:00 | `scripts/backup_db.sh` | `logs/cron_backup.log` | Backs up custom tables and the unverified legacy-gates archive with 7-day retention. |
| 4 | **Weekly DB Vacuum & Analyze** | Every Sunday at 04:00 | `docker compose exec db vacuumdb` | `logs/cron_vacuum.log` | Reclaims space and updates PostgreSQL query planner statistics for fast spatial and trigram GIN queries. |
| 5 | **Weekly Maintenance & Log Rotation** | Every Sunday at 05:00 | `scripts/cleanup_logs.sh` | `logs/cron_cleanup.log` | Truncates logs exceeding 50MB, deletes logs older than 14 days, and prunes dangling Docker images. |
@@ -42,8 +42,8 @@ The installer will automatically:
# View active crontab entries
crontab -l
# Watch live output of places & gates sync
tail -f /home/hamzadoctor/app/logs/cron_places_sync.log
# Watch live output of map, place, and gate sync
tail -f /home/hamzadoctor/app/infrastructure/logs/update-data.log
# Check recent database backups
ls -lh /home/hamzadoctor/app/backups/db/
@@ -59,8 +59,8 @@ tail -n 100 /home/hamzadoctor/app/logs/cron_osm_update.log
You can manually trigger any of these jobs at any time without waiting for the cron schedule:
```bash
# Run Bi-Monthly Places & Gates Sync
/bin/bash /home/hamzadoctor/app/scripts/master_cron_sync.sh
# Run the 10-day map, places, and source-backed gate sync
/bin/bash /home/hamzadoctor/app/infrastructure/scripts/update-data.sh
# Run Database Backup
/bin/bash /home/hamzadoctor/app/scripts/backup_db.sh