feat: add place gate population scripts, map style assets, automated cron sync tasks, and expand dashboard and landing applications

2026-09-19 21:22:05 +03:00
parent 82b1045f9c
commit ce31d79ba6
76 changed files with 118969 additions and 115428 deletions
@@ -0,0 +1,70 @@
+# SIRO Maps — Crontab & Scheduled Jobs Directory
+# دليل وجدولة المهام الدورية والتشغيل الآلي لمنصة سيرو
+
+This directory contains the central crontab configuration, installer, and documentation for all scheduled background jobs running on the SIRO Maps server.
+
+---
+
+## 📋 Overview of Scheduled Jobs
+
+| # | Task Name | Schedule | Script / Command | Log File | Description |
+|---|-----------|----------|-------------------|----------|-------------|
+| 1 | **Bi-Monthly Geospatial Sync** | Every 1st & 15th at 03:00 | `scripts/master_cron_sync.sh` | `logs/cron_places_sync.log` | Syncs OSM & Overture places for JO, IQ, EG, SY. Generates and classifies gates & entrances for complexes. Refreshes `unified_search_index` concurrently and flushes Redis cache. |
+| 2 | **Road Network & Routing Sync** | Every 1st, 10th, 20th at 01:00 | `scripts/update-data.sh` | `logs/cron_osm_update.log` | Downloads fresh OSM PBF for Jordan & Syria, updates PostGIS road tables, and rebuilds GraphHopper routing index. |
+| 3 | **Daily Database Backup** | Daily at 02:00 | `scripts/backup_db.sh` | `logs/cron_backup.log` | Backs up custom tables (`places_*`, `place_gates`, `users`, `api_keys`, `billing_*`, `telemetry_*`) with 7-day retention. |
+| 4 | **Weekly DB Vacuum & Analyze** | Every Sunday at 04:00 | `docker compose exec db vacuumdb` | `logs/cron_vacuum.log` | Reclaims space and updates PostgreSQL query planner statistics for fast spatial and trigram GIN queries. |
+| 5 | **Weekly Maintenance & Log Rotation** | Every Sunday at 05:00 | `scripts/cleanup_logs.sh` | `logs/cron_cleanup.log` | Truncates logs exceeding 50MB, deletes logs older than 14 days, and prunes dangling Docker images. |
+
+---
+
+## 🚀 Quick Setup on Any Server / Migration Guide
+
+When setting up a new server or migrating the platform:
+
+```bash
+# 1. Navigate to the app directory
+cd /home/hamzadoctor/app
+
+# 2. Run the Crontab Installer
+bash crontab/install_cron.sh
+```
+
+The installer will automatically:
+1. Create required directories (`logs/` and `backups/db/`).
+2. Make all scripts in `scripts/` executable (`chmod +x`).
+3. Load all jobs from `crontab/crontab.txt` into the server's crontab.
+
+---
+
+## 🔍 How to Monitor & Inspect Logs
+
+```bash
+# View active crontab entries
+crontab -l
+
+# Watch live output of places & gates sync
+tail -f /home/hamzadoctor/app/logs/cron_places_sync.log
+
+# Check recent database backups
+ls -lh /home/hamzadoctor/app/backups/db/
+
+# Check road network update log
+tail -n 100 /home/hamzadoctor/app/logs/cron_osm_update.log
+```
+
+---
+
+## ⚡ Manual Execution (Testing)
+
+You can manually trigger any of these jobs at any time without waiting for the cron schedule:
+
+```bash
+# Run Bi-Monthly Places & Gates Sync
+/bin/bash /home/hamzadoctor/app/scripts/master_cron_sync.sh
+
+# Run Database Backup
+/bin/bash /home/hamzadoctor/app/scripts/backup_db.sh
+
+# Run Log & Docker Maintenance
+/bin/bash /home/hamzadoctor/app/scripts/cleanup_logs.sh
+```