index.md

HTML wrapper for HTML-only fetchers; content is shown verbatim.

Raw file

Links (extracted)

Clickable list for reviewers; avoids copy/paste line-wrap issues.
# GGQ stack quickstart (mtl-01)

Purpose: give a zero-context agent enough verified detail to understand where GGQ runs, how it is routed, how media is stored (R2), how the DB mirror works, and how to verify changes safely.

Black/white:
- This document is operational (where things live + how to verify). It does not claim the GGQ content is correct.
- Do not paste secrets (tokens/keys/DSNs) into docs or chats. Use **paths only**.

---

## 1) Public surfaces (no login)

Canonical public entrypoints:
https://ww2.guidesgq.com/ggq/
https://infrafabric.io/ggq/

Microcache entrypoint (short TTL; useful for perf testing):
https://infrafabric.io/ggq-fast/

Legacy asset namespace (must be `200`, not a redirect):
https://infrafabric.io/system/

---

## 2) Where the stack runs (verified)

### Edge / routing (pct 210)

- Caddy config:
  - `pct 210:/etc/caddy/Caddyfile`
- Routes (verified in Caddyfile):
  - `/ggq*` → `10.10.10.250:3100`
  - `/ggq-fast*` → `10.10.10.250:3101` (adds `X-Forwarded-Prefix: /ggq-fast`)
  - `/system*` → `10.10.10.250:3100` (served directly; avoids per-image redirect RTT)
- Hostname:
  - `https://ww2.guidesgq.com/` → `10.10.10.250:3100` with `/` redirecting to `/ggq/`

Verify (from host):
```bash
pct exec 210 -- caddy validate --config /etc/caddy/Caddyfile
curl -fsSI https://infrafabric.io/ggq/ | head -n 1
curl -fsSI https://infrafabric.io/system/ | head -n 1
```

### App stack (pct 250)

Root:
- `pct 250:/srv/ggq/`

Docker Compose services (verified):
- `web` (Rails/Refinery, Puma) → host port `3100`
- `microcache` (nginx) → host port `3101`
- `db` (Postgres 17)
- `memcached`

Verify:
```bash
pct exec 250 -- sh -c "docker ps --format 'table {{.Names}}\\t{{.Image}}\\t{{.Ports}}'"
```

Compose file:
- `pct 250:/srv/ggq/docker-compose.yml`

Important runtime env (verified in compose):
- `RAILS_RELATIVE_URL_ROOT=/ggq` (app expects to be mounted under `/ggq`)
- `MEMCACHIER_SERVERS=memcached:11211`
- `PUMA_WORKERS=0` + low thread counts (OOM/r15 avoidance)

Trap:
- `docker compose restart web` does not reliably reload `env_file` changes; use a recreate:
```bash
pct exec 250 -- bash -lc "docker compose -f /srv/ggq/docker-compose.yml up -d --no-deps --force-recreate web"
```

---

## 3) Cloudflare + caching behavior (ww2.guidesgq.com)

Black/white:
- `ww2.guidesgq.com` is behind Cloudflare (verify: `server: cloudflare`).
- HTML responses are typically `cf-cache-status: DYNAMIC` (Rails sets private/no-cache).
- `/system/images/*` responses are typically cacheable and should become `cf-cache-status: HIT`.

Verify:
```bash
curl -fsSI https://ww2.guidesgq.com/ggq/ | rg -n 'HTTP/|server:|cf-cache-status:'
img=$(curl -fsS https://ww2.guidesgq.com/ggq/ | grep -oE '\"/system/images/[^\"]+\"' | head -n 1 | tr -d '\"')
curl -fsSI "https://ww2.guidesgq.com${img}" | rg -n 'HTTP/|content-type:|cache-control:|age:|cf-cache-status:'
```

Stop rule:
- Do not change Cloudflare zone settings (DNS/WAF/rules) unless a task explicitly scopes it.

---

## 4) Media storage (Dragonfly → S3-compatible, R2 + mirror)

### A) App-side: Dragonfly datastore override (pct 250)

Initializer (verified):
- `pct 250:/srv/ggq/app/config/initializers/ggq_dragonfly_s3_endpoint.rb`

Behavior (verified by reading code):
- If env vars are present (`S3_BUCKET`, `S3_KEY`, `S3_SECRET`; optionally `S3_ENDPOINT`, `S3_REGION`, `S3_PATH_STYLE`, `S3_DISABLE_ACL`), Dragonfly `refinery_images` + `refinery_resources` datastores are set to `Dragonfly::S3DataStore`.
- Read path ensures a `mime_type` is present based on file extension (helps with correct `Content-Type` on `/system/images/*`).

Where the env lives (do not paste values):
- `pct 250:/srv/ggq/s3.env`

### B) Host-side: legacy S3 bucket → Cloudflare R2 mirror (mtl-01)

Scripts (verified):
- `/usr/local/sbin/ggq-s3-to-r2-sync`
- `/usr/local/sbin/ggq-r2-rclone-setup`

Timers/services (verified):
- `/etc/systemd/system/ggq-s3-to-r2-sync.service`
- `/etc/systemd/system/ggq-s3-to-r2-sync.timer`

Secret paths (paths only):
- `/root/.secrets/rclone.conf` (must contain `ggq_s3:` and optionally `ggq_r2:`)
- `/root/.secrets/ggq_r2.env` (R2 S3-compatible creds for rclone remote; optional)
- `/root/.secrets/cf_guidesgq_token.env` (Cloudflare API token fallback path; optional)

Verify:
```bash
systemctl status ggq-s3-to-r2-sync.timer --no-pager
systemctl status ggq-s3-to-r2-sync.service --no-pager
```

---

## 5) DB mirror + backups (Heroku Postgres → local)

Black/white:
- Remote DB access is credentialed; credentials live in `/root/.secrets/ggq_db.env` (paths only).
- Sync is **event-driven**: LISTEN/NOTIFY on remote `sync.change_queue` triggers a local refresh.

Services/scripts (verified):
- `/etc/systemd/system/ggq-db-sync-listen.service` → `/usr/local/sbin/ggq-db-sync-listen` (python; LISTEN/NOTIFY + debounce)
- `/usr/local/sbin/ggq-db-sync-from-heroku` (pg_dump custom → restore into `pct 250` `ggq-db-1`)
- `/etc/systemd/system/ggq-db-backup.{service,timer}` → `/usr/local/sbin/ggq-db-backup`

Verify (host):
```bash
systemctl status ggq-db-sync-listen.service ggq-db-backup.timer --no-pager
```

Verify (pct 250 local DB exists):
```bash
pct exec 250 -- docker exec -u postgres ggq-db-1 psql -d ggq -Atc "select 1"
```

Trap:
- `ggq-db-sync-from-heroku` stops the `web` container during restore to avoid writes and active connections.

---

## 6) Minimal smoke test checklist (safe, no login)

```bash
curl -fsSI https://infrafabric.io/ggq/ | head -n 1
curl -fsSI https://infrafabric.io/ggq/_demo/v2 | head -n 1
curl -fsSI https://infrafabric.io/ggq/geo/index | head -n 1
curl -fsSI https://infrafabric.io/ggq-fast/ | rg -n "HTTP/|x-ggq-cache" || true

img=$(curl -fsS https://infrafabric.io/ggq/ | grep -oE '\"/system/images/[^\"]+\"' | head -n 1 | tr -d '\"')
curl -fsSI "https://infrafabric.io${img}" | rg -n "HTTP/|content-type:|cache-control:"
```

---

## 7) “If something is broken” starting points

Edge (pct 210):
```bash
pct exec 210 -- systemctl status caddy --no-pager
```

App (pct 250):
```bash
pct exec 250 -- docker ps
pct exec 250 -- bash -lc "cd /srv/ggq && docker compose logs --tail=200 web microcache"
```

DB sync (host):
```bash
journalctl -u ggq-db-sync-listen.service -n 200 --no-pager
```

---

## 8) Coordination notes (blackboard)

- Work should be tracked via if.blackboard events (append-only), not by editing derived tables.
- CLI name: `if-cli` (not `ift`).