fix(pb-init): use python3 for JSON parsing in ensure_field; add heartbeat_at fields
All checks were successful
CI / Scraper / Test (pull_request) Successful in 14s
CI / UI / Build (pull_request) Successful in 17s
CI / UI / Docker Push (pull_request) Has been skipped
CI / Scraper / Lint (pull_request) Successful in 21s
CI / Scraper / Docker Push (pull_request) Has been skipped
iOS CI / Build (pull_request) Successful in 8m22s
iOS CI / Test (pull_request) Successful in 12m35s

The sed-based collection id and fields extraction was greedy and broke on
collections with multiple fields (grabbed the last field id instead of the
top-level collection id → PATCH to wrong URL → 404).

Rewrite ensure_field to use python3 for reliable JSON parsing. Also adds the
missing heartbeat_at (date) field to scraping_tasks and audio_jobs which was
never applied on the initial deploy because the bug prevented the PATCH.
This commit is contained in:
Admin
2026-03-15 21:53:54 +05:00
parent 3918bc8dc3
commit 22b6ee824e

View File

@@ -98,22 +98,34 @@ ensure_field() {
-H "Authorization: Bearer $TOKEN" \
"$PB_URL/api/collections/$COLL" 2>/dev/null)
# Check if the field already exists (look for "name":"<FIELD_NAME>" in the fields array)
if echo "$SCHEMA" | grep -q "\"name\":\"$FIELD_NAME\""; then
# Use python3 to reliably parse the JSON schema.
PARSED=$(echo "$SCHEMA" | python3 -c "
import sys, json
try:
d = json.load(sys.stdin)
fields = d.get('fields', [])
exists = any(f.get('name') == '$FIELD_NAME' for f in fields)
print('exists=' + str(exists))
print('id=' + d.get('id', ''))
if not exists:
fields.append({'name': '$FIELD_NAME', 'type': '$FIELD_TYPE'})
print('fields=' + json.dumps(fields))
except Exception as e:
print('error=' + str(e))
" 2>/dev/null)
if echo "$PARSED" | grep -q "^exists=True"; then
log "field $COLL.$FIELD_NAME already exists — skipping"
return
fi
COLLECTION_ID=$(echo "$SCHEMA" | sed 's/.*"id":"\([^"]*\)".*/\1/')
if [ -z "$COLLECTION_ID" ] || [ "$COLLECTION_ID" = "$SCHEMA" ]; then
COLLECTION_ID=$(echo "$PARSED" | grep "^id=" | sed 's/^id=//')
if [ -z "$COLLECTION_ID" ]; then
log "WARNING: could not get id for collection $COLL — skipping ensure_field"
return
fi
# Extract current fields array and append the new field before the closing bracket.
CURRENT_FIELDS=$(echo "$SCHEMA" | sed 's/.*"fields":\(\[.*\]\).*/\1/')
TRIMMED=$(echo "$CURRENT_FIELDS" | sed 's/]$//')
NEW_FIELDS="${TRIMMED},{\"name\":\"${FIELD_NAME}\",\"type\":\"${FIELD_TYPE}\"}]"
NEW_FIELDS=$(echo "$PARSED" | grep "^fields=" | sed 's/^fields=//')
PATCH_BODY="{\"fields\":${NEW_FIELDS}}"
STATUS=$(curl -s -o /dev/null -w "%{http_code}" \