fix(pb-init): use python3 for JSON parsing in ensure_field; add heartbeat_at fields
All checks were successful
CI / Scraper / Test (pull_request) Successful in 14s
CI / UI / Build (pull_request) Successful in 17s
CI / UI / Docker Push (pull_request) Has been skipped
CI / Scraper / Lint (pull_request) Successful in 21s
CI / Scraper / Docker Push (pull_request) Has been skipped
iOS CI / Build (pull_request) Successful in 8m22s
iOS CI / Test (pull_request) Successful in 12m35s

The sed-based collection id and fields extraction was greedy and broke on
collections with multiple fields (grabbed the last field id instead of the
top-level collection id → PATCH to wrong URL → 404).

Rewrite ensure_field to use python3 for reliable JSON parsing. Also adds the
missing heartbeat_at (date) field to scraping_tasks and audio_jobs which was
never applied on the initial deploy because the bug prevented the PATCH.
This commit is contained in:
Admin
2026-03-15 21:53:54 +05:00
parent 3918bc8dc3
commit 22b6ee824e

View File

@@ -98,22 +98,34 @@ ensure_field() {
-H "Authorization: Bearer $TOKEN" \ -H "Authorization: Bearer $TOKEN" \
"$PB_URL/api/collections/$COLL" 2>/dev/null) "$PB_URL/api/collections/$COLL" 2>/dev/null)
# Check if the field already exists (look for "name":"<FIELD_NAME>" in the fields array) # Use python3 to reliably parse the JSON schema.
if echo "$SCHEMA" | grep -q "\"name\":\"$FIELD_NAME\""; then PARSED=$(echo "$SCHEMA" | python3 -c "
import sys, json
try:
d = json.load(sys.stdin)
fields = d.get('fields', [])
exists = any(f.get('name') == '$FIELD_NAME' for f in fields)
print('exists=' + str(exists))
print('id=' + d.get('id', ''))
if not exists:
fields.append({'name': '$FIELD_NAME', 'type': '$FIELD_TYPE'})
print('fields=' + json.dumps(fields))
except Exception as e:
print('error=' + str(e))
" 2>/dev/null)
if echo "$PARSED" | grep -q "^exists=True"; then
log "field $COLL.$FIELD_NAME already exists — skipping" log "field $COLL.$FIELD_NAME already exists — skipping"
return return
fi fi
COLLECTION_ID=$(echo "$SCHEMA" | sed 's/.*"id":"\([^"]*\)".*/\1/') COLLECTION_ID=$(echo "$PARSED" | grep "^id=" | sed 's/^id=//')
if [ -z "$COLLECTION_ID" ] || [ "$COLLECTION_ID" = "$SCHEMA" ]; then if [ -z "$COLLECTION_ID" ]; then
log "WARNING: could not get id for collection $COLL — skipping ensure_field" log "WARNING: could not get id for collection $COLL — skipping ensure_field"
return return
fi fi
# Extract current fields array and append the new field before the closing bracket. NEW_FIELDS=$(echo "$PARSED" | grep "^fields=" | sed 's/^fields=//')
CURRENT_FIELDS=$(echo "$SCHEMA" | sed 's/.*"fields":\(\[.*\]\).*/\1/')
TRIMMED=$(echo "$CURRENT_FIELDS" | sed 's/]$//')
NEW_FIELDS="${TRIMMED},{\"name\":\"${FIELD_NAME}\",\"type\":\"${FIELD_TYPE}\"}]"
PATCH_BODY="{\"fields\":${NEW_FIELDS}}" PATCH_BODY="{\"fields\":${NEW_FIELDS}}"
STATUS=$(curl -s -o /dev/null -w "%{http_code}" \ STATUS=$(curl -s -o /dev/null -w "%{http_code}" \