fix(pb-init): use python3 for JSON parsing in ensure_field; add heartbeat_at fields
All checks were successful
CI / Scraper / Test (pull_request) Successful in 14s
CI / UI / Build (pull_request) Successful in 17s
CI / UI / Docker Push (pull_request) Has been skipped
CI / Scraper / Lint (pull_request) Successful in 21s
CI / Scraper / Docker Push (pull_request) Has been skipped
iOS CI / Build (pull_request) Successful in 8m22s
iOS CI / Test (pull_request) Successful in 12m35s
All checks were successful
CI / Scraper / Test (pull_request) Successful in 14s
CI / UI / Build (pull_request) Successful in 17s
CI / UI / Docker Push (pull_request) Has been skipped
CI / Scraper / Lint (pull_request) Successful in 21s
CI / Scraper / Docker Push (pull_request) Has been skipped
iOS CI / Build (pull_request) Successful in 8m22s
iOS CI / Test (pull_request) Successful in 12m35s
The sed-based collection id and fields extraction was greedy and broke on collections with multiple fields (grabbed the last field id instead of the top-level collection id → PATCH to wrong URL → 404). Rewrite ensure_field to use python3 for reliable JSON parsing. Also adds the missing heartbeat_at (date) field to scraping_tasks and audio_jobs which was never applied on the initial deploy because the bug prevented the PATCH.
This commit is contained in:
@@ -98,22 +98,34 @@ ensure_field() {
|
||||
-H "Authorization: Bearer $TOKEN" \
|
||||
"$PB_URL/api/collections/$COLL" 2>/dev/null)
|
||||
|
||||
# Check if the field already exists (look for "name":"<FIELD_NAME>" in the fields array)
|
||||
if echo "$SCHEMA" | grep -q "\"name\":\"$FIELD_NAME\""; then
|
||||
# Use python3 to reliably parse the JSON schema.
|
||||
PARSED=$(echo "$SCHEMA" | python3 -c "
|
||||
import sys, json
|
||||
try:
|
||||
d = json.load(sys.stdin)
|
||||
fields = d.get('fields', [])
|
||||
exists = any(f.get('name') == '$FIELD_NAME' for f in fields)
|
||||
print('exists=' + str(exists))
|
||||
print('id=' + d.get('id', ''))
|
||||
if not exists:
|
||||
fields.append({'name': '$FIELD_NAME', 'type': '$FIELD_TYPE'})
|
||||
print('fields=' + json.dumps(fields))
|
||||
except Exception as e:
|
||||
print('error=' + str(e))
|
||||
" 2>/dev/null)
|
||||
|
||||
if echo "$PARSED" | grep -q "^exists=True"; then
|
||||
log "field $COLL.$FIELD_NAME already exists — skipping"
|
||||
return
|
||||
fi
|
||||
|
||||
COLLECTION_ID=$(echo "$SCHEMA" | sed 's/.*"id":"\([^"]*\)".*/\1/')
|
||||
if [ -z "$COLLECTION_ID" ] || [ "$COLLECTION_ID" = "$SCHEMA" ]; then
|
||||
COLLECTION_ID=$(echo "$PARSED" | grep "^id=" | sed 's/^id=//')
|
||||
if [ -z "$COLLECTION_ID" ]; then
|
||||
log "WARNING: could not get id for collection $COLL — skipping ensure_field"
|
||||
return
|
||||
fi
|
||||
|
||||
# Extract current fields array and append the new field before the closing bracket.
|
||||
CURRENT_FIELDS=$(echo "$SCHEMA" | sed 's/.*"fields":\(\[.*\]\).*/\1/')
|
||||
TRIMMED=$(echo "$CURRENT_FIELDS" | sed 's/]$//')
|
||||
NEW_FIELDS="${TRIMMED},{\"name\":\"${FIELD_NAME}\",\"type\":\"${FIELD_TYPE}\"}]"
|
||||
NEW_FIELDS=$(echo "$PARSED" | grep "^fields=" | sed 's/^fields=//')
|
||||
PATCH_BODY="{\"fields\":${NEW_FIELDS}}"
|
||||
|
||||
STATUS=$(curl -s -o /dev/null -w "%{http_code}" \
|
||||
|
||||
Reference in New Issue
Block a user