Build Log · Data Pipeline
A serverless pipeline that pulls weight from Withings, macros from Cronometer's unpublished GWT-RPC endpoints, and steps + sleep from Zepp's own cloud API, then writes all of it into a coach's spreadsheet every morning — built with a "prove it locally first" discipline before a single AWS resource existed.
Every credential and every coordinate got proven on a laptop before a single AWS resource existed: a 40-test pure-logic suite with no network calls, then a DRY_RUN=1 mode that logs every intended spreadsheet write without touching the workbook. Debugging a cold Lambda start is worse than debugging a local shell, and every real failure mode here turned out to be a credential or a coordinate — both easier to see on a laptop.
A one-time bootstrap script spins up a throwaway localhost listener, opens the Google consent screen, and exchanges the resulting code for a refresh token — the only artifact the Lambda ever needs afterward.
Error 403: access_denied — "has not completed the Google verification process." The consent screen was set to External but the account authorizing it hadn't been added under Test users yet. Adding it there took effect immediately, no propagation wait.
Withings issues a brand-new refresh token on every use and invalidates the one before it — the opposite of a "set once" secret. Lambda environment variables are immutable at runtime, so the token lives in SSM Parameter Store instead, and the client writes the rotated value back before it ever risks losing the only valid copy.
Personal Cronometer accounts get no API — that tier is enterprise-only. The pipeline instead drives the web app's own internal GWT-RPC endpoints, captured from DevTools: a login flow, a permutation/header pair baked into each Cronometer deploy, and an account ID that lets the client skip the one RPC never fully verified against a live response.
Content-Type: application/json — but the body is GWT wire format (//OK[...]), not JSON. The server mislabels itself; the client has to know to ignore the header and parse GWT anyway.
The account has roughly ten exports a day before Cronometer's quota kicks in, so retries are capped rather than looped — and logins are rate-limited hard enough that the authenticated session gets cached in SSM between runs instead of re-logging in every time.
Weight and nutrition are pulled and written as two isolated stages, each wrapped so one failing never sinks the other — a Cronometer outage shouldn't cost a valid weigh-in, and a missed weigh-in shouldn't block a nutrition update. The handler returns a per-stage status (ok / skipped / error) rather than throwing, and "skipped" is deliberately never conflated with "error": no weigh-in logged, no food logged, and a date outside the coaching window are all normal conditions, not failures.
=average(B:H) for both. A day of the week isn't a cosmetic detail here — it's load-bearing, the same way a missing default root object was on the last build.
Dependencies get resolved with pip install --platform manylinux2014_x86_64 --only-binary=:all: rather than a bare pip install, so the build host's own OS and CPU never leak into the artifact — packaging from Windows or Apple Silicon would otherwise produce a zip Lambda can't import. boto3 is deliberately left out; the Lambda runtime already provides it, and bundling it would roughly double the package for no benefit. Final artifact: about 2.6 MB, comfortably under the console's 50 MB upload ceiling.
Wiring the daily trigger through EventBridge's newer Scheduler service failed immediately, and it looked at first like the same IAM-propagation delay seen earlier when a fresh role needed a few seconds before Lambda would trust it. It wasn't.
aws scheduler create-schedule recognized as a commandFAILA daily EventBridge schedule invokes the function at 06:00 UTC — after the overnight weigh-in, well ahead of the coach's evening deadline. A CloudWatch alarm watches the Errors metric with missing-data treated as good, since the pipeline is silent by design on ordinary skips; an SNS topic forwards a breach straight to email. Log retention got set to 30 days explicitly — the console default is Never expire, which quietly accrues cost for logs nobody will read a year from now.
Weight and macros were covered; steps and sleep from a Zepp/Amazfit band were not. The same reverse-engineered-client pattern that worked for Cronometer looked like the obvious move: Zepp's private Huami Cloud API has a login endpoint, a token exchange, and a data call — no different in shape from what was already running.
429 {"code":12,"message":"too many requests"}. A differential test (an unrelated account logging in cleanly against the same endpoint moments later) confirmed the block was tied to this account specifically, not a generic rate limit — and it survived a completely normal, successful login on the real Zepp app itself. No decay timer, no trust reset. Paused rather than risk a forced password reset by continuing to poke it.
A look at the community projects built around this API (Gadgetbridge's ecosystem, a couple of Python CLIs, one repo that turned out to be Huami's own official partner-API docs, misleadingly named like a community one) showed the provider had overhauled their login in the meantime: the mobile app's flow now requires an encrypted payload. That's plausibly part of why the earlier plaintext attempt got flagged in the first place.
A browser-flavored version of the same original login endpoint — matching the Zepp web app's headers instead of the mobile app's — sidestepped the encryption requirement entirely and logged in clean, with no lockout. The data endpoint turned out to be friendlier than expected too: its summary field is plain base64-encoded JSON, not the opaque binary blob some docs implied. Decoded steps matched the app's own displayed count exactly on the first real request.
Steps and sleep are now a third pulled stage sitting right alongside Withings and Cronometer in the one function.
The schedule moved from a fixed 06:00 UTC to cron(0 9 * * ? *) in the named timezone America/New_York rather than a raw UTC offset — a fixed offset would drift an hour off "9am local" every time the clocks change. 9am gives the morning weigh-in time to actually happen before the run, so it lands in the same pass as the previous night's sleep and the prior day's completed food and steps.
DRY_RUN had been left on from initial setup, and it silently broke Withings' auth a day later. Withings invalidates a refresh token the instant it's used and issues a replacement — but the client only persists that replacement if not dry_run, so the safety flag meant to prevent side effects also quietly discarded the one live credential the next run needed. Two features colliding, neither one wrong on its own.
Last addition: a best-effort summary email after every run — weight, each day's macros, each day's steps and sleep — sent through SES. It's wrapped in its own try/except in the handler; a failed email should never be allowed to mask or fail an otherwise-successful write.
A week into unattended runs, the whole pipeline went dark — not one stage, all of them. SheetsWriter is constructed before the Withings, Cronometer, and Zepp stages ever run, so a dead Google credential took the entire day's run down with it: RefreshError: invalid_grant: Token has been expired or revoked. Nothing had been revoked. Nobody had touched permissions.
Fixed by publishing the app (a one-time click, no verification review required at this account's usage level — just a one-screen "unverified app" warning on the next consent) and re-minting the token afterward. The first re-mint, done before publishing, kept the 7-day expiry anyway — a token's lifetime is set at issue time, not by whatever the app's status happens to be later. Order mattered: publish first, mint second.