refactor(scripts): rework eco static-data refresh - #268
Merged
Merged
Conversation
Pure renames; update mypy/pylint hook and cert-doc references.
Handles Ecovacs error 1013 by verifying the device via RSA-encrypted email code (ECO_VERIFY_CODE, ECO_DEVICE_ID/FILE). Rewrites combination: DE > US > rest country priority, recursive key normalization, table-driven COMBINE_SPECS, stable diffs. Adds dependency checks, fixes EXIT-trap bug.
Parallel update_check fetches (ECO_PARALLEL), per-key fragment/status files, AWS signing params stripped from URLs, sorted output for deterministic diffs.
Old output was invalid JSON and crashed load_json_object_files. Parallel downloads, valid sorted {main_folder: deployment_key} JSON copied to bumper/web/static_api/, empty runs leave the committed file unchanged.
Covers the update scripts, usage incl. device verification, combination rules and env vars; linked from dev-guide, added to mkdocs nav.
Regenerated with update-eco-data.sh (country-priority combination, normalized key order).
Regenerated with the codepush scripts (sorted keys, valid JSON mapping).
Updated via update-web-assets.sh.
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #268 +/- ##
=======================================
Coverage 77.34% 77.34%
=======================================
Files 84 84
Lines 4683 4683
Branches 546 546
=======================================
Hits 3622 3622
Misses 876 876
Partials 185 185 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
15 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Reworks the three Ecovacs static-data refresh scripts (renamed, see below),
adds a maintenance guide for the whole update flow and refreshes the bundled
static data under
bumper/web/static_api/with the new scripts.Changed
update-eco-data.sh- device verification login flow1013for unverified device ids: the script nowfetches the RSA public key, encrypts the account email, requests a
verification code (
sendEmailVerifyCode) and completesverifyDevice.Code is prompted or provided via
ECO_VERIFY_CODE..eco-device-id(overridable withECO_DEVICE_ID/ECO_DEVICE_ID_FILE) — verification is only needed once.requestIdnow md5 of current timestamp, matching the reference clientimplementation; hints added for the
0002rate-limit error.Combination step
DE > US > remaining; countries whose request failed areskipped instead of poisoning the result.
jq walk), table-drivenCOMBINE_SPECS,committed files appended as fallback → deterministic output, stable diffs.
Codepush scripts parallelized
update-eco-codepush.sh:update_checkfetches viaxargs -P(
ECO_PARALLEL, default 4), AWS signing params stripped from download URLs,output sorted by key.
update-eco-codepush-mapping.sh: parallel bundle downloads, valid sorted{main_folder: deployment_key}JSON copied tobumper/web/static_api/updateCheck_mapping.json; empty runs leave thecommitted file unchanged.
Robustness
Added
docs/development/static-data-update.mdcode), script order, usage incl. the verification flow, combination rules,
env-variables table. Linked from
dev-guide.md, added to the mkdocs nav.Renamed
scripts/clean-domains.pyscripts/cert-san-domains.pyscripts/eco-check-bot-version.shscripts/check-eco-firmware.shscripts/create-unpinned-app-old.shscripts/create-unpinned-app-legacy.shscripts/run-in-env.shscripts/run-in-dev-env.shscripts/update-statics.shscripts/update-web-assets.shscripts/eco-json-downloader.shscripts/update-eco-data.shscripts/eco-update-check.shscripts/update-eco-codepush.shscripts/eco-update-check-mapping.shscripts/update-eco-codepush-mapping.shData refresh
configGroupsResponse.json,configNetAllResponse.json,productIotMap.jsonupdate-eco-data.sh(country-priority combination, normalized key order).updateCheck.json,updateCheck_mapping.jsonbumper/web/static_web/js/htmx.min.jsupdate-web-assets.sh.