Skip to content
daggerokPublic

About

SPDR ETF holdings-to-watchlist: static ./api/spdr feed generated by a Bun updater from SSGA / State Street public feeds

Topics

Resources

Stars

0 stars

Watchers

0 watching

Forks

Latest commit

 

History

97 Commits

Folders and files

Repository files navigation

SPDR

One of the app's features lets you select SPDR ETFs in the Watchlist and aggregate their holdings to see how often each ticker appears across the selected funds. Repeated holdings make overlapping exposure visible: the more selected funds include a ticker, the greater its potential influence on the portfolio; gains in that holding may help, while declines may hurt, and actual impact also depends on each fund's position size. Another feature makes it faster and easier to find funds with stronger growth over different periods, higher dividend yields or distributions, greater Total Return (price performance plus dividends), and other key performance metrics. A client-side tool that reads the generated ./api/spdr static feed (SSGA daily holdings XLSX, NAV history XLSX, daily premium/discount history XLSX, the bulk product-data XLSX for ISIN/CUSIP/official SEC and dividend yield, distributions) into a searchable ETF/asset-class catalog with per-fund tabs, watchlist aggregation, ticker copy and CSV/TXT export — the same look, feel, columns and business logic as the sibling applications.

Using Bun

bunx degit daggerok/SPDR#main ./12345 && cd $_
bun install
bun run serve
open http://localhost:1234

bun run serve starts the Parcel dev server (it copies api/ to dist/api first). bun run build writes the production site to dist/ and bun run build-github-pages does the same with the /SPDR/ public URL used by the GitHub Pages workflow.

The published application is available at https://daggerok.github.io/SPDR/.

Column types and filters

Every column of the ETF catalog and of the Watchlist, Holdings, History and Distributions tabs has a type: text (ABC), number (123), percentage (%), money ($), date (D), date and time (DT) or time of day (T). The type is detected from the texts the column shows (80% of the filled cells must agree, otherwise text) and is written in the badge next to the column title: click it to cycle the type, Shift+click to return to auto-detection. Dates are read as 2024-06-15, 6/15/2024, 15.06.2024, Jun 15, 2024 or 15-Jun-2024, date and time as 2024-06-15T09:30:00Z or 2024-06-15 09:30, time as 09:30, 16:00:00 or 9:30 PM

A row of filter inputs sits under the column headers (the Filters button hides it, Clear filters empties it). Filters of different columns are combined with AND, the search box applies on top, and Copy Tickers and the exports use the filtered rows. Filters and type overrides are remembered in the browser. Sticky # (next to Filters, off by default, remembered in the browser) numbers the rows by their rank in the table sorted by the current column before the column filters, so a filtered fund keeps its rank and the numbers keep gaps; the sort, the search and the category and blacklist choices rank again. The catalog starts sorted by Net Assets, largest first, unavailable values sort last in both directions, and every export starts with the # column. The red Clear button forgets everything saved in the browser without asking, except the blacklist and the theme, so the page looks like a first visit (also after a reload)

Inside one filter: a space means AND, a comma means OR, a leading ! means NOT, ? matches an empty or unavailable value and !? a value that is there; a value that is unavailable matches only ? and negated conditions. An unquoted space ends the value, so quote values that contain one (>="2024-06-15 09:30")

Type Examples
Text bank contains, "two words", !bank, =exact, ^starts, ends$, /regex/, tech, health
Number, percentage, money >10, >=10 <50, =22 (matches what rounds to 22), !=22, 10..50, ..50, 10.., >1B and K M B T suffixes, an optional $ or %
Date, date and time >2024-06-01, 2024 (the whole year), 2024-06 (the whole month), 2024-01..2024-06, today, yesterday, -7d.. (the last 7 days), +2w, -3m, -1y
Time >09:30, 09:30..16:00, =12:00 (the whole minute)

The Columns menu next to Filters lists every column of the ETF table from the first to the last, all of them shown by default, with a search box and the All, Clear, Toggle and Reset buttons. Use and Ticker are listed but locked. Hiding a column only removes it from the table: the filters, the sorting, the exports and Copy Tickers still use it. The choice is remembered in the browser (localStorage, never the data) and the menu is shown on the ETF catalog only

The asset classes are one Asset classes multi-select next to the All ETFs pill instead of one tab per class: every class is selected by default (= all ETFs), Only or unchecking narrows the table, and the All ETFs pill is lit only while nothing narrows it (all or none of the classes checked); clicking the pill clears the selection. The choice is remembered in the browser (localStorage, never the data)

Updating the static SPDR data

Run the updater with Bun:

bun scripts/update-data.ts

Run bun scripts/update-data.ts --help to print every control with its default and usage examples.

Defaults live in scripts/update-data.config.json, one string value per control. Precedence, lowest to highest: file defaults < advanced JSON < nonblank workflow inputs < environment variables (locally) or the protected Actions variable (CI). Blank workflow inputs inherit the file value, and advanced can set a control to an empty string on purpose. An explicitly set environment variable wins even when empty, and the legacy SPDR_<NAME> aliases still work. The Update SPDR ETF data workflow runs weekly (Sunday 00:00 UTC) with the file defaults, and manual runs can override them through individual inputs or one advanced JSON object such as {"CONCURRENCY":"1","VERBOSE":"true"}. CLI and workflow share the same resolveControls validation, and the workflow only writes to api/spdr. All supplied filters use AND logic.

Data sources

Block Source
Catalog (all US SPDR ETFs) https://www.ssga.com/us/en/intermediary/etfs/fund-finder (SSGA fund finder JSON)
Holdings per fund https://www.ssga.com/us/en/intermediary/etfs/library-content/products/fund-data/etfs/us/holdings-daily-us-{TICKER}.xlsx (per-fund holdings XLSX)
Daily history, distributions https://www.ssga.com/us/en/intermediary/etfs/library-content/products/fund-data/etfs/us/navhistory-daily-us-{TICKER}.xlsx (NAV history XLSX)
Daily Premium/Discount history ssga.com/library-content/products/fund-data/etfs/us/pdhist-us-en-{TICKER}.xlsx
ISIN, CUSIP, official 30-Day SEC Yield (subsidized/unsubsidized), official Fund Dividend Yield (whole lineup, one fetch per run) ssga.com/library-content/products/fund-data/etfs/us/spdr-product-data-us-en.xlsx
Fallback Previously published api/spdr/index.json

Metrics and caveats

Each fund carries a derived metrics object that powers the catalog columns shared with the sibling sites:

  • ytd / tr1y - official YTD and 1-year returns -> YTD Return, TR 1Y
  • cagr3y / cagr5y / cagr10y - published annualized 3Y/5Y/10Y figures -> CAGR 3Y/5Y/10Y
  • tr3y / tr5y / tr10y - cumulative 3Y/5Y/10Y figures (1 + CAGR)^n - 1 -> TR 3Y/5Y/10Y
  • siAnn - since-inception annualized -> SI Ann.; null for funds with less than one year of history at the performance date
  • dividendYield - official SSGA Fund Dividend Yield, else an indicated yield (latest distribution x payments per year / price; semi-annual = 2), an estimate derived from SSGA data. HECO, XITK and XSW publish an official 0.00%, which is kept as 0 (a real zero, not a missing value)
  • dividendYieldBasis - code of the definition behind dividendYield, null exactly when dividendYield is null (table below)
  • secYield - official 30-day SEC yield when published; unavailable otherwise, never 0
  • returnsBasis - always a non-empty label of how the returns are computed: official SSGA month-end NAV total returns, with tr3y/tr5y/tr10y derived from the official annualized figures; no Yahoo or market-price estimates
  • performanceAsOf - ISO YYYY-MM-DD date of SSGA's month-end performance table the returns are as of (not the NAV date); null when SSGA publishes no performance yet (very young funds)
dividendYieldBasis Meaning for SPDR
official-other SSGA Fund Dividend Yield from the product-data workbook; SSGA does not state the window there, so it is not claimed to be trailing 12 months
indicated updater estimate: latest distribution x inferred payments per year / NAV, used only for funds missing from the workbook

official-trailing-12m, official-distribution-rate and computed-trailing-12m are never produced for SPDR. dividendYieldSource (official or indicated) is kept as is. Funds with a retained index row get the code derived from the yield they carry.

returnsBasis and performanceAsOf are always the last two keys of metrics. Funds launched within the last few months (for example MYCP, MYHF, MYML, QNDX, UCBG) have performanceAsOf: null because SSGA has not published a performance table for them yet.

Expense ratio: terValue / ter are the NET expense ratio (after waivers; the single published figure when there is no waiver), terGrossValue / terGross the gross ratio. Both come from the product-data workbook ("Gross Expense Ratio", "Net Expense Ratio"), with the fund finder expense ratio as the gross fallback. The TER filter uses the net value.

Sources and scope: SPDR has no Yahoo Finance and no SEC EDGAR source. Everything comes from SSGA's own files, so there is no SKIP_YAHOO, EDGAR_FALLBACK or SEC_UA control. Yields are the official 30-day SEC yield and Fund Dividend Yield from SSGA's product-data workbook.

Commodity trusts such as GLD and GLDM hold metal and publish no holdings workbook. That is valid: they get meta.json, NAV, returns and history, with holdings status: "empty" (holdingsStatus in index.json). A fund that published holdings before is never emptied by a transient 404.

Consistency: each fund is fetched and computed completely in memory, then written once (pages, then meta.json, then stale pages are removed; every JSON file is written through a temp file and renamed). If any required file of a fund fails to download, that fund keeps its previous meta.json, pages and index.json row together, so the workflow never publishes a new return next to stale files. A fund without funds/<T>/meta.json has dataFile: null in index.json. A rerun with identical upstream data changes nothing, and generatedAt moves only when content moved. Newly discovered funds are announced as NEW FUNDS: A, B in the run output and in the job summary. Every request times out after 45 s (headers and body) and is retried per MAX_RETRIES; the run stops starting new funds after 25 minutes and still writes the index; it exits non-zero when every selected fund failed.

Returns come from SSGA's own NAV series: PERFORMANCE_* filters use month-end NAV returns and TOTAL_RETURN_* filters use the separate quarter-end series (3Y/5Y/10Y are annualized CAGR). DIVIDEND_YIELD and SEC_YIELD filters use the same metrics values. There are no market-price or Yahoo estimates in this feed, and no ticker exclusions. Funds not selected for a successful update keep their prior published metadata and data files. TICKERS combines with the other filters using AND logic; it does not override them.

Update controls

Every control is in scripts/update-data.config.json; the table shows the shipped defaults.

Control Default Meaning
MAX_FETCHES 0 Batch size: with a positive value the updater continues after the committed cursor in api/spdr/update-state.json (funds passing every filter only, wrapping around; a cursor saved under other filters is ignored and a TICKERS run never writes it); 0 is a full pass, every fund is refreshed in one run
REQUEST_SLEEP 1 Minimum delay in seconds between outgoing request starts, including retries
CONCURRENCY 2 Number of parallel fund update workers; request starts are still spaced by REQUEST_SLEEP
AUM : Net Assets range; each bound may be a USD amount, a K/M/B/T amount, or one of nano, micro, small, mid, large
TER : Net expense ratio range in % (strict min:max; gross when no waiver is published)
DIVIDEND_YIELD : Dividend yield range in % (strict min:max); official Fund Dividend Yield, else the indicated yield from the latest distribution; funds without a value fail a bounded range
SEC_YIELD : Official 30-day SEC yield range in % (strict min:max); funds without a published value fail a bounded range
TICKERS empty (all) Space-, comma- or semicolon-separated ticker allowlist, e.g. SPY SPYG SPYD SDY XLK; a ticker that is not in the SPDR catalog is an error
HOLDINGS_PAGE_SIZE 250 Rows in each generated current-holdings JSON page
HISTORY_PAGE_SIZE 1000 Rows in each generated daily-history JSON page
HISTORY_RANGE max Window for the generated NAV and premium/discount history: max, Ny or Nmo counted back from the newest row (for example 5y, 18mo); SSGA serves the full workbook, so the window trims the generated pages, not the download; older rows already published are kept, a shorter window never deletes history
STORE_RAW_DOWNLOADS false Keep the latest source XLSX files under api/spdr/raw
MAX_RETRIES 2 Retries after the initial request, integer >= 1; network errors, 45 s timeouts and HTTP 408/425/429/5xx are retried with exponential backoff
VERBOSE false Print per-fund retry and fallback notices
USE_SYSTEM_CA auto TLS trust store: auto restarts the updater once with Bun's --use-system-ca when a request fails with an untrusted-certificate error; true always uses the system CA store; false never restarts. Not an individual workflow input: use advanced, the config file or the CLI environment.
PERFORMANCE_YTD / _1Y / _3Y / _5Y / _10Y : Month-end NAV return range in %; 3Y/5Y/10Y are CAGR; the colon is required (5:, :20, 5:20)
TOTAL_RETURN_YTD / _1Y / _3Y / _5Y / _10Y : Quarter-end NAV return range in %, same syntax

Examples

MAX_FETCHES=10 bun scripts/update-data.ts
TICKERS="SPY SPYG SPYD SDY XLK" bun scripts/update-data.ts
AUM="1B:" TER=":0.5" bun scripts/update-data.ts
DIVIDEND_YIELD="3:" SEC_YIELD="2:" bun scripts/update-data.ts
HISTORY_RANGE=5y bun scripts/update-data.ts
PERFORMANCE_1Y="15:" bun scripts/update-data.ts

TypeScript and verification

The browser app is built with Parcel and Tailwind CSS v4: src/index.html carries the markup, src/index.css the styles and src/main.tsx is the TypeScript application, bundled into dist/ by bun run build. There is no tsconfig.json and no typescript dependency; Bun runs the updater TypeScript out of the box and GitHub Pages is deployed by .github/workflows/github-pages.yml.

Verification before every publish:

bun install --frozen-lockfile
bun test
bun build --target=bun scripts/update-data.ts --outfile=/dev/null
git diff --check

bun test also checks that the config file, CONTROL_NAMES, --help, this controls table, the README structure and the workflow inputs stay in sync.

Brands table

Brand Where to get the data
AAM aamlive.com | AAM
abrdn (Aberdeen) aberdeeninvestments.com | aberdeen
Amplify amplifyetfs.com | Amplify
ARK Invest ark-funds.com | ARK
Capital Group capitalgroup.com | Capital-Group
Fidelity fidelity.com | Fidelity
First Trust ftportfolios.com | First-Trust
Franklin Templeton franklintempleton.com | Franklin
Global X globalxetfs.com/explore | Global-X
Goldman Sachs am.gs.com | Goldman-Sachs
Invesco invesco.com | Invesco
iShares ishares.com | iShares
JPMorgan am.jpmorgan.com | JPMorgan
NEOS neosfunds.com | Neos
Northern Trust etfs.ntam.northerntrust.com | Northern-Trust
Pacer ETFs paceretfs.com | Pacer
Parametric eatonvance.com | Parametric
ProShares proshares.com | ProShares
Schwab schwabassetmanagement.com | Schwab
SP Funds sp-funds.com | SP-Funds
SPDR ssga.com | SPDR
Sprott ETFs sprottetfs.com | Sprott
Tema ETFs temaetfs.com | Tema
Themes ETFs themesetfs.com/etfs | Themes
VanEck vaneck.com | VanEck
Vanguard investor.vanguard.com | Vanguard
VictoryShares vcm.com VictoryShares ETFs | VictoryShares
WisdomTree wisdomtree.com | WisdomTree
Xtrackers etf.dws.com | Xtrackers

Sibling applications

Application Data provider Repository
AAM Official AAM catalog/detail HTML + full holdings XLS + SEC N-PORT holdings fallback + Yahoo market history/dividends AAM
abrdn (Aberdeen) Official Aberdeen gateway + SEC N-PORT holdings fallback + Yahoo history/dividends aberdeen
Amplify Amplify ETFs Firestore data feed + SEC EDGAR N-PORT-P holdings fallback + Yahoo Finance history/dividends Amplify
ARK Invest ark-funds.com fund pages + overview/NAV-history/performance JSON + official daily holdings CSV + SEC EDGAR N-PORT-P holdings fallback + Yahoo Finance distributions/history fallback ARK
Capital Group Official Capital Group fund data + SEC N-PORT holdings fallback + Yahoo history fallback Capital-Group
Fidelity SEC EDGAR N-PORT-P + Yahoo Finance Fidelity
First Trust ftportfolios.com official ETF list + fund summary, holdings, distribution and price-history export pages + SEC EDGAR N-PORT-P holdings fallback + Yahoo Finance history fallback First-Trust
Franklin Templeton franklintempleton.com ETF listings + product pages + SEC EDGAR N-PORT-P Franklin
Global X globalxetfs.com Next.js catalog and fund pages + dated full-holdings CSV Global-X
Goldman Sachs am.gs.com fund finder + detail pages + SEC EDGAR N-PORT-P Goldman-Sachs
Invesco invesco.com fund pages and sitemap + official Invesco fund API (monthly returns, NAV, AUM, yields, daily holdings, expense ratio) + SEC EDGAR N-PORT-P holdings fallback + Yahoo Finance history/dividends Invesco
iShares iShares (BlackRock) product workbooks iShares
JPMorgan am.jpmorgan.com fund explorer + product-data JSON JPMorgan
NEOS neosfunds.com lineup table + official fund pages + daily holdings CSV Neos
Northern Trust etfs.ntam.northerntrust.com funds list + per-fund CSV/JSON downloads Northern-Trust
Pacer ETFs paceretfs.com product catalog and fund pages (Cloudflare WAF; r.jina.ai proxy fallback) + SEC EDGAR N-PORT-P (Pacer Funds Trust) + Yahoo Finance history/dividends Pacer
Parametric eatonvance.com ETF catalog and Parametric product pages + SEC EDGAR N-PORT-P holdings + Yahoo Finance history/dividends Parametric
ProShares proshares.com ETF finder + fund pages + official data host ProShares
Schwab schwabassetmanagement.com product pages + CSV exports Schwab
SP Funds sp-funds.com homepage catalog, fund pages and daily holdings CSV + SEC EDGAR N-PORT-P holdings fallback + Yahoo Finance history/dividends SP-Funds
SPDR SSGA / State Street public feeds SPDR
Sprott ETFs sprottetfs.com fund pages + SEC EDGAR N-PORT-P (Sprott Funds Trust) + Yahoo Finance history/dividends Sprott
Tema ETFs Tema official fund pages + dated daily holdings CSV; SEC EDGAR N-PORT-P holdings fallback only + Yahoo Finance price/history/dividend fallback Tema
Themes ETFs themesetfs.com catalog + daily holdings CSV + Yahoo Finance history/dividends + SEC N-PORT-P holdings fallback Themes
VanEck vaneck.com ETF finder + product pages VanEck
Vanguard Vanguard product pages + SEC EDGAR N-PORT-P Vanguard
VictoryShares VCM VictoryShares catalog and product JSON + SEC EDGAR N-PORT-P holdings fallback + Yahoo Finance adjusted-market-price history VictoryShares
WisdomTree WisdomTree product table + SEC EDGAR N-PORT-P + Yahoo Finance WisdomTree
Xtrackers Official DWS catalog/US sitemap + PDP/XLSX + SEC N-PORT-P holdings fallback + Yahoo Finance daily prices/history/dividends Xtrackers

License

MIT - same as all sibling ETF repositories.

SPDR® is a registered trademark of Standard & Poor's Financial Services LLC (S&P), licensed to S&P Dow Jones Indices LLC and sublicensed for certain purposes by State Street Global Advisors; State Street® is a trademark of State Street Corporation. The fund names/tickers referenced here are trademarks of their respective owners. This is an independent, unofficial tool; it is not affiliated with, endorsed by, or sponsored by State Street Global Advisors, State Street Corporation or S&P. All data is reproduced from SSGA's own public downloads for research purposes. All other trademarks, including index names, are the property of their respective owners.

About

SPDR ETF holdings-to-watchlist: static ./api/spdr feed generated by a Bun updater from SSGA / State Street public feeds

Topics

Resources

Stars

0 stars

Watchers

0 watching

Forks

Contributors

Languages