SEC Financial Statement Data Sets: What They Are and Their Limits
The SEC's quarterly Financial Statement Data Sets (num, sub, tag, pre) explained: what each file holds, how to join them by accession number, and where they fall short for backtests and screens.
What the Financial Statement Data Sets are
The SEC publishes the Financial Statement Data Sets as quarterly zip archives named by year and quarter, such as 2020q1.zip. They hold numeric data from the face financial statements, extracted from XBRL-based corporate filings and presented in a flattened format. The SEC states the data is not altered from the as-filed reports, that it does not guarantee accuracy, and that the sets are not a substitute for the filings themselves. Source: sec.gov/dera/data/financial-statement-data-sets.
The four tables: sub, num, tag, pre
Each archive contains tab-delimited text files. sub holds one row per submission (the filing: company, form type, period, filing date). num holds the numeric facts (tag, value, unit, period end date, number of quarters covered). tag holds the tag definitions, including custom company tags. pre holds the presentation of the statements (which tag appears on which line of which statement). The authoritative column layout is in the SEC's data-sets documentation PDF (sec.gov/files/financial-statement-data-sets.pdf). Every row links back through the adsh column, the filing's accession number.
import duckdb # After unzipping 2020q1.zip next to this scriptcon = duckdb.connect()con.sql(""" SELECT s.name, s.form, n.tag, n.ddate, n.qtrs, n.uom, n.value FROM read_csv('sub.txt', delim='\t', header=true) s JOIN read_csv('num.txt', delim='\t', header=true) n USING (adsh) WHERE s.cik = 320193 AND n.tag = 'Revenues'""").show()Where raw data sets hurt
Four practical problems show up quickly. First, no standardization: companies pick different XBRL tags for the same economic concept, so Revenues, SalesRevenueNet and RevenueFromContractWithCustomer all mean revenue and you must map them yourself. Second, every filing is its own snapshot, so a value restated in a later filing appears again under a new accession number and you must decide which one to trust. Third, the quarterly archives are keyed to filings, not to a timeline of what was knowable on a date, so building a point-in-time view means reconstructing it from filing dates yourself. Fourth, every query is a join on adsh across several files, repeated for every quarterly archive you download.
How Valuein relates to them
Valuein starts from the same source, SEC EDGAR filings, and does the mapping and point-in-time work once. Facts are standardized to common concepts, each fact keeps the accession of the filing it came from and the SEC acceptance timestamp (accepted_at), and restated values are kept alongside the originals so you can ask what was known on a given date. See /datasets for the tables, /methodology for how facts are mapped and checked, and /developers/pit for the point-in-time query pattern.
Try it without a token
The free sample needs no token and no card. Install the SDK and query the sample tables directly.
pip install valuein-sdk from valuein_sdk import ValueinClientwith ValueinClient() as client: print(client.run_query("SELECT * FROM fact LIMIT 5"))Up next
SEC Accession Numbers: How to Read and Look Up a Filing
What an SEC accession number is, how its 18 digits are built, how to find the filing on EDGAR from one, and how Valuein ties every fact to the accession of its source filing.