About edgar-explorer

edgar-explorer is an interactive analytics surface over the standard academic EDGAR-CORPUS dataset of SEC 10-K filings, scored with the canonical Loughran-McDonald financial sentiment dictionary.

Universe

Current S&P 500 constituents with continuous 10-K history (companies lacking a 10-K across the window are dropped: post-IPO gaps, spin-offs without a merged predecessor CIK, fiscal-year shifts). We ingest and analyse the last ~10 fiscal years, FY2015 through FY2025: FY2015-FY2020 from EDGAR-CORPUS and FY2021-FY2025 straight from SEC EDGAR (see “Recent filings” below). Recent years may include a few names outside the continuous-history S&P 500 set (e.g. semiconductor watchlist).

Data sources

EDGAR-CORPUS

Loukas, L., Fergadiotis, M., Androutsopoulos, I., & Malakasiotis, P. (2021). EDGAR-CORPUS: Billions of Tokens Make The World Go Round. In Proceedings of the 3rd Workshop on Economics and Natural Language Processing (ECONLP) at EMNLP 2021.

Distributed on Hugging Face as eloukas/edgar-corpus. Approximately 220,000 10-K filings between 1993 and 2020, with sections (Item 1, Item 1A Risk Factors, Item 7 MD&A, etc.) pre-extracted into individual fields. We ingest the FY2015-FY2020 window filtered to the S&P 500 universe via the open-source EDGAR-CRAWLER toolkit's data.

Recent filings (FY2021+)

EDGAR-CORPUS ends at fiscal 2020, which omits topics that only became material afterward — most notably the semiconductor export controls the U.S. tightened from October 2022. To cover them, we pull recent 10-Ks directly from SEC EDGAR using the open-source edgartools library and extract the full narrative/qualitative section set (Items 1, 1A, 1B, 2-7A, 9-15). Item 8 (financial statements) is deliberately excluded — it is large and almost entirely numeric tables, so it adds little for text analysis. Items 1, 1A and 7 parse most reliably; some others (e.g. items incorporated by reference) may occasionally be empty. These rows share the same schema, CIK normalization, fiscal-year keying, and tokenization as the EDGAR-CORPUS rows, so sentiment and search treat both identically.

Loughran-McDonald Master Dictionary

Loughran, T., & McDonald, B. The Loughran-McDonald Master Dictionary. Software Repository for Accounting and Finance, University of Notre Dame. sraf.nd.edu.

The Loughran-McDonald dictionary is free for academic use only. For commercial use, contact the maintainers at loughranmcdonald@gmail.com.

The dictionary classifies words into seven categories: negative, positive, uncertainty, litigious, strong-modal, weak-modal, constraining. For each (filing, section), we count category hits and compute proportion = hits / word_count, stored in the sentiment_facts table at ingest time.

S&P 500 universe

Current constituents from Wikipedia's S&P 500 page, cross-referenced against SEC's authoritative company_tickers.json for ticker→CIK resolution. The harvester also maintains a list of CIK reorganizations (Dow/DuPont split, Disney holding-co reorg, Linde/Praxair merger, BLK 2024 re-registration, etc.) so historical filings under predecessor CIKs are correctly merged in.

Methodology notes

• Tokenization for sentiment scoring uses a simple alphabetic-token regex ([A-Za-z][A-Za-z'-]*) and uppercases tokens to match the LM dictionary convention.
• Sentiment proportions are computed against each section's own word count, not the full filing.
• Section-level sentiment facts and YoY diffs are pre-computed at ingest time. The /sentiment dashboards aggregate over sentiment_facts and never tokenize text.
• Topic sentiment (the /topic page) is the exception: it scores tone at query time. For a search term it takes a ±30-word window around each occurrence, counts LM categories within those windows, and reports net tone = (positive − negative) / (positive + negative) in [−1, 1]. Negation is handled with a 3-word look-back, so a negated positive counts as negative and a negated negative ("no material risk") is suppressed. This answers "with what sentiment does a company discuss X" — which the section-wide scores cannot. Runtime cost is bounded because scoring only runs over the FTS-matched result set.
• Item 1A YoY similarity uses term-frequency cosine over the full section text (not TF-IDF). For v1 this gives a useful relative signal; v2 may switch to TF-IDF or sentence-embedding cosine for more nuanced "is this section really different?" detection.

Validation

The precomputed sentiment facts the dashboards serve are checked for correctness by an independent re-implementation. pipeline/validate_sentiment.py re-tokenizes each sampled Item 1A section straight from the raw text and the LM dictionary, recomputes the negative-word proportion, and asserts it matches the value stored in sentiment_facts to within floating-point tolerance. It also prints the negative proportion for marquee filings (e.g. AAPL FY2020) so the numbers can be sanity-checked against published Loughran-McDonald reproductions. The check runs in CI whenever a database is available.

Related

For the spoken counterpart to these filings, see Subtext — a companion app that runs the same kind of analysis over S&P 500 earnings-call transcripts, speaker by speaker. Company and search pages here link across to the matching view on Subtext.

What's not here yet

• Item 8 (financial statements) is excluded for all years — it is large and almost entirely numeric, with little text-analysis value.
• 10-Q and 8-K filings (10-K only).
• Embedding/transformer sentiment — scoring is lexicon-based (Loughran-McDonald).
• Multi-user accounts; saved queries are stored client-side.