# OYOTTA Public Source Library

- **Built**: 2026-09-13T06:35:21.031228+00:00
- **Source Catalog**: `public-catalog.json` (generated `2026-09-12T22:01:11Z`)
- **Total Records**: 80
- **Records with Text**: 80
- **Texts Output**: 80 files in `texts/`

## Contents

- `source-library.json` — Complete structured dataset with full cleaned text, titles, URLs, and cryptographic hashes.
- `source-library.csv` — Metadata index of all source records without full text bodies.
- `texts/*.txt` — Individual cleaned text files keyed by URL slug for direct inspection and NLP use.
- `checksums.sha256` — SHA-256 verification hashes for all generated files.

## Record Schema

| Field | Type | Description |
|-------|------|-------------|
| `source_url` | string | Original catalog source URL requested |
| `retrieval_time_utc` | string | ISO 8601 UTC timestamp of retrieval |
| `http_status` | integer | HTTP response status code (all retained records are 200 OK) |
| `final_url` | string | Resolved URL after following HTTP and client-side meta-refresh redirects |
| `canonical_url` | string | Target specified in `<link rel="canonical">` |
| `title` | string | Extracted page title |
| `cleaned_text` | string | Extracted article/sample text with navigation, scripts, and boilerplate stripped |
| `raw_html_sha256` | string | SHA-256 hash of the raw HTTP response body |
| `cleaned_text_sha256` | string | SHA-256 hash of the extracted cleaned text |
| `source_limitations` | string | Provenance notes (e.g. `meta_refresh_to:...`, `http_redirect_to:...`, `none`) |
| `content_length` | integer | Byte length of raw HTML response |

## Selection & Provenance Rules

- **Allowed Roots**: Public OYOTTA domains only (`publishing.oyotta.org`, `music.oyottaofficial.com`, `oyottaofficial.com`, `features.oyottaofficial.com`, `art.oyottaofficial.com`, `archive.oyottaofficial.com`, `knowledge.oyottaofficial.com`, `stream.oyotta.org`, `oyotta.org`, `theoyotta.com`).
- **Exclusions Enforced**: `/edition/` and `/editions/` (paid manuscript editions), `/cron/`, `/account/`, `/admin/`, `/private/`, and preview queries are strictly excluded.
- **Client-Side Redirect Resolution**: Pages serving `<meta http-equiv="refresh">` client-side redirects are resolved to their target destinations up to 5 hops with domain boundary checks.
- **Deduplication**: Records are deduplicated by canonical URL and cleaned text SHA-256 hash, keeping only substantive content captures.
- **Independence**: Zero external dependencies (standard Python library only). Zero Git operations. Zero invented dates or citations.