How this was built

From raw interview to analyzed corpus

A twelve-step pipeline across eleven archives — source identification, transcript collection, multi-pass AI analysis, two-model cross-validation, and automated quality assurance. Applied to first-hand accounts.

The pipeline

End-to-end process, applied to all accounts

01
Corpus Assembly
Source identification

Identifying long-form interview archives that feature first-hand awakening and NDE accounts. Each source was evaluated on interview depth (typically 60–180 minutes), back-catalog size, caption availability on YouTube, and whether the content focuses on first-person experiential accounts rather than commentary or teaching. Eleven archives met the bar: Buddha at the Gas Pump, Conscious TV, Shaman Oaks, Simply Always Awake, a curated YouTube Mixed collection, the Kundalini Conversations series from Brent Spirit, a hand-curated selection from the Guru Viking channel, long-form NDE documentaries from Anthony Chene Production, first-hand spiritual emergency accounts from Integrative Mental Health University, and first-hand spiritually transformative experience accounts from Spiritual Awakenings International.

02
Corpus Assembly
Video library compilation

The complete video library for each archive is compiled automatically, then filtered by title to remove known non-qualifying content — panel discussions, book reviews, educational how-tos, and multi-guest roundtables. The same filter rules apply consistently at both the library and import stages, so nothing slips through from one step to the next. Output is a list of qualifying video URLs for each source.

03
Corpus Assembly
Transcript retrieval

Captions are retrieved from YouTube for each qualifying video — using auto-generated or manually-uploaded transcripts depending on what each video has available. Raw transcripts are saved as structured data, one file per source archive, ready for import.

04
Corpus Assembly
Import & normalization

Each transcript is imported and normalized into a structured record, with a second filtering pass that checks for minimum length and drops duplicates. Every episode record is a container that accumulates tags from each subsequent pipeline pass — all analysis layers are stored together in one place per account.

05
Quality Filtering
Quality rating & pruning

Each transcript receives a quality rating (high, medium, or low) during the first AI pass. Episodes rated low — insufficient experiential content, transcripts too short for analysis, apparent third-party summaries, or interviews where the guest's own experience is barely touched — are removed before the full analysis pipeline runs. The accounts in the corpus all cleared this threshold; the raw import count was higher.

06
AI Analysis
Broad tagging

Claude reads each full transcript and assigns tags across 17 experience categories. The interpretive standard is inclusive: any mention of a phenomenon counts — whether explicitly stated, implied through description, or referenced in passing. The broad pass also identifies trigger types, extracts divine presence descriptions, and captures guest metadata. These are the upper-bound prevalence figures seen throughout the data.

07
AI Analysis
Strict tagging

A second AI pass applies a higher evidentiary standard to the same transcripts. Only phenomena with explicit first-person description and sufficient phenomenological detail are tagged — implied references, single-word mentions, and metaphorical language do not qualify. The experiencer must clearly describe the phenomenon as something they directly underwent. These are the lower-bound figures: the most conservative reading of what was explicitly reported.

08
AI Analysis
Physical & integration extraction

Two dedicated passes extract additional data layers: somatic phenomena (energy surges, involuntary movement, physical collapse, sleep changes, visual and auditory shifts) and integration challenges (social isolation, identity disorientation, dark night of the soul, difficulty communicating the experience). Each layer is extracted at both broad and strict levels, following the same interpretive standards as the primary passes.

09
Cross-validation
Independent second-model tagging

All episodes are independently analyzed by a second AI model — GPT-4o — using an equivalent prompt, with no access to the first model's results. The two sets of tags are then compared for every episode across every category. Disagreements — cases where one model tagged a feature and the other did not — are flagged for adjudication. This step surfaces tags that are genuinely on the margin and forces closer reading of ambiguous passages.

10
Cross-validation
Adjudication

For each disagreement between the two models, a dedicated third pass re-reads the full transcript alongside both tag sets and issues a final determination with per-tag reasoning. This three-pass pipeline — broad analysis → independent cross-check → adjudication — is the quality-assurance backbone of the corpus. It reduces but does not eliminate tagging error; the broad/strict range holds that residual uncertainty explicitly rather than papering over it.

11
Quality Assurance
Corpus validation

An automated validation script checks the post-tagging corpus for structural integrity — including a check that every strict-tagged feature is also broad-tagged, required field presence, word count thresholds, episode counts against a saved baseline, and vocabulary consistency. Any episode failing validation is flagged for inspection before statistics are published.

12
Output
Statistical export

A generation script computes corpus-wide prevalence figures across all experience categories, trigger types, and sub-corpus groupings (NDE, by source). Results are published to the public site automatically. All figures reflect the post-filter, post-adjudication corpus of episodes.

Methodology

Understanding the broad / strict distinction

Broad
Broad interpretation

The broad read captures any mention of a phenomenon — whether it was stated explicitly, implied through description, or referenced in passing. If an account described "feeling like everything was connected" without using the word "unity," unity was tagged broadly.

Broad figures represent the upper bound: the scope of the corpus. They answer the question — across all accounts, how often does this phenomenon appear in any form? Use broad figures when you want to understand the reach of a given experience type.

Limitation: broad figures may include marginal mentions that fall short of explicit first-person confirmation. They should not be read as precise event counts.

Strict
Strict interpretation

The strict read requires an explicit, first-person description with sufficient phenomenological detail. Vague mentions, metaphors, and implications did not qualify. The experiencer had to clearly describe the phenomenon as something they underwent — not something they believe in generally or heard about from others.

Strict figures represent the evidentiary lower bound: what was explicitly confirmed. They answer the question — how often was this phenomenon clearly, unambiguously described in first-person terms? Use strict figures when precision matters.

How to read the gap: a large gap between broad and strict for a given feature suggests that the phenomenon is commonly referenced but less commonly described in explicit phenomenological detail. A narrow gap suggests consistent, direct description.

Concrete example

"If a subject says 'I felt like everything was one' during a discussion of their general worldview, the broad pass might tag unity-oneness; the strict pass would not — because it lacks the explicit first-person phenomenological description of the experience itself. If the same subject then described 'a moment where the boundary between me and everything else simply vanished,' both passes would tag it."

On accuracy

Why AI — and why it works

The most common reaction to AI-powered analysis is reasonable skepticism: AI makes mistakes, AI hallucinates, how do you know the tags are right? These are fair questions and the methodology was designed to address them directly.

Isn't AI less reliable than trained human researchers?
Not necessarily — and not in the ways that matter most here. The frontier AI models used in this project (Claude and GPT-4o) have been trained on essentially all published human knowledge, including psychology, psychiatry, neuroscience, philosophy of mind, and a vast literature on contemplative and transformative experiences. They bring domain expertise comparable to a PhD-level specialist to every transcript. What they don't bring is fatigue, drift in standards across a -interview project, or the subtle framing effects that come from a human analyst's own views on awakening.
AI hallucinates. How do you catch errors?
This concern is exactly why the pipeline uses two completely independent models. Claude and GPT-4o analyze every transcript separately, without access to each other's results. Where they agree, confidence is high. Where they disagree — one model tagged a feature and the other didn't — a third adjudication pass re-reads the full transcript and issues a final determination with explicit reasoning. The broad/strict dual standard also builds in structural honesty: rather than publishing a single point estimate, every figure is presented as a range that acknowledges interpretive uncertainty.
How does this improve over time?
This is perhaps the most important property of AI-powered analysis: it gets better automatically. Think of self-driving cars. Early autonomous systems were less reliable than experienced human drivers. But as they accumulated data and the models improved, they began to exceed human safety standards — because they apply what they learn consistently and at scale, without lapses. The same dynamic applies here. As AI models become more capable, re-running this analysis will produce more accurate, more nuanced results. A corpus analyzed today by frontier AI will be better analyzed tomorrow, as the models improve — with no additional human effort.
Does the AI bring its own interpretive biases?
Every human analyst brings assumptions — the contemplative tradition they practice, the therapeutic model they were trained in, the experiences that have shaped their intuitions. Managing this in qualitative research typically requires calibration sessions, inter-rater review, and reflexivity statements. AI models bring something different: no community affiliation, no spiritual practice, no career investment in what the data shows. Claude and GPT-4o have absorbed a vast literature on awakening, NDEs, and transformative experience — from Vedantic philosophy to clinical psychiatry to contemplative neuroscience — without having adopted any one framework as their interpretive home. The analysis is accountable to what the speaker actually said, not to a tradition the analyst belongs to.
Scale

What this analysis would have cost the traditional way

The interviews in this corpus average roughly 90 minutes each — about 1,100 hours of recorded testimony and 10 million words of transcript. Here is what a traditional human-researcher approach would have required, compared to the AI pipeline used.

Traditional approach — human researchers
~2,500 hrs
Analyst time — two independent readers per transcript, plus adjudication of disagreements. At 3 hours of analyst time per account across accounts.
$150k–$250k
Estimated cost at typical research rates ($60–$100/hr for PhD-level researchers with domain expertise in psychology and transpersonal studies).
6–12 months
Realistic timeline with a team of 2–3 full-time analysts — not including time spent on recruitment, training, calibration, or quality review.
Diminishing consistency
Human analysts drift in their interpretive standards over hundreds of transcripts. Tagging decisions made on interview 700 are rarely as consistent with interview 1 as they were at the start.
This project — AI pipeline
~$1,000
Total AI API cost across all tagging passes, cross-validation, and adjudication for the full corpus — including both Claude and GPT-4o.
Weeks, not months
Pipeline runs overnight and in parallel. No scheduling, no fatigue, no calibration sessions. New accounts can be added and analyzed in hours.
Consistent to the last transcript
The same analytical framework is applied to account 1 and the last account identically. No drift, no fatigue effects, no between-analyst disagreements about standards.
Improves automatically
As AI models become more capable, re-running the pipeline produces better results with no additional human investment. The corpus benefits from every advance in AI — indefinitely.
Pre-analysis filter
Quality filtering before full analysis

Before the full pipeline runs on any episode, each transcript receives a quality rating: high, medium, or low. This rating is assigned by the broad tagging pass as part of its first contact with the transcript.

Low-quality episodes — those with insufficient experiential content, transcripts that are too short to yield meaningful data, accounts that appear to be third-party summaries rather than direct first-person testimony, or interviews where the guest's own experience is barely touched — were removed from the corpus before analysis proceeded further.

The episodes in the final corpus all cleared the quality threshold. Some were borderline, and those were generally retained and allowed to receive a low quality rating rather than deleted — allowing downstream inspection rather than silent removal.

Quality filtering operates upstream of all prevalence figures presented on this site. The accounts are post-filter; the raw import count was higher.

Honest accounting
What this corpus is — and isn't

The corpus is a meaningful analytical tool. It is not a comprehensive or representative sample of all human awakening experiences. The following limitations should be kept in mind when interpreting the data.

1
Sample size. accounts is meaningful for pattern detection but not large enough to support fine-grained statistical inference. Prevalence figures, particularly for rare phenomena (below ~5%), should be read with caution.
2
English-language bias. All eleven source archives are English-language. This excludes the vast majority of documented awakening accounts globally, and likely skews the corpus toward Western, and particularly Anglo-American, cultural framings of the experience.
3
Long-form interview bias. The corpus is drawn entirely from video interviews typically lasting 60–180 minutes. Accounts obtained through long-form conversation may differ systematically from written accounts, brief descriptions, or accounts elicited through structured questionnaires. Interviewers' questions and framings may shape what subjects describe.
4
Self-selection bias. People who participate in public interviews about their awakening experience may differ in meaningful ways from those who don't. They may be more integrated, more verbally articulate about the experience, more embedded in spiritual communities, or more comfortable with the language of awakening. The corpus cannot speak for the silent majority.
5
AI analysis limitations. Both the broad and strict passes can err — through misinterpretation, inconsistency, or failure to detect a phenomenon described obliquely. The cross-validation and adjudication process reduces but does not eliminate this error. The broad/strict range is one tool for managing this uncertainty, not a solution to it.
6
The broad/strict range is not a confidence interval. It reflects two different interpretive standards applied to the same texts — not a statistical range around a true population prevalence. The "true" rate of any phenomenon in the full corpus almost certainly lies somewhere between the broad and strict figures, but the range does not map cleanly onto a frequentist confidence interval.