Phase 1.2 — Transformer Operations Specification

Status: Not started — Target: Weeks 3–8

Objective

Describe transformer architecture at the level of precision necessary to evaluate each Lacanian property from Phase 1.1. The description must be technically accurate (withstanding review by ML researchers) while remaining accessible to readers from the humanities.

Primary Sources

1. Vaswani et al., “Attention Is All You Need” (2017)
2. Elhage et al., “A Mathematical Framework for Transformer Circuits” (2021, Anthropic)
3. Olsson et al., “In-Context Learning and Induction Heads” (2022, Anthropic)
4. Templeton et al., “Scaling Monosemanticity: Extracting Interpretable Features from Claude 3 Sonnet” (2024, Anthropic)
5. Geva et al., “Transformer Feed-Forward Layers Are Key-Value Memories” (2021)
6. nostalgebraist, “interpreting GPT: the logit lens” (2020)
7. Belrose et al., “Eliciting Latent Predictions from Transformers with the Tuned Lens” (2023)

Deliverable: Computational Correlate Analysis

For each of the 10–15 Lacanian properties identified in Phase 1.1, provide:

• (a) Most plausible computational correlate — the transformer mechanism that most closely parallels the Lacanian operation
• (b) Most plausible point of disanalogy — where the parallel breaks down or the mechanisms diverge
• (c) Confirming/disconfirming evidence — what empirical findings would strengthen or weaken the correspondence

Property 1: Two Fundamental Axes → Attention vs. Feedforward?

Computational correlate:

Disanalogy:

Evidence criteria:

Property 2: Retroactive Meaning → Layer-wise Representation Change

Computational correlate:

Disanalogy:

Evidence criteria:

Property 3: Signifier-to-Signifier → Token Embedding Relations

Computational correlate:

Disanalogy:

Evidence criteria:

Property 4: Overdetermination → Superposition

Computational correlate:

Disanalogy:

Evidence criteria:

Property 5: Inaccessibility → No Self-Introspection

Computational correlate:

Disanalogy:

Evidence criteria:

Property 6: Constitutive Lack → Autoregressive Drive

Computational correlate:

Disanalogy:

Evidence criteria:

Property 7: Quilting Points → Phase Transitions Across Layers

Computational correlate:

Disanalogy:

Evidence criteria:

Property 8: Formations of the Unconscious → Hallucinations/Anomalous Outputs

Computational correlate:

Disanalogy:

Evidence criteria:

Properties 9–15

To be completed in parallel with Phase 1.1.

Technical Notes

Space for notes on transformer mechanisms encountered during source reading.