Is a chimpanzee or an LLM more like a human? Humans pass for LLMs; LLMs don't pass for humans. Subject-swapped, blind-rated sentences — for any five things, shared by link.
---
format: typebulb/v1
name: Chimp vs Chatbot
---
**code.tsx**
```tsx
import React, { useEffect, useMemo, useRef, useState } from "react"
import { createRoot } from "react-dom/client"
/* Run writes a corpus and rates every substituted sentence 0-100; Results reads it. The three
* objects on Results carry three different numbers: the grid is DIRECTED (column word in the row
* word's sentences), the ranking POOLED (both directions), the chart PAIRED (both candidates in
* the anchor's own sentences). Design record and limits: `notes.md`. */
/* Seeds the Run field only; everything drawn comes from the loaded run's own words and forms. */
const WORDS = ["human", "LLM", "chimpanzee", "frog", "virus"]
const CONC = 8 // polite enough not to 429, fast enough to be worth it
/** Unordered pair; `x` is whichever word comes first in the word list, so the key is stable. */
type Pair = { key: string; x: string; y: string }
/** One dot: a sentence written about `from`, scored with the other word as its subject. */
type Dot = { from: string; score: number | null }
type Cell = { anchor: string; subject: string; scores: (number | null)[] }
/** `subj` begins every rated sentence about the word (`An LLM`, `Golf`), `lab` is what prose
* calls it (`LLMs`). Model-resolved per run and stored with it. */
type Form = { subj: string; lab: string }
type Forms = Record<string, Form>
/** The forms plus the FRAME they belong to. Two runs of the same words in different frames are
* not comparable, so the frame is recorded with the run. */
type Resolved = { forms: Forms; frame: string }
type Run = {
timestamp: string; judge: string; provider?: string
n: number; words: string[]
forms?: Forms
frame?: string
corpus: Record<string, string[]>
results: Cell[]
}
type TbModel = { provider: string; name: string; friendlyName: string; providerName: string }
/** The id is listed under both `openrouter` and `openai` and only the OpenRouter key is funded,
* so the provider is part of the match and not just the name. */
const defaultModel = (ms: TbModel[]) => {
const luna = (p?: string) =>
ms.findIndex((m) => /luna/i.test(m.name) && (!p || m.provider === p))
const via = luna("openrouter")
return via >= 0 ? via : Math.max(0, luna())
}
// ── shape ────────────────────────────────────────────────────────────────────
/** No populated `results` is "no run yet" — a state, not an error. Cells naming a word this bulb
* no longer scores are hidden, so a run file stays the archive of what was measured. Also the
* validator: a run can arrive in a `#tb=` link, so nothing below trusts a field's type. */
const str = (v: unknown): v is string => typeof v === "string"
const num = (v: unknown) => (typeof v === "number" && Number.isFinite(v) ? v : null)
function normalize(raw: unknown): Run | null {
const d = raw as Partial<Run>
if (!d || !Array.isArray(d.results) || !d.results.length || !d.corpus) return null
const words = Array.isArray(d.words) && d.words.some(str) ? d.words.filter(str) : WORDS
/* a cell past the word list is the archive working; a mis-shaped cell is a bad payload */
const shaped = d.results.filter((c) => c && Array.isArray(c.scores) && str(c.anchor) &&
str(c.subject))
const live = shaped
.filter((c) => words.includes(c.anchor) && words.includes(c.subject))
.map((c) => ({ anchor: c.anchor, subject: c.subject, scores: c.scores.map(num) }))
const dropped = shaped.length - live.length, junk = d.results.length - shaped.length
if (dropped) tb.log(`[data] hiding ${dropped} cell(s) naming a word no longer scored`)
if (junk) tb.log(`[data] ignoring ${junk} cell(s) that are not shaped like cells`)
if (!live.length) return null
/* prepended to a subject and rendered as-is, so string-only */
const src = d.corpus as Record<string, unknown>
const corpus: Record<string, string[]> = {}
for (const w of words) corpus[w] = Array.isArray(src[w]) ? (src[w] as unknown[]).filter(str) : []
/* older runs stored `forms` as a bare subject string; read it, degrading the label */
const fsrc = (d.forms ?? {}) as Record<string, unknown>
const forms: Forms = {}
for (const w of words) {
const f = fsrc[w]
if (str(f)) forms[w] = { subj: f, lab: w }
else if (f && typeof f === "object" && str((f as Form).subj))
forms[w] = { subj: (f as Form).subj, lab: str((f as Form).lab) ? (f as Form).lab : w }
}
return {
timestamp: str(d.timestamp) ? d.timestamp : "",
judge: str(d.judge) ? d.judge : "an unrecorded model",
provider: str(d.provider) ? d.provider : undefined,
n: num(d.n) ?? Math.max(...live.map((c) => c.scores.length)),
words, forms: forms && Object.keys(forms).length ? forms : undefined,
frame: str(d.frame) ? d.frame : undefined,
corpus, results: live,
}
}
function baked(): Run | null {
try { return normalize(tb.json<unknown>(0)) } catch { return null }
}
// ── words ────────────────────────────────────────────────────────────────────
/* Every rated string is `<subject> <predicate>` and the predicate is shared VERBATIM between the
* candidates, so every word must take the SAME verb agreement or the rating measures grammar
* instead of the claim. `resolveForms` picks that slot per run. */
const cap = (w: string) => w.charAt(0).toUpperCase() + w.slice(1)
/** Naming comes from the run, not from this build: its forms are the literal text of the
* sentences that were rated. A run predating recorded forms falls back to the bare word. */
type Naming = {
subj: (w: string) => string // what a rated sentence begins with
lab: (w: string) => string // what the UI calls the word in prose
say: (w: string, predicate: string) => string // the invariant: subject, space, predicate
}
const naming = (forms?: Forms): Naming => {
const subj = (w: string) => cap((forms?.[w]?.subj ?? w).trim())
const lab = (w: string) => (forms?.[w]?.lab ?? w).trim()
return { subj, lab, say: (w, predicate) => `${subj(w)} ${predicate}` }
}
// ── stats ────────────────────────────────────────────────────────────────────
const key = (a: string, b: string) => `${a}|${b}`
/** Word-list order, which decides which of a pair is `a` — purple, and the circle. */
const byList = (words: string[]) => (m: string, n: string) => words.indexOf(m) - words.indexOf(n)
const mean = (xs: number[]) => (xs.length ? xs.reduce((s, v) => s + v, 0) / xs.length : 0)
const CEIL = 90, FLOOR = 10
type Stat = {
pair: Pair; ok: Dot[]
score: number // the row's number: mean substituted rating, 0-100
xy: number; yx: number // the same, split by direction: y in x's sentences, and back
asym: number // yx − xy; drawn as the dot's shape, not in the headline
saturated: boolean // interchangeable, and the scale has run out of room to say so
floored: boolean // the same failure upside down
}
const pairsOf = (words: string[]): Pair[] => {
const out: Pair[] = []
for (let i = 0; i < words.length; i++)
for (let j = i + 1; j < words.length; j++)
out.push({ key: key(words[i], words[j]), x: words[i], y: words[j] })
return out
}
const cellMap = (run: Run) => new Map(run.results.map((c) => [key(c.anchor, c.subject), c]))
/** Both directions concatenated. A sentence appears once, under the word it was written about,
* with the OTHER word swapped in — never twice, and never against itself. */
function dotsFor(run: Run, cells: Map<string, Cell>, p: Pair): Dot[] {
const half = (from: string, into: string) =>
(run.corpus[from] ?? []).map((_s, i) => ({
from, score: cells.get(key(from, into))?.scores[i] ?? null,
}))
return [...half(p.x, p.y), ...half(p.y, p.x)]
}
function stat(pair: Pair, dots: Dot[]): Stat {
const ok = dots.filter((d) => d.score != null)
const avg = (ds: Dot[]) => mean(ds.map((d) => d.score as number))
const score = avg(ok)
const xy = avg(ok.filter((d) => d.from === pair.x))
const yx = avg(ok.filter((d) => d.from === pair.y))
return {
pair, ok, score, xy, yx, asym: yx - xy,
saturated: ok.length > 0 && score >= CEIL,
floored: ok.length > 0 && score <= FLOOR,
}
}
/** Every pair, best first — the ranking IS the finding, so it is also the row order. */
function pairStats(run: Run): Stat[] {
const cells = cellMap(run)
return pairsOf(run.words)
.map((p) => stat(p, dotsFor(run, cells, p)))
.filter((s) => s.ok.length)
.sort((a, b) => b.score - a.score)
}
// ── the triad ────────────────────────────────────────────────────────────────
/* An anchor and two candidates. The anchor names the SHARED sentences, so both candidates are
* paired sentence by sentence — cleaner than the pooled row, on half the dots (n, not 2n). */
type Tri = { anchor: string; a: string; b: string }
type TriRow = { sentence: string; a: number | null; b: number | null; was: number | null }
type TriStat = {
tri: Tri; rows: TriRow[]; ok: TriRow[]
meanA: number; meanB: number
saturated: boolean // both candidates fit, and the scale has no room to say more
floored: boolean // and the same failure upside down
}
/** Two distinct pairs share at most one word, so the anchor is never ambiguous. */
function triOf(words: string[], p: Pair, q: Pair): Tri | null {
const anchor = [p.x, p.y].find((w) => w === q.x || w === q.y)
if (!anchor) return null
const ca = p.x === anchor ? p.y : p.x
const cb = q.x === anchor ? q.y : q.x
if (ca === cb) return null
const [a, b] = [ca, cb].sort(byList(words))
return { anchor, a, b }
}
function triStat(run: Run, cells: Map<string, Cell>, tri: Tri): TriStat {
const at = (w: string, i: number) => cells.get(key(tri.anchor, w))?.scores[i] ?? null
const rows = (run.corpus[tri.anchor] ?? []).map((sentence, i) => ({
sentence, a: at(tri.a, i), b: at(tri.b, i), was: at(tri.anchor, i),
}))
const ok = rows.filter((r) => r.a != null && r.b != null)
const meanA = mean(ok.map((r) => r.a as number))
const meanB = mean(ok.map((r) => r.b as number))
return {
tri, rows, ok, meanA, meanB,
saturated: ok.length > 0 && meanA >= CEIL && meanB >= CEIL,
floored: ok.length > 0 && meanA <= FLOOR && meanB <= FLOOR,
}
}
// ── generation ───────────────────────────────────────────────────────────────
async function pool<T>(jobs: (() => Promise<T>)[], width: number,
tick: () => void, stop: () => boolean): Promise<void> {
let next = 0
await Promise.all(Array.from({ length: Math.min(width, jobs.length) }, async () => {
for (;;) {
const i = next++
if (i >= jobs.length || stop()) return
await jobs[i]()
tick()
}
}))
}
/** What is stored is the PREDICATE, not the sentence — substitution is a prepend. Asked for N in
* one call a model returns fewer, repeats itself, or both, so `generate` dedupes and tops up. */
const norm = (s: string) => s.toLowerCase().replace(/[^a-z0-9 ]/g, "").replace(/\s+/g, " ").trim()
/** `stringify(x, null, 1)` puts every score on its own line, so number arrays are folded back
* onto one line. String arrays are left expanded: one sentence per line reads well. */
const toJson = (o: unknown) =>
JSON.stringify(o, null, 1).replace(
/\[\s*((?:-?\d+(?:\.\d+)?|null)(?:\s*,\s*(?:-?\d+(?:\.\d+)?|null))*)\s*\]/g,
(_m, body: string) => `[${body.split(/\s*,\s*/).join(", ")}]`)
/* "A human body requires…" substitutes a MODIFIER, not a subject, so the rating is of a claim
* about something else. The prompt is the real fix; telling a bare noun from a verb needs a POS
* tagger, so only the unambiguous cases are rejected — a determiner, possessive or hyphen. */
const NOT_A_PREDICATE = /^(?:(?:a|an|the|its|their|his|her|our|your|my)\b|['’]s\b|-)/i
const looksLikePredicate = (body: string) => !NOT_A_PREDICATE.test(body)
async function generate(m: TbModel, anchor: string, n: number, N: Naming): Promise<string[]> {
const subj = N.subj(anchor)
const out: string[] = []
const seen = new Set<string>()
let calls = 0, dupes = 0, shaped = 0
for (let pass = 0; pass < 4 && out.length < n; pass++) {
const want = n - out.length
const avoid = out.length
? ` Do not repeat any of these, or restate them in other words:\n` +
out.map((s) => N.say(anchor, s)).join("\n")
: ""
const { text } = await tb.ai({
provider: m.provider, model: m.name, webSearch: false,
system: "You write plain declarative sentences. No preamble, no numbering, no commentary.",
messages: [{
role: "user",
content:
`Write ${Math.max(want, 6)} different general statements about ${N.lab(anchor)}, ` +
`each beginning with the exact phrase "${subj}" followed immediately by a VERB.\n` +
`"${subj}" must be the grammatical subject, not part of a longer noun phrase: write ` +
`"${subj} uses tools", never "${subj} hand is dexterous" or "${subj} mind wanders" — ` +
`the words after "${subj}" are handed to a different subject unchanged, so anything ` +
`that turns "${subj}" into a modifier breaks the sentence it is handed to.\n` +
`Make each one SUBSTANTIVE — a specific claim with real content, around 8 to 16 words. ` +
`"${subj} learns from experience" is too thin to tell anything apart; ` +
`"${subj} learns from experience and changes its behaviour over time" is the weight ` +
`wanted.\n` +
`Vary the subject matter widely. One per line, nothing else.${avoid}`,
}],
})
calls++
for (const line of text.split("\n")) {
const l = line.trim().replace(/^[-*\d.)\s]+/, "")
if (!l.toLowerCase().startsWith(subj.toLowerCase())) continue
const body = l.slice(subj.length).trim()
if (body.length <= 12) continue
if (!looksLikePredicate(body)) { shaped++; continue }
const k = norm(body)
if (seen.has(k)) { dupes++; continue }
seen.add(k)
out.push(body)
if (out.length >= n) break
}
}
tb.log(`[corpus] ${anchor}: ${out.length}/${n} in ${calls} call(s)` +
`${dupes ? `, ${dupes} duplicate(s) dropped` : ""}` +
`${shaped ? `, ${shaped} not a predicate` : ""}` +
`${out.length < n ? " — SHORT, the anchor starved" : ""}`)
return out
}
/** 0-100: how much sense does this sentence make as a claim about the world?
*
* Blind by construction — one sentence, no candidates named, no topic stated, no way to infer
* which answer is wanted. That is the whole defence against the model playing along. */
async function rate(m: TbModel, sentence: string): Promise<number | null> {
const { text } = await tb.ai({
provider: m.provider, model: m.name, webSearch: false,
system:
"You rate how much sense a statement makes as a claim about the real world. " +
"100 = obviously true and unremarkable. 50 = arguable or strained. " +
"0 = nonsense, or plainly false of the thing named. " +
"Reply with a single integer from 0 to 100 and nothing else.",
messages: [{ role: "user", content: sentence }],
})
const mm = text.match(/\d+/)
if (!mm) return null
const v = parseInt(mm[0], 10)
return v >= 0 && v <= 100 ? v : null
}
/** The words go in TOGETHER, because what a word should become depends on its company: `porn,
* video games, gambling` has no per-word subject form that works, but as ACTIVITIES all three
* fall out substitutable (`Watching porn`, …). The objective given is substitutability, not
* correctness — a "does this make sense" rating cannot tell an absurd claim from an
* ungrammatical one. A form is therefore not always a form OF the typed word.
*
* Retried once, then fatal: a missing form is a whole word's worth of ungrammatical sentences. */
async function resolveForms(m: TbModel, words: string[]): Promise<Resolved> {
const ask = async (): Promise<Resolved> => {
const { text } = await tb.ai({
provider: m.provider, model: m.name, webSearch: false,
/* the one call worth paying for thinking: it decides the wording of every call after it,
* and a bad frame cannot be recovered downstream */
effort: 3,
system: "You answer in the exact line format asked for, and nothing else.",
messages: [{
role: "user",
content:
`These ${words.length} things are about to be compared with each other. Each will ` +
`begin a general statement, and the remainder of that statement is then handed to ` +
`every other one VERBATIM. So all ${words.length} subjects must agree with the same ` +
`verb and be interchangeable in one another's sentences.\n\n` +
`YOUR JOB IS TO CHOOSE THE FRAME. There is no fixed one, and picking well is the ` +
`whole task: the sentences are rated on whether they make sense, so a frame that ` +
`makes any swap read awkwardly poisons those ratings with grammar nobody meant to ` +
`be judging. We want the comparison to turn on real differences between these ` +
`things.\n\n` +
`Work out what kind of thing the set is comparing, then pick the frame that carries ` +
`that kind most naturally. Frames that work, and when:\n` +
`• PLURAL — "Humans", "Chimpanzees", "LLMs". Best for countable kinds of thing. ` +
`Takes "their", never needs an article, and cannot be absorbed into a noun phrase.\n` +
`• GENERIC SINGULAR — "A human", "An LLM", "A virus"; article by sound. Use when the ` +
`individual is the natural unit.\n` +
`• ACTIVITY — "Watching porn", "Playing video games", "Gambling". The only frame ` +
`that holds mass nouns and activities, which have no plural at all.\n` +
`• Any other frame, if it fits the set better than these.\n\n` +
`A mixed set is cast INTO one frame, not left mixed: "porn, video games, ` +
`gambling" is a comparison of activities, so it goes to Watching porn | Playing ` +
`video games | Gambling — departing from the words as typed, because no ` +
`per-word answer would fit the others' sentences.\n\n` +
`Reply with exactly ${words.length + 1} lines.\n` +
`Line 1: "FRAME: <name> — <one clause on why this set takes it>"\n` +
`Then one line per thing, in the order given, as "word -> Subject | label", where ` +
`Subject is what begins its sentences and label is what prose calls it:\n` +
` plural frame: human -> Humans | humans LLM -> LLMs | LLMs\n` +
` singular frame: human -> A human | humans LLM -> An LLM | LLMs\n` +
` activity frame: golf -> Playing golf | golf gambling -> Gambling | gambling\n` +
`Every Subject must belong to the SAME frame. Nothing else.\n\n` +
words.join("\n"),
}],
})
const out: Forms = {}
let frame = ""
for (const line of text.split("\n")) {
const fr = /^\s*FRAME\s*[:\-—–]\s*(.+?)\s*$/i.exec(line)
if (fr) { frame = fr[1].replace(/^["“]+|["”]+$/g, "").trim(); continue }
const hit = /^\s*(.+?)\s*(?:->|—|–|:)\s*(.+?)\s*\|\s*(.+?)\s*$/.exec(line)
if (!hit) continue
const asked = hit[1].replace(/^[-*\d.)\s]+/, "").toLowerCase()
const w = words.find((x) => x.toLowerCase() === asked)
const clean = (s: string) => s.replace(/^["“]+|["”.]+$/g, "").trim()
// a subject not starting with a letter puts the predicate first — treat it as no answer
if (w && /^[A-Za-z]/.test(clean(hit[2]))) out[w] = { subj: cap(clean(hit[2])), lab: clean(hit[3]) }
}
return { forms: out, frame }
}
let { forms, frame } = await ask()
let missing = words.filter((w) => !forms[w])
/* a mixed frame is checkable from the strings: a leading article is the generic singular and
* nothing else, so it is all of the countable words or none */
const split = (f: Forms) => {
const arts = words.filter((w) => f[w] && /^an?\s/i.test(f[w].subj)).length
return arts > 0 && arts < words.filter((w) => f[w]).length
}
if (missing.length || split(forms)) {
tb.log(`[forms] retrying — ${missing.length ? `nothing usable for ${missing.join(", ")}`
: "the frame came back mixed"}`)
const again = await ask()
if (missing.length) { forms = { ...again.forms, ...forms }; frame = frame || again.frame }
else if (!split(again.forms)) ({ forms, frame } = again)
missing = words.filter((w) => !forms[w])
}
if (missing.length)
throw new Error(`could not work out how to say ${missing.join(", ")} — try other wording`)
if (split(forms)) tb.log(`[forms] WARNING: frame still mixed — some subjects take an article`)
tb.log(`[forms] ${frame || "frame unstated"} :: ` +
words.map((w) => `${w} → ${forms[w].subj} / ${forms[w].lab}`).join(", "))
return { forms, frame }
}
// ── chart primitives ─────────────────────────────────────────────────────────
/* Colour carries whatever the geometry cannot. In the matrix position is identity, so colour is
* the VALUE — a sequential ramp read off a key. In the strip position is the value, so colour is
* IDENTITY — purple for the first candidate, orange for the second, with circle and triangle
* saying the same thing again for colour vision. The two systems must not share hues, which is
* why purple and orange sit outside the cool wedge the ramp walks. */
/** Verbatim from `you-re-relatively-right.bulb.md`, so the two bulbs can be diffed against each
* other. Direction is flipped there: the `1 - t` in `shade` does it, not a reversed array. */
const RAMP = ["#4458cb", "#3e9bfe", "#18d6cb", "#46f884", "#a2fc3c", "#e1dd37", "#cfcb62", "#b5b285", "#9c9a90", "#878683"]
/** Opaque, so the same number is the same colour in either theme. sRGB and linear like the
* source bulb — a shared palette walked two different ways is not a shared palette. */
const shade = (v: number) => {
const t = 1 - Math.max(0, Math.min(100, v)) / 100
const x = t * (RAMP.length - 1)
const i = Math.min(RAMP.length - 2, Math.floor(x))
const f = x - i
const hex = (h: string, k: number) => parseInt(h.slice(k, k + 2), 16)
const ch = (k: number) => Math.round(hex(RAMP[i], k) + (hex(RAMP[i + 1], k) - hex(RAMP[i], k)) * f)
return `rgb(${ch(1)} ${ch(3)} ${ch(5)})`
}
/** Sampled from `shade`, so the key cannot drift from the cells. One stop per anchor exactly:
* gradient and `shade` both interpolate linearly, so the scale is reproduced, not approximated. */
const RampStops = () => (
<>
{RAMP.map((_c, i) => (
<stop key={i} offset={i / (RAMP.length - 1)} stopColor={shade((i / (RAMP.length - 1)) * 100)} />
))}
</>
)
/** The one geometry both marks wear. */
const Shape = ({ tri }: { tri: boolean }) =>
tri ? <path d="M12 6 L18.5 17.5 L5.5 17.5 Z" /> : <circle cx="12" cy="12" r="6.2" />
/** The candidate glyph at text size, for the title and the quoted sentences. */
const Glyph = ({ tri }: { tri: boolean }) => (
<svg className={"glyph " + (tri ? "b" : "a")} viewBox="0 0 24 24" aria-hidden="true">
<Shape tri={tri} />
</svg>
)
/** Filled with the surface colour and edged in the candidate's: a mark on a cell is a POINTER
* and must read on any colour the ramp can produce, where a strip dot is a DATUM. */
const CellMark = ({ tri }: { tri: boolean }) => (
<svg className={"cmark " + (tri ? "b" : "a")} viewBox="0 0 24 24" aria-hidden="true">
<Shape tri={tri} />
</svg>
)
/** The picker: a row is the "given a human" clause and two cells in it are the two candidates.
*
* THE CELL IS DIRECTED — the COLUMN word read in the ROW word's sentences — so the grid is not
* symmetric, a square against its mirror is the direction gap, and the two picked squares are
* exactly the two averages the chart below draws. The pooled number lives in the ranking.
*
* The diagonal is drawn at the top of the scale but not selectable. */
function Matrix({ words, lab, val, dir, row, pair, onCell, onRow }: {
words: string[]; lab: (w: string) => string
val: (a: string, b: string) => number | null
dir: (sentencesAbout: string, substituted: string) => number | null
row: string; pair: string[] // ordered by the word list, so pair[0] is the circle
onCell: (r: string, c: string) => void; onRow: (r: string) => void
}) {
return (
<div className="mx-wrap">
<div className="mx"
style={{ gridTemplateColumns: `auto repeat(${words.length}, var(--cell))` }}>
<div className="corner" />
{words.map((c) => <div className="colLabel" key={c}>{lab(c)}</div>)}
{words.map((r) => (
<React.Fragment key={r}>
<button className={"rowLabel" + (row === r ? " on" : "")} aria-pressed={row === r}
aria-label={`which substitutes for ${lab(r)}`} onClick={() => onRow(r)}>
{lab(r)}
</button>
{words.map((c) => {
/* a word in its own sentences is a real measurement, so it is drawn — but a word
is not a candidate to replace itself, so it is not selectable */
const v = dir(r, c)
if (v == null) return <div className="cell none" key={c} aria-label="no data" />
if (r === c) return (
<div className="cell diag" key={c} role="img" style={{ background: shade(v) }}
aria-label={`${lab(r)} in their own sentences: ${v.toFixed(0)} of 100`} />
)
const on = row === r && pair.includes(c)
const back = dir(c, r), pooled = val(r, c)
// attributive singular — "in human sentences", not "in humans sentences"
const more =
(back == null ? "" : `\nthe other way round, ${lab(r)} in ${c} sentences ` +
back.toFixed(1)) +
(pooled == null ? "" : `\nthe pair, both ways pooled ${pooled.toFixed(1)}`)
return (
<button key={c} aria-pressed={on}
className={"cell" + (on ? " sel" : "")}
style={{ background: shade(v) }}
title={`${lab(c)} in ${r} sentences — ${v.toFixed(1)} of 100${more}`}
aria-label={
`${lab(c)} in ${r} sentences: ${v.toFixed(0)} of 100`}
onClick={() => onCell(r, c)}>
{on && <CellMark tri={c !== pair[0]} />}
</button>
)
})}
</React.Fragment>
))}
</div>
<div className="legend">
{/* its own gradient id: ids resolve document-wide, and the legend must not depend on
the strip being mounted */}
<svg className="lgBar" viewBox="0 0 302 12" preserveAspectRatio="none" aria-hidden="true">
<defs>
<linearGradient id="lgramp"><RampStops /></linearGradient>
</defs>
<rect className="axisbar" fill="url(#lgramp)" x="1" y="1" width="300" height="10" />
</svg>
<span className="lgEnds">
<span>nonsense</span>
<span className="mid">← substitution →</span>
<span>makes sense</span>
</span>
</div>
</div>
)
}
// ── views ────────────────────────────────────────────────────────────────────
/** A viewBox scales the labels and dots down with everything else, so narrow gets a second
* geometry rather than a smaller drawing: fewer viewBox units, same marks, real sizes. */
const narrowQ = window.matchMedia("(max-width: 640px)")
function useNarrow(): boolean {
const [narrow, setNarrow] = useState(narrowQ.matches)
useEffect(() => {
const on = () => setNarrow(narrowQ.matches)
narrowQ.addEventListener("change", on)
return () => narrowQ.removeEventListener("change", on)
}, [])
return narrow
}
/* LAB and TICK are in viewBox units, so they track W rather than sit in CSS; `clear` is what the
* first dot lane needs to duck the mean labels. */
const GEO = {
wide: { W: 900, H: 251, PAD: 54, R: 8, base: 210, TOP_A: 26, TOP_B: 56, LAB: 22, TICK: 13,
clear: 16, ticks: [0, 25, 50, 75, 100] },
narrow: { W: 380, H: 273, PAD: 22, R: 7, base: 232, TOP_A: 24, TOP_B: 52, LAB: 15, TICK: 12,
clear: 12, ticks: [0, 50, 100] },
}
/** Two dots per sentence, one per candidate, on the anchor's own sentences — so the two clouds'
* separation is fully paired. SHAPE means which candidate. */
function TriStrip({ t, N, sel, onSel, narrow }: {
t: TriStat; N: Naming; sel: number; onSel: (i: number) => void; narrow: boolean
}) {
/* `base` is set outright rather than derived from H, which the viewBox silently clips when it
* overshoots. */
const { W, H, PAD, R, base, TOP_A, TOP_B, LAB, TICK, ticks, clear } =
narrow ? GEO.narrow : GEO.wide
const BIN = R * 2 + 2
const x = (v: number) => PAD + (v / 100) * (W - PAD * 2)
const pts: { i: number; v: number; tri: boolean }[] = []
t.rows.forEach((r, i) => {
if (r.a != null) pts.push({ i, v: r.a, tri: false })
if (r.b != null) pts.push({ i, v: r.b, tri: true })
})
if (!pts.length) return <p className="note">Nothing scored for this triad.</p>
/* Stacks spill SIDEWAYS at the top of the plot rather than growing through the labels or being
* clipped. Neighbouring bins are tried emptiest-first, so a pile-up spreads evenly. */
const laneH = R * 2 + 3
const top = TOP_B + clear // the first lane must clear the labels
const lanes = Math.max(1, Math.floor((base - top) / laneH) + 1)
const binMax = Math.round((W - PAD * 2) / BIN)
const fill: Record<number, number> = {}
const seat = (want: number) => {
for (let d = 0; d <= binMax; d++) {
const side = d === 0 ? [want] : [want - d, want + d]
const open = side
.filter((b) => b >= 0 && b <= binMax && (fill[b] ?? 0) < lanes)
.sort((a, b) => (fill[a] ?? 0) - (fill[b] ?? 0))
if (open.length) {
const b = open[0]
const lane = fill[b] ?? 0
fill[b] = lane + 1
return { b, lane }
}
}
return { b: want, lane: 0 } // every bin full: only with absurd counts
}
const placed = pts.sort((p, q) => p.v - q.v).map((p) => {
const { b, lane } = seat(Math.round((x(p.v) - PAD) / BIN))
return { ...p, cx: PAD + b * BIN, lane }
})
const mark = (cx: number, cy: number, tri: boolean) =>
tri ? `M${cx},${cy - R - 1} L${cx + R + 1},${cy + R} L${cx - R - 1},${cy + R} Z`
: `M${cx - R},${cy} a${R},${R} 0 1 0 ${R * 2},0 a${R},${R} 0 1 0 ${-R * 2},0`
return (
<div className="strip-wrap">
{/* not "more LLM-like": likeness reads as symmetric and this measure is not */}
<p className="tagline">Which substitutes for <b>{N.lab(t.tri.anchor)}</b> better?</p>
<svg viewBox={`0 0 ${W} ${H}`} className="strip" role="img"
aria-label={`${N.lab(t.tri.a)} and ${N.lab(t.tri.b)} read into sentences about ` +
N.lab(t.tri.anchor)}>
{/* the axis IS the matrix legend's ramp: both `shade()`-sampled, so neither can drift */}
<defs>
<linearGradient id="ramp"><RampStops /></linearGradient>
</defs>
<rect className="axisbar" fill="url(#ramp)" x={PAD} y={base + 11}
width={W - PAD * 2} height={10} />
{ticks.map((v) => (
<g key={v}>
<line x1={x(v)} y1={base + 23} x2={x(v)} y2={base + 27} className="axis" />
<text x={x(v)} y={base + 37} textAnchor="middle" className="tick"
style={{ fontSize: TICK }}>{v}</text>
</g>
))}
{/* drawn before the dots so the cloud sits over its own average marker */}
{([[t.meanA, t.tri.a, false], [t.meanB, t.tri.b, true]] as const).map(([m, w, isB]) => (
<g key={w}>
<text x={x(m)} y={isB ? TOP_B : TOP_A} textAnchor="middle"
className={"meanlab" + (isB ? " b" : "")} style={{ fontSize: LAB }}>
{N.lab(w)} {Math.round(m)}%
</text>
<line x1={x(m)} y1={(isB ? TOP_B : TOP_A) + 10} x2={x(m)} y2={base + 11}
className={"meanline" + (isB ? " b" : "")} />
</g>
))}
{placed.map((p, k) => (
<path key={k} d={mark(p.cx, base - p.lane * laneH, p.tri)}
className={"dot " + (p.tri ? "b" : "a") + (p.i === sel ? " on" : "")}
onClick={() => onSel(p.i)}>
<title>
{`${N.say(p.tri ? t.tri.b : t.tri.a, t.rows[p.i].sentence)}\n${p.v} of 100`}
</title>
</path>
))}
</svg>
{/* the axis's label. HTML rather than SVG text, so it holds its size at any width and
wraps instead of colliding; the inline padding aligns it with the bar's ends. */}
<div className="ends" style={{ padding: `0 ${((PAD / W) * 100).toFixed(1)}%` }}>
<span>nonsense</span>
<span className="mid">← substitution →</span>
<span>makes sense</span>
</div>
</div>
)
}
/** Every pair in order, which no colour scale can show. A row is POOLED (symmetric) where a cell
* is directed. Clicking a row is the same move as clicking its square: the anchor is kept if
* the pair contains it, otherwise taken from the pair. */
function Ranks({ stats, lab, row, a, b, onPick }: {
stats: Stat[]; lab: (w: string) => string
row: string; a: string; b: string; onPick: (p: Pair) => void
}) {
const side = (p: Pair) => {
const has = (w: string) => p.x === w || p.y === w
if (!has(row)) return ""
return has(a) ? " a" : has(b) ? " b" : ""
}
return (
<div className="ranks" role="group" aria-label="most substitutable pairs">
<div className="ranks-h">Most substitutable pairs</div>
{stats.map((s, i) => (
<button key={s.pair.key} className={"rank" + side(s.pair)}
aria-label={
`rank ${i + 1}, ${lab(s.pair.x)} and ${lab(s.pair.y)}: ` +
`${s.score.toFixed(0)} of 100`}
onClick={() => onPick(s.pair)}>
<span className="rank-n">{s.pair.x} · {s.pair.y}</span>
<span className="rank-v">{s.score.toFixed(1)}</span>
</button>
))}
</div>
)
}
/** One stroke to each candidate in its own colour, carrying the relation the heading no longer
* states. Fixed geometry, not measured to the words, so a long word cannot throw it off. */
const Fork = () => (
<svg className="fork" viewBox="0 0 140 30" aria-hidden="true">
<g fill="none" strokeWidth="1.6" strokeLinecap="round" strokeLinejoin="round">
<path className="fk-a" d="M70,3 L17,25 M24,26 L17,25 L21,18.5" />
<path className="fk-b" d="M70,3 L123,25 M116,26 L123,25 L119,18.5" />
</g>
</svg>
)
function Read({ run }: { run: Run }) {
const stats = useMemo(() => pairStats(run), [run])
const cells = useMemo(() => cellMap(run), [run])
/* from the RUN, so a file is drawn and quoted in the forms it was measured in */
const N = useMemo(() => naming(run.forms), [run])
const narrow = useNarrow()
const score = useMemo(() => {
const m = new Map(stats.map((s) => [s.pair.key, s.score]))
return (a: string, b: string) => {
const [x, y] = [a, b].sort(byList(run.words))
return m.get(key(x, y)) ?? null
}
}, [stats, run])
/** One direction: how the SUBSTITUTED word read in sentences written about the other. This is
* exactly what the pooled cell averages away, kept so the tooltip can hand it back. */
const dir = useMemo(() => {
const means = new Map<string, number>()
for (const c of run.results) {
const ok = c.scores.filter((v): v is number => v != null)
if (ok.length) means.set(key(c.anchor, c.subject), mean(ok))
}
return (about: string, sub: string) => means.get(key(about, sub)) ?? null
}, [run])
/* the default triad: the top pair plus the best pair sharing a word with it */
const start = useMemo(() => {
const first = stats[0]
if (!first) return null
for (const s of stats.slice(1)) {
const t = triOf(run.words, first.pair, s.pair)
if (t) return t
}
return { anchor: first.pair.x, a: first.pair.y, b: first.pair.y }
}, [stats, run])
const [row, setRow] = useState<string>(() => start?.anchor ?? run.words[0])
const [pair, setPair] = useState<string[]>(() =>
start ? [start.a, start.b] : run.words.slice(1, 3))
/* a SENTENCE, not an index, so it survives any move whose corpus still holds it */
const [pin, setPin] = useState<string | null>(null)
/* a word does not stand in for itself, so a move leaving fewer than two candidates tops up
* with the word this row is most similar to */
const fill = (r: string, want: string[]) => {
const out = want.filter((w) => w !== r)
const rest = run.words
.filter((w) => w !== r && !out.includes(w))
.sort((m, n) => (score(r, n) ?? 0) - (score(r, m) ?? 0))
while (out.length < 2 && rest.length) out.unshift(rest.shift() as string)
return out.slice(-2)
}
const pickCell = (r: string, c: string) => {
setRow(r)
setPair((p) => fill(r, [...p.filter((w) => w !== c), c]))
}
const pickRow = (r: string) => { setRow(r); setPair((p) => fill(r, p)) }
// a ranked row names a pair, not a triad: keep the anchor if the pair contains it
const pickPair = (p: Pair) => {
const anchor = row === p.x || row === p.y ? row : p.x
pickCell(anchor, anchor === p.x ? p.y : p.x)
}
// candidates ordered by the word list, which is the strip's own left to right
const [a, b] = [...pair].sort(byList(run.words))
const tri: Tri = { anchor: row, a, b }
const t = useMemo(() => triStat(run, cells, tri), [run, cells, row, a, b])
if (!stats.length) return <p className="note">No run to show yet.</p>
let idx = pin ? t.rows.findIndex((r) => r.sentence === pin) : -1
if (idx < 0) { // the sentence the two candidates most disagree about
let wv = -Infinity
t.rows.forEach((r, i) => {
if (r.a == null || r.b == null) return
if (Math.abs(r.b - r.a) > wv) { wv = Math.abs(r.b - r.a); idx = i }
})
if (idx < 0) idx = 0
}
const r = t.rows[idx]
const onSel = (i: number) => setPin(t.rows[i].sentence)
/* a `#tb=` payload whose `corpus` does not cover the words its cells name */
if (!r) return (
<p className="note warn">
This run has ratings but no sentences behind {N.lab(row)} — nothing to quote. The
file or link is incomplete.
</p>
)
return (
<>
{/* the anchor above, a fork down to the two candidates: position carries the relation, so
the pair keeps its symmetry. Bare words — the subject forms belong in the sentences. */}
<header className="triad">
<div className="anchor" aria-hidden="true">{cap(row)}</div>
<Fork />
<h1>
<span className="t-a">{cap(a)}</span> <span className="vs">vs.</span>{" "}
<span className="t-b">{cap(b)}</span>
{/* the relation is positional, and position is invisible to a screen reader */}
<span className="sr"> in place of {N.lab(row)}</span>
</h1>
</header>
{/* 2×2 so no block spans the width and the prose is never squeezed to the object above */}
<div className="head">
<Matrix words={run.words} lab={N.lab} val={score} dir={dir} row={row} pair={[a, b]}
onCell={pickCell} onRow={pickRow} />
<Ranks stats={stats} lab={N.lab} row={row} a={a} b={b} onPick={pickPair} />
<p className="pickhint">
Pick two cells in a row to see how well those 2 words substitute that row's word.
</p>
<p className="sub">
This is mutual substitutability, so both directions are averaged. So it’s the
blend of the two cells reflected along the blue diagonal.
</p>
</div>
<TriStrip t={t} N={N} sel={idx} onSel={onSel} narrow={narrow} />
{/* one sentence, written twice — the substitution shown rather than described */}
<div className="quote">
{([["a", a, r.a], ["b", b, r.b]] as const).map(([side, w, v]) => (
<div className="q-line" key={side}>
<div className="q-text">
<Glyph tri={side === "b"} />{" "}
<s className="sw-was">{N.subj(row)}</s>{" "}
<span className={"sw " + side}>{N.subj(w)}</span>{" "}
{r.sentence}
</div>
<span className={"qs-n " + side}>{v ?? "—"}</span>
</div>
))}
<div className="q-cap">
Written about {N.lab(row)}, and rated <b>{r.was ?? "—"}</b> as written.
</div>
</div>
{t.saturated && (
<p className="note warn">
Both candidates average above 90, so this comparison is at the ceiling of the scale:
everything sensible reads about the same up there and the two cannot be separated. Read
it as “both fit”, not as a measured tie. Resolving it would need a forced
choice between the two sentences rather than a rating of each.
</p>
)}
{t.floored && (
<p className="note warn">
Both candidates average below 10, so this comparison is at the floor of the scale,
the same failure as the ceiling upside down. Nothing that makes no sense can be
told apart from anything else that makes no sense. Read it as “both
nonsense”, not as a measured tie.
</p>
)}
<details className="tableview">
<summary>All {t.rows.length} sentences</summary>
<table>
<thead>
<tr>
<th>sentence about {N.lab(row)}</th>
<th>{row}</th><th className="a">{a}</th><th className="b">{b}</th>
</tr>
</thead>
<tbody>
{t.rows.map((w, i) => (
<tr key={i} className={i === idx ? "on" : ""} onClick={() => onSel(i)}>
<td>{w.sentence}</td>
<td className="num dim">{w.was ?? "—"}</td>
<td className="num a">{w.a ?? "—"}</td>
<td className="num b">{w.b ?? "—"}</td>
</tr>
))}
</tbody>
</table>
</details>
</>
)
}
type Prog = { done: number; total: number; label: string }
/** The method, not any one run: no per-run facts, no numbers from the archive. The examples are
* FIXED — `WORDS` and the vices link, never the loaded run, because the epigraphs are already
* about that subject matter. So this page takes no props. */
function Method() {
return (
<div className="method">
<h2>The problem of comparing anything</h2>
<p>
Henry Shevlin{" "}
<a href="https://x.com/dioscuri/status/2083852207758221782" target="_blank"
rel="noopener">contends</a>:
</p>
<blockquote>
Everything is like and unlike everything else, and there are no absolute judgments of
similarity, only qua-judgments. LLMs are more like humans qua language and reasoning,
chimpanzees more like humans qua perception and action.
</blockquote>
<p>
Yet also{" "}
<a href="https://x.com/dioscuri/status/2092619260895838396" target="_blank"
rel="noopener">contends</a>, with respect to gambling, porn, and video games, that
</p>
<blockquote>One of these things is not like the others.</blockquote>
<p>
The first statement expresses a ground truth: any arbitrary comparison must <i>assume something</i> to
be meaningful. The second statement can work insofar as its assumptions are accepted;
without which, it must acquiesce to the ground truth.
</p>
<p>
People judge similarity through the lens of what they care about. We have {" "}
<a href="https://en.wikipedia.org/wiki/Similarity_measure" target="_blank"
rel="noopener">formal measures</a> that make such judgements more objective, but this
objectivity itself is <i>relative</i> to those priorities.
</p>
<p>
LLMs innately provide a beautiful way of measuring the similarity between things. Well, if
they map to <i>tokens</i>. This is done geometrically, by measuring the angle
between two token embeddings, which can live in a high-dimensional space. However, for
concepts that don’t neatly map to a token, or
where we don’t have access to the weights, we need another approach. One such
approach is to use substitutability as a measure of similarity.
</p>
<h2>Substitutability as a measure of similarity</h2>
<p>To compare these words:</p>
{/* `WORDS`, so this page and the Run field cannot drift apart */}
<blockquote className="spec">{WORDS.join(", ")}</blockquote>
<p>Write sentences about each word, for example:</p>
<blockquote className="spec">A human seeks meaning in its experiences</blockquote>
<p>Then substitute for every other word in the set:</p>
<blockquote className="spec">A chimpanzee seeks meaning in its experiences</blockquote>
<p>
Coherence of each sentence is rated from 0 to 100. We can then measure the coherence
across all sentence substitutions, that gives us an overall similarity measure.
Importantly, it's not symmetric, so for example, a human can stand in for an
LLM better than an LLM can stand in for a human.
</p>
{/* the vices run itself, riding in a shortened `#tb=` link */}
<p>
The larger the set of words, the better results, but with caveats. Firstly, this
already takes up to 1000 API calls to run! But more importantly, scale alone can't solve all the problems here. Take for example,
these <a href="https://tinyurl.com/5cttcxbx" target="_blank" rel="noopener">results</a>,
which compare:
</p>
<blockquote className="spec">video games, porn, gambling, dating apps, watching sports</blockquote>
<h2>Challenges and Fundamental Limitations</h2>
<p>
Asked for sentences about porn, GPT stops producing sentences about porn per se, and
instead delivers PSAs about porn's consequences. These sentences also happen to be so general that they're far too broadly substitutable, ruining the similarity measure.
That is obviously a special case due to intervention.
</p>
<p>
The bigger problem is that <i>all</i> generated sentences too narrowly vary,
regardless of the topic. LLMs struggle with this type of creativity, and ironically, RL can make more sophisticated models <i>worse</i>. The {" "}
<a href="https://novelty-bench.github.io/" target="_blank"
rel="noopener">NoveltyBench</a> has an excellent explanation for that.
</p>
<p>
And even if we had a highly creative LLM, we still couldn't escape the fact that
human text innately biases relations. And depending on context, some relations might be more
important than others, affecting similarity judgements. Consider two people:
incredibly similar in every respect, except one is <i>fictional</i>. Now compare their
moral value! Obviously incredibly different. How would we weight
{" "}“existence” itself? There’s no universal criterion.
</p>
<p>
Which brings us full circle:
</p>
<blockquote>Everything is like and unlike everything else, and
there are no absolute judgments of similarity, only qua-judgments.
</blockquote>
</div>
)
}
/* Transport icons — the Material paths every bulb here uses, filled with `currentColor` so the
* button's own colour carries the state and the mark never needs a second one. */
const PLAY = "M8 5v14l11-7z"
const STOP = "M6 6h12v12H6z"
const Icon = ({ d }: { d: string }) => (
<svg viewBox="0 0 24 24" fill="currentColor" aria-hidden="true"><path d={d} /></svg>
)
function Generate({ models, access, busy, prog, err, draft, onDraft, onRun, onStop,
hasRun, onDownload, onUpload, onSave }: {
models: TbModel[]; access: AiAccess
busy: boolean; prog: Prog; err: string
draft: Draft; onDraft: (d: Draft) => void
onRun: (m: TbModel, n: number, words: string[]) => void
onStop: () => void
hasRun: boolean
onDownload: () => void; onUpload: () => void; onSave: () => void
}) {
/* held by `App`, not here — see the `draft` comment there */
const { words: wordText, n, model: pick } = draft
const setWordText = (v: string) => onDraft({ ...draft, words: v })
const setN = (v: number) => onDraft({ ...draft, n: v })
const setPick = (v: number) => onDraft({ ...draft, model: v })
/* Exactly five: cost is quadratic (`words + words² × n`) and the whole page is built for a
* five-square grid. Spaces are allowed — `language model` against `LLM` is worth running.
* Nothing is dropped silently; a bad word disables the run rather than vanishing from it. */
const N_WORDS = 5, MAX_LEN = 20
const SHAPE = /^[A-Za-z][A-Za-z0-9'’ -]*$/
const parsed = useMemo(() => {
const seen = new Set<string>()
const words: string[] = []
const issues: string[] = []
let dupes = 0
for (const raw of wordText.split(",")) {
const w = raw.trim().replace(/\s+/g, " ")
if (!w) continue // trailing comma while typing is not an error
const k = w.toLowerCase()
if (seen.has(k)) { dupes++; continue }
seen.add(k)
if (w.length > MAX_LEN)
issues.push(`“${w}” is too long — ${MAX_LEN} characters at most.`)
else if (w.length < 2)
issues.push(`“${w}” is too short — two letters at least.`)
else if (!SHAPE.test(w))
issues.push(`“${w}” — letters, digits, spaces and hyphens only, starting with a letter.`)
else words.push(w)
}
if (dupes) issues.push(`${dupes} repeated word${dupes > 1 ? "s" : ""} ignored.`)
if (!issues.length && words.length !== N_WORDS)
issues.push(`Five words exactly — there ${words.length === 1 ? "is" : "are"} ${words.length}.`)
return { words, issues }
}, [wordText])
const words = parsed.words
const ready = parsed.issues.length === 0 // a wrong count is itself an issue
useEffect(() => {
if (pick >= 0 || !models.length) return
setPick(defaultModel(models))
}, [models, pick])
/* the leading 1 is `resolveForms` — the call that decides what the other 380 will say */
const calls = 1 + words.length + words.length * words.length * n
/* the one blocking state on the page, so it is coloured like one. Every other `.warn` is a
caveat you read past; this one means nothing below it can run. */
if (access === "none") return (
<p className="warn noai">
No AI access. This bulb needs <code>--trust</code> and an <code>OPENROUTER_API_KEY</code>
{" "}(or another provider key) in <code>.env</code>.
</p>
)
return (
<div className="setup">
<section className="fld">
<h2>Model</h2>
<select value={pick} aria-label="model"
onChange={(e) => setPick(+e.target.value)}>
{models.map((m, i) => (
<option key={i} value={i}>
{m.providerName ? `${m.providerName}: ${m.friendlyName}` : m.friendlyName}
</option>
))}
</select>
<p className="hint">
The model that both generates and judges sentences for coherence. Every judgement
is made independently.
</p>
</section>
<section className="fld">
<h2>Corpus</h2>
{/* fixed choices, not a number field: typing into a spinner means fighting the
OS over selection and increment, for a value that only ever takes a few sane sizes */}
<div className="choices" role="group" aria-label="sentences per word">
{[5, 10, 20, 40].map((v) => (
<button key={v} type="button" className={"choice" + (n === v ? " on" : "")}
aria-pressed={n === v} onClick={() => setN(v)}>
{v}
</button>
))}
<span className="choices-l">sentences per word</span>
</div>
<p className="hint">
More sentences give a more predictable result.
</p>
</section>
<section className="fld">
<h2>Words</h2>
<input className="wordbox" value={wordText} aria-label="words, comma separated"
spellCheck={false} onChange={(e) => setWordText(e.target.value)} />
<p className="hint">
The 5 words to compare.
</p>
{parsed.issues.length > 0 && (
<div className="note warn">
{parsed.issues.map((m, i) => <div key={i}>{m}</div>)}
</div>
)}
</section>
{/* One button, two states, and one line under it: the price before, the progress during.
Both answer "what is this going to do", so they never both speak at once. */}
<div className="run">
<div className="action">
<button className={"go" + (busy ? " stop" : "")}
disabled={busy ? false : pick < 0 || !ready || access !== "own"}
aria-label={busy ? "Stop" : "Run"} title={busy ? "Stop" : "Run"}
onClick={() => (busy ? onStop() : onRun(models[pick], n, words))}>
<Icon d={busy ? STOP : PLAY} />
</button>
{/* what Download writes is what Upload reads; Save makes that run the one the bulb
opens on */}
<span className="files">
<button className="choice" onClick={onDownload} disabled={!hasRun}>Download</button>
<button className="choice" onClick={onUpload}>Upload</button>
{HAS_SOURCE &&
<button className="choice" onClick={onSave} disabled={!hasRun}>Save</button>}
</span>
</div>
{busy ? (
<div className="prog">
<div className="prog-track">
<div className="prog-fill"
style={{ width: `${prog.total ? (prog.done / prog.total) * 100 : 0}%` }} />
</div>
<p className="prog-lab">{prog.label} {prog.total ? `${prog.done}/${prog.total}` : ""}</p>
</div>
) : (
<p className="prog-lab">Will make {calls} calls</p>
)}
{/* what just happened, in the place that says what is about to: a run that failed, a
file that would not read, a Save the host refused. All three are answers to a button
in the row above, so none of them belongs at the top of the page. */}
{err && <p className="note warn">{err}</p>}
{/* gated, not just warned: even 131 calls is far past the free quota, so the run would
stall partway and be discarded */}
{access === "courtesy" && (
<p className="hint">
Courtesy model only: {calls} calls is far past its free quota, so the run is
disabled. To run it, use your own keys — locally with the <b>typebulb CLI</b>,
or sign in at <b>typebulb.com</b> and add your keys.
</p>
)}
</div>
</div>
)
}
// ── app ──────────────────────────────────────────────────────────────────────
/* A RUN TRAVELS IN THE LINK. `tb.setData` deflates it into the URL's `#tb=` fragment, and a page
* opening that address decodes it before this bulb's first synchronous read — so `baked()`
* already returns the shared run and there is no load path. `data.txt` is never touched. */
/* A `#tb=` link is someone sending you THIS comparison, so `latest.json` — newer than almost any
* link — must not overrule it. Read once at load: finishing a run sets the fragment too. */
const OPENED_SHARED = typeof location !== "undefined" && /(^|[#&])tb=/.test(location.hash)
/* Filing a run into `data.txt` wants the source file on disk and a host that will write to it —
* the CLI, and nothing else. A published bulb IS the app and not the file behind it, an inline
* frame is a sandbox with no file at all, and off the CLI `tb.infer()` answers "script has no
* inference instructions". Download lands a results file in every mode; Save appears only where
* it lands in the bulb. */
const HAS_SOURCE = tb.mode === "local"
/* A tab is a place to go, so Download, Upload and Save are not tabs: they sit beside the Run
* button, on the page where a run is made. */
type View = "read" | "make" | "about"
const VIEWS: { id: View; label: string }[] = [
{ id: "read", label: "Results" }, { id: "make", label: "Run" },
{ id: "about", label: "About" },
]
/** What the Run form holds between visits — a draft of a run, which is why it is not a run. */
type Draft = { words: string; n: number; model: number }
function App() {
const first = useMemo(baked, []) // parsing the data block once, not on every render
const [run, setRun] = useState<Run | null>(first)
const [view, setView] = useState<View>(first ? "read" : "make")
const [models, setModels] = useState<TbModel[]>([])
const [access, setAccess] = useState<AiAccess>("own")
const [busy, setBusy] = useState(false)
const [prog, setProg] = useState<Prog>({ done: 0, total: 0, label: "" })
const [err, setErr] = useState("")
/* the views are a conditional, not a router, so `Generate` unmounts on leaving Run; the draft
* is held up here to survive that */
const [draft, setDraft] = useState<Draft>({
words: (first?.words ?? WORDS).join(", "), n: 10, model: -1,
})
/* keyed on the word list and not the run object, so finishing a run of the same five does not
* overwrite an edit started since */
const runWords = run ? run.words.join(", ") : ""
useEffect(() => {
if (runWords) setDraft((d) => (d.words === runWords ? d : { ...d, words: runWords }))
}, [runWords])
const abort = useRef(false)
const busyRef = useRef(false)
useEffect(() => {
tb.aiAccess().then(setAccess)
tb.models().then(setModels)
}, [])
/* `latest.json` is consulted ALWAYS and the NEWER of it and the bake wins, so whichever was
* measured last is what you see. A `#tb=` run is outside that contest; see OPENED_SHARED. */
useEffect(() => {
if (OPENED_SHARED) {
tb.log(`[data] opened a shared run (${first?.timestamp ?? "unreadable"}) — ` +
`the #tb= link wins; latest.json not consulted`)
return
}
tb.fs.read("latest.json")
.then((t) => {
const d = normalize(JSON.parse(t))
if (!d) return
if (first && first.timestamp >= d.timestamp) {
tb.log(`[data] keeping the baked run (${first.timestamp}) — newer than latest.json`)
return
}
setRun(d); setView("read")
tb.log(`[data] loaded latest.json (${d.timestamp}, n=${d.n}) — newer than the bake`)
})
.catch(() => {})
}, [])
/* The forms are the FIRST thing bought, before any sentence exists, because the whole corpus
* is written to them: a bad set here is not a blemish on the run, it is the run. So one call,
* failing loudly rather than proceeding into hundreds of sentences nobody can quote. */
async function go(m: TbModel, n: number, words: string[]) {
busyRef.current = true
abort.current = false
setBusy(true); setErr(""); setView("make")
try {
setProg({ done: 0, total: 1, label: "working out how to say the words…" })
const { forms, frame } = await resolveForms(m, words)
if (abort.current) throw new Error("stopped")
const N = naming(forms)
const corpus: Record<string, string[]> = {}
setProg({ done: 0, total: words.length, label: "writing the sentences…" })
// one writer per word — the passes inside a word are sequential, the words are not
await pool(words.map((a) => async () => { corpus[a] = await generate(m, a, n, N) }),
CONC, () => setProg((p) => ({ ...p, done: p.done + 1 })), () => abort.current)
if (abort.current) throw new Error("stopped")
// one job per unique (word, sentence) — every column of a row shares the diagonal's scores
const seen = new Map<string, { word: string; sentence: string }>()
for (const anchor of words)
for (const word of words)
for (const s of corpus[anchor] ?? []) seen.set(`${word} ${s}`, { word, sentence: s })
const scored = new Map<string, number>()
const jobs = [...seen.entries()].map(([k, j]) => async () => {
const v = await rate(m, N.say(j.word, j.sentence))
if (v != null) scored.set(k, v)
})
let done = 0
setProg({ done: 0, total: jobs.length, label: `rating ${jobs.length} sentences…` })
await pool(jobs, CONC, () => setProg((p) => ({ ...p, done: ++done })), () => abort.current)
if (abort.current) throw new Error("stopped")
const results: Cell[] = []
for (const anchor of words)
for (const subj of words)
results.push({
anchor, subject: subj,
scores: (corpus[anchor] ?? []).map((s) => scored.get(`${subj} ${s}`) ?? null),
})
const out: Run = {
timestamp: new Date().toISOString(), judge: m.name, provider: m.provider,
n, words, forms, frame, corpus, results,
}
setRun(out); setView("read"); setProg({ done: 0, total: 0, label: "" })
const stamp = out.timestamp.replace(/[:.]/g, "-")
const body = toJson(out)
/* runtime data unlocks STATE ▸ Save to Tabs on typebulb.com, the only route to filing a run
* from a published copy. It also leaves the `#tb=` fragment OPENED_SHARED honours. */
try {
await tb.setData(body)
tb.log(`[run] runtime data set — STATE ▸ Save to Tabs promotes it to data.txt on .com`)
} catch (e) {
tb.log(`[run] could not set runtime data: ${String(e)}`)
}
try {
await tb.fs.write(`matrix-${stamp}.json`, body)
await tb.fs.write("latest.json", body) // what the page reloads from
tb.log(`[run] wrote matrix-${stamp}.json (n=${n}, ${results.length} cells, judge=${m.name})`)
} catch (e) {
tb.log(`[run] could not write results: ${String(e)}`)
}
} catch (e) {
setErr(String(e))
tb.log(`[run] error ${String(e)}`)
} finally {
busyRef.current = false
setBusy(false)
}
}
/* Terminal channel. `send … selftest --wait` prints what the read view is drawing, so a run
* can be checked without a screenshot; `send … make` / `read` switch views. Re-registered
* when the run changes so the handler reads the current one; `busy` arrives via a ref so a
* mid-run selftest answers truthfully without re-registering every progress tick. */
useEffect(() => tb.onMessage((msg: unknown) => {
if (VIEWS.some((v) => v.id === msg)) { setView(msg as View); return `view: ${msg}` }
/* `send … "forms: porn, video games, gambling"` — ONE call, the same one a run opens with, so
* a word set can be checked for a penny before 256 are spent on it, and every sentence
* afterwards is written to whatever comes back. `… | openrouter` picks the model, matched
* against provider and name, since the same id is often reachable through more than one key
* and only one of them may be funded. */
if (typeof msg === "string" && msg.startsWith("forms:")) {
const [list, want] = msg.slice(6).split("|")
const words = list.split(",").map((w) => w.trim()).filter(Boolean)
if (!words.length) return { ok: false, reason: "no words given" }
// every token has to appear somewhere in "<provider> <name>", so "openrouter luna" hits
// `openrouter openai/gpt-5.6-luna` without anyone having to spell the id out
const hunt = want?.trim().toLowerCase().split(/\s+/).filter(Boolean) ?? []
const m = hunt
.length
? models.find((x) => {
const hay = `${x.provider} ${x.name}`.toLowerCase()
return hunt.every((t) => hay.includes(t))
})
: models[defaultModel(models)]
if (!m) return { ok: false, reason: hunt.length
? `no model matching ${hunt.join(" ")}` : "no models loaded yet" }
return resolveForms(m, words)
.then((r) => ({ ok: true, provider: m.provider, judge: m.name, ...r }))
.catch((e) => ({ ok: false, reason: String(e) }))
}
/* `send … "corpus: human, LLM, chimpanzee, frog, virus | 10"` — the generation half of a run,
* so the sentences can be READ before 375 ratings are bought on them. A defect in the corpus
* is invisible in the numbers afterwards and fatal to them.
*
* PASS THE WHOLE SET even though only the first word's sentences come back: the frame is a
* property of the set, and one word alone always resolves to a singular. */
if (typeof msg === "string" && msg.startsWith("corpus:")) {
const [list, howMany] = msg.slice(7).split("|")
const words = list.split(",").map((w) => w.trim()).filter(Boolean)
const n = Math.max(2, Math.min(20, parseInt(howMany ?? "6", 10) || 6))
const m = models[defaultModel(models)]
if (!m || !words.length) return { ok: false, reason: "need a word and a loaded model" }
return resolveForms(m, words)
.then(async ({ forms: f, frame }) => {
const N = naming(f)
const w = words[0]
const corpus = (await generate(m, w, n, N)).map((s) => N.say(w, s))
return { ok: true, judge: m.name, frame, forms: f, of: w, corpus }
})
.catch((e) => ({ ok: false, reason: String(e) }))
}
if (msg !== "selftest") return
if (!run) return { ok: false, reason: "no run loaded" }
const pairs = pairStats(run).map((s) => ({
pair: s.pair.key, score: +s.score.toFixed(1), n: s.ok.length,
xy: +s.xy.toFixed(1), yx: +s.yx.toFixed(1), asym: +s.asym.toFixed(1),
saturated: s.saturated, floored: s.floored,
}))
return { ok: true, busy: busyRef.current, judge: run.judge, n: run.n,
when: run.timestamp, words: run.words, pairs }
}), [run, models])
const download = () => {
if (!run) return
const blob = new Blob([toJson(run)], { type: "application/json" })
const a = document.createElement("a")
a.href = URL.createObjectURL(blob)
/* the same name the run writer puts on disk */
a.download = `matrix-${run.timestamp.replace(/[:.]/g, "-")}.json`
a.click()
URL.revokeObjectURL(a.href)
}
/* Filing a run into `data.txt` is the one thing no `tb.*` call does by itself: `tb.setData`
* swaps it in for the life of the page, and `tb.infer()` raises the host panel whose **Save to
* bulb** writes it. No model is called. Closing the panel is the ordinary ending. */
const toSource = async () => {
if (!run) return
try {
await tb.setData(toJson(run))
await tb.infer()
} catch (e) { setErr(`could not offer it to the bulb: ${String(e)}`) }
}
const upload = () => {
const input = document.createElement("input")
input.type = "file"; input.accept = ".json"
input.onchange = async () => {
const f = input.files?.[0]
if (!f) return
try {
const d = normalize(JSON.parse(await f.text()))
if (!d) throw new Error("no results in that file")
setRun(d); setView("read"); setErr("")
} catch (e) { setErr(`could not read that file: ${String(e)}`) }
}
input.click()
}
return (
<div className="wrap viz-root">
{/* spelled exactly as the frontmatter `name:` — the one place the bulb names itself */}
<div className="masthead">
<div className="brand">
<div className="brand-n">Chimp vs Chatbot</div>
<div className="brand-s">Compare any 5 things for similarity.</div>
</div>
<div className="bar">
{VIEWS.map((v) => (
/* the message under the buttons belongs to the action that raised it, not to the
page — leaving the view is done with it */
<button key={v.id} className={"tab" + (view === v.id ? " on" : "")}
disabled={v.id === "read" && !run}
onClick={() => { setErr(""); setView(v.id) }}>{v.label}</button>
))}
</div>
</div>
{/* keyed on the word list: `Read` holds its selection in `useState` initializers, which run
once per mount, so different words must remount. The same words keep the selection. */}
{view === "read" && run && <Read run={run} key={run.words.join("|")} />}
{view === "read" && !run && (
<p className="note">
No run here yet. Switch to <b>Run</b>, upload a results file, or open a link someone
shared — a run travels whole in the <code>#tb=</code> fragment.
</p>
)}
{view === "make" && (
models.length
? <Generate models={models} access={access} busy={busy} prog={prog} err={err}
onRun={go} draft={draft} onDraft={setDraft}
onStop={() => { abort.current = true }}
hasRun={!!run} onDownload={download} onUpload={upload} onSave={toSource} />
: <p className="note">Loading models…</p>
)}
{view === "about" && <Method />}
</div>
)
}
createRoot(document.getElementById("root")!).render(<App />)
```
**styles.css**
```css
.viz-root {
color-scheme: light;
--cell: 34px;
--ink: #0b0b0b; --ink-2: #45443f; --muted: #6f6e68;
--border: rgba(11,11,11,.12); --panel: rgba(11,11,11,.04);
--a: #7b3fd4; --b: #eb6834; --surface: #fcfcfb;
}
@media (prefers-color-scheme: dark) {
:root:where(:not([data-theme="light"])) .viz-root {
color-scheme: dark;
--ink:#fff; --ink-2:#d8d7cd; --muted:#9d9c93;
--border:rgba(255,255,255,.14); --panel:rgba(255,255,255,.05);
--a:#9d78ea; --b:#d95926; --surface:#1a1a19;
}
}
:root[data-theme="dark"] .viz-root {
color-scheme: dark;
--ink:#fff; --ink-2:#d8d7cd; --muted:#9d9c93;
--border:rgba(255,255,255,.14); --panel:rgba(255,255,255,.05);
--a:#9d78ea; --b:#d95926; --surface:#1a1a19;
}
/* the charts earn the extra width; each text block caps its own measure */
.wrap { max-width: 1080px; margin: 0 auto; padding: 20px 22px 60px;
font: 17px/1.55 system-ui, -apple-system, "Segoe UI", sans-serif; color: var(--ink); }
h1 { font-size: 31px; margin: 0; letter-spacing: -.025em; text-align: center; }
.sub { margin: 0; color: var(--muted); font-size: 16px; }
.t-a { color: var(--a); } .t-b { color: var(--b); }
.vs { font-weight: 400; color: var(--muted); }
/* anchor above, candidates below, fork between: nothing is wider than the fork, so the block
centres as one object */
.triad { display: grid; justify-items: center; }
.anchor { font-size: 16.5px; font-weight: 500; letter-spacing: -.01em; color: var(--muted); }
.fork { display: block; width: 140px; height: 30px; margin: 1px 0 -1px; }
.fk-a { stroke: var(--a); } .fk-b { stroke: var(--b); }
/* for the reader who cannot see the fork: the sentence the layout draws */
.sr { position: absolute; width: 1px; height: 1px; overflow: hidden; clip-path: inset(50%);
white-space: nowrap; }
.tagline { font-size: 25px; margin: 0 0 12px; letter-spacing: -.02em;
font-weight: 500; color: var(--ink-2); text-align: center; }
.tagline b { text-decoration: underline; text-underline-offset: 3px; }
/* the candidate glyph, sized to whatever text it sits in */
.glyph { display: inline-block; width: .78em; height: .78em; vertical-align: baseline; }
.glyph.a { fill: var(--a); } .glyph.b { fill: var(--b); }
/* equal columns rather than sized to the objects: a column only as wide as the matrix squeezes
the paragraph under it to 250px */
.head { display: grid; grid-template-columns: minmax(0,1fr) minmax(0,1fr);
gap: 18px 34px; align-items: start; margin-top: 22px; }
@media (max-width: 760px) {
/* stacked, each picker still followed by its own text rather than by the other picker */
.head { grid-template-columns: minmax(0,1fr); }
.head > .mx-wrap { order: 1 }
.head > .pickhint { order: 2 }
/* on its own row it is a 290px block in a full-width column, so it centres like the matrix
above it rather than hugging the left edge */
.head > .ranks { order: 3; justify-self: center; }
.head > .sub { order: 4 }
}
/* capped rather than filling the column: stretched wide, the number drifts away from its name */
.ranks { display: grid; gap: 1px; align-content: start; max-width: 290px; }
.ranks-h { font-size: 15.5px; font-weight: 640; letter-spacing: -.015em; color: var(--ink-2);
text-align: center; padding: 0 6px 8px; }
.rank { display: grid; grid-template-columns: minmax(0,1fr) auto; gap: 10px;
font: inherit; font-size: 13px; text-align: left; padding: 3px 6px; cursor: pointer;
border: 0; border-left: 3px solid transparent; border-radius: 3px;
background: transparent; color: var(--muted); white-space: nowrap; }
.rank:hover { background: var(--panel); color: var(--ink-2); }
.rank.a { color: var(--a); border-left-color: var(--a); font-weight: 640; }
.rank.b { color: var(--b); border-left-color: var(--b); font-weight: 640; }
.rank-v { font-variant-numeric: tabular-nums; }
/* chrome — one underlined row, thicker under the selected tab -------------- */
/* the rule belongs to the masthead, not the tab row, so a selected tab's underline meets it —
hence `.bar`'s -1px pull */
.masthead { display: flex; align-items: flex-end; justify-content: space-between;
flex-wrap: wrap; gap: 8px 28px; margin-bottom: 20px;
border-bottom: 1px solid var(--border); }
.brand { padding-bottom: 8px; }
.brand-n { font-size: 18px; font-weight: 600; letter-spacing: -.02em; }
.brand-s { font-size: 14px; color: var(--muted); letter-spacing: -.005em; margin-top: 1px; }
.bar { display: flex; align-items: flex-end; flex-wrap: wrap; gap: 22px; }
.tab { font: inherit; font-size: 15px; padding: 6px 2px 9px; cursor: pointer;
background: transparent; color: var(--muted);
border: 0; border-bottom: 3px solid transparent; margin-bottom: -1px; }
.tab:hover:not(:disabled) { color: var(--ink); }
.tab.on { color: var(--ink); font-weight: 600; border-bottom-color: var(--ink); }
.tab:disabled { opacity: .4; cursor: default; }
.prog { margin: 0; }
.prog-track { height: 6px; border-radius: 3px; background: var(--panel); overflow: hidden; }
.prog-fill { height: 100%; background: var(--b); border-radius: 3px; transition: width .2s; }
.prog-lab { margin: 6px 0 0; font-size: 14.5px; color: var(--muted);
font-variant-numeric: tabular-nums; }
/* the price stands where the progress label will, so pressing play swaps one line for another
in place; the 6px belongs to the bar above it, which this one does not have */
.run > .prog-lab { margin: 0; }
/* a row of the run grid, which spaces its own rows — the note's own top margin would double it */
.run > .note { margin: 0; }
/* matrix — the picker. A ROW is every pair one word belongs to, and two cells in it are the two
candidates. The diagonal is drawn but never selectable. ---------------------------------- */
.mx { display: grid; gap: 2px; }
.mx-wrap { width: max-content; margin: 0 auto; }
.colLabel { writing-mode: vertical-rl; transform: rotate(180deg);
font-size: 13px; color: var(--muted); justify-self: center;
/* the rotation flips physical sides, so the gap under the label is padding-TOP */
max-height: 104px; white-space: nowrap; padding-top: 6px; }
.rowLabel { font: inherit; font-size: 13px; text-align: right; white-space: nowrap;
padding: 0 9px 0 0; border: 0; background: transparent; color: var(--muted);
cursor: pointer; }
.rowLabel:hover { color: var(--ink); }
.rowLabel.on { color: var(--ink); font-weight: 640; }
.cell { width: var(--cell); height: var(--cell); border-radius: 4px; border: 0; padding: 0;
cursor: pointer; }
/* a real measurement, but never picked: a word is not a candidate to replace itself */
.cell.diag { cursor: default; }
/* the selected pair is marked by its SHAPE, not an outline: an outline only says "selected",
while the shape also says which of the two candidates this square is */
.cmark { display: block; width: 100%; height: 100%;
fill: var(--surface); stroke-width: 2.2; stroke-linejoin: round; }
.cmark.a { stroke: var(--a); } .cmark.b { stroke: var(--b); }
.cell:focus-visible { outline: 2px solid var(--ink); outline-offset: 2px; }
.cell.none { background: transparent; cursor: default; border: 1px dashed var(--border); }
.legend { display: grid; gap: 5px; margin-top: 12px; }
/* width 0 + min-width 100%: the svg's intrinsic (viewBox) width must not size the max-content
matrix wrap — the matrix sizes the column, the bar stretches to it */
.lgBar { display: block; width: 0; min-width: 100%; height: 12px; }
.lgEnds { display: flex; justify-content: space-between; gap: 12px;
font-size: 13px; color: var(--muted); }
/* no max-width: the grid cell is the measure */
.pickhint { margin: 0; font-size: 15px; color: var(--muted); line-height: 1.6; }
/* strip ------------------------------------------------------------------- */
.strip-wrap { margin: 20px 0 24px; }
/* no min-width: below 640px the chart is re-laid-out rather than scrolled sideways */
.strip { width: 100%; height: auto; display: block; }
.ends { display: flex; justify-content: space-between; flex-wrap: wrap; gap: 4px 20px;
font-size: 16px; font-weight: 600; letter-spacing: -.01em; color: var(--muted);
margin-top: 0; }
/* the poles are values, the centre is the axis's name — weight makes the hierarchy */
.ends .mid, .lgEnds .mid { font-weight: 400; }
/* both gradient bars — the matrix legend and the strip's axis — are this one rect */
.axisbar { color: var(--ink); stroke: rgba(128,128,128,.25); }
.axis { stroke: var(--border); stroke-width: 1.5; }
.tick { fill: var(--muted); font-family: system-ui, sans-serif;
font-variant-numeric: tabular-nums; }
/* the summary of the whole page, so it is sized like one */
.meanlab { fill: var(--a); font-weight: 660; letter-spacing: -.02em;
font-family: system-ui, sans-serif; font-variant-numeric: tabular-nums; }
.meanlab.b { fill: var(--b); }
/* a position marker, not a mark of its own — it drops from the label to the axis and the dots
are drawn over it */
.meanline { stroke: var(--a); stroke-width: 2; opacity: .45; }
.meanline.b { stroke: var(--b); }
/* hue AND shape both say which candidate — position already says the score, so colour is free
to do the job the geometry cannot */
.dot { cursor: pointer; stroke: var(--surface); stroke-width: 2.5; opacity: .55; }
.dot.a { fill: var(--a); } .dot.b { fill: var(--b); }
.dot:hover { opacity: .85; }
.dot.on { opacity: 1; stroke: var(--ink); stroke-width: 3; }
/* quote ------------------------------------------------------------------- */
.quote { padding: 20px 22px; border-radius: 12px;
border: 1px solid var(--border); background: var(--panel); }
.q-text { font-size: 16.5px; line-height: 1.5; }
/* the triad's two readings of one sentence, stacked so the swap is the only visible difference */
.q-line { display: grid; grid-template-columns: minmax(0,1fr) 46px; gap: 14px;
align-items: baseline; }
.q-line + .q-line { margin-top: 10px; }
/* the substitution, shown rather than explained: what was replaced, struck, then what replaced it */
.sw { font-weight: 640; color: var(--ink); }
.sw.a { color: var(--a); } .sw.b { color: var(--b); }
.sw-was { color: var(--muted); font-weight: 400; text-decoration-thickness: 2px; }
.qs-n { font-size: 17px; font-weight: 640; font-variant-numeric: tabular-nums;
text-align: right; }
.qs-n.a { color: var(--a); } .qs-n.b { color: var(--b); }
.q-cap { margin-top: 14px; padding-top: 12px; border-top: 1px solid var(--border);
font-size: 14.5px; color: var(--muted); line-height: 1.55; }
/* table view -------------------------------------------------------------- */
.tableview { margin-top: 22px; font-size: 15px; }
.tableview summary { cursor: pointer; color: var(--muted); }
.tableview table { display: block; overflow-x: auto; width: 100%; border-collapse: collapse;
margin-top: 12px; }
.tableview th, .tableview td { text-align: left; padding: 5px 8px; vertical-align: top;
border-bottom: 1px solid var(--border); }
.tableview th { font-weight: 600; font-size: 13px; letter-spacing: .03em; color: var(--muted); }
.tableview tbody tr { cursor: pointer; }
.tableview tbody tr:hover, .tableview tbody tr.on { background: var(--panel); }
.num { text-align: right; font-variant-numeric: tabular-nums; width: 70px; }
.num.dim { color: var(--muted); }
/* the two candidate columns carry the same two colours as the fork, the dots and the quote, so a
column and the thing it is about are never matched up by position alone */
.num.a { color: var(--a); } .num.b { color: var(--b); }
.tableview th.a { color: var(--a); } .tableview th.b { color: var(--b); }
/* setup ------------------------------------------------------------------- */
.setup { display: grid; gap: 26px; }
.fld h2 { font-size: 17px; font-weight: 630; margin: 0 0 8px; letter-spacing: -.015em; }
input, select { font: inherit; font-size: 15px; padding: 5px 8px; border-radius: 6px;
border: 1px solid var(--border); background: Canvas; color: CanvasText; }
/* in `ch` because the content is words, plus a flat 100px so five 20-character ones fit */
.wordbox { width: 100%; max-width: calc(46ch + 100px); margin-bottom: 10px; }
option { background: Canvas; color: CanvasText; }
/* a short row of sane sizes — the whole range this dial is ever set to */
.choices { display: flex; align-items: center; flex-wrap: wrap; gap: 8px; }
.choice { font: inherit; font-size: 15px; font-variant-numeric: tabular-nums;
min-width: 46px; padding: 6px 12px; border-radius: 7px; cursor: pointer;
border: 1px solid var(--border); background: transparent; color: var(--ink-2); }
.choice:hover { color: var(--ink); border-color: var(--muted); }
.choice.on { background: var(--ink); border-color: var(--ink); color: var(--surface);
font-weight: 640; }
.choices-l { margin-left: 6px; font-size: 15px; color: var(--ink-2); }
.hint { margin: 10px 0 0; font-size: 14.5px; color: var(--muted); line-height: 1.55; }
/* tighter than the gap between the fields above, so the line under the button reads as that
button's second line and not as another section of the form */
.run { display: grid; gap: 13px; }
.action { display: flex; align-items: center; gap: 14px; flex-wrap: wrap; }
/* plain `.choice`, so every button on the page is one height — quieter than Run in colour and
weight, never in size */
.files { display: flex; flex-wrap: wrap; gap: 8px; margin-left: auto; }
/* once the row has stacked, `margin-left: auto` would strand the three against the right edge
under a left-aligned Run — dropped, so the whole band reads as one left-aligned column */
@media (max-width: 560px) {
.files { margin-left: 0; }
}
/* type and padding are `.choice`'s, and the svg is sized in `em` off that same line box, so play
stands exactly as tall as every other button without a pixel being named */
.go { font: inherit; font-size: 15px; padding: 6px 12px; border-radius: 7px; cursor: pointer;
display: inline-flex; align-items: center;
border: 1px solid var(--ink-2); background: transparent; color: var(--ink); }
.go svg { display: block; width: 1.55em; height: 1.55em; }
.go:disabled { opacity: .45; cursor: default; }
/* the same button saying the other thing it can do, in the hue the progress bar under it fills
with — pressing it and watching it are one colour */
.go.stop { border-color: var(--b); color: var(--b); }
.note { font-size: 16px; color: var(--ink-2); line-height: 1.6; margin: 16px 0 0;
max-width: 88ch; }
/* prose measure, not caption measure. Three registers: prose, quoted material, `.note dim`. */
.method { max-width: 72ch; font-size: 16.5px; line-height: 1.65; color: var(--ink-2); }
.method h2 { font-size: 21px; font-weight: 640; letter-spacing: -.02em; color: var(--ink);
margin: 30px 0 6px; }
.method h2:first-child { margin-top: 4px; }
.method p { margin: 12px 0; }
.method b { color: var(--ink); }
.method blockquote { margin: 14px 0; padding: 2px 0 2px 16px;
border-left: 3px solid var(--border); color: var(--muted); font-style: italic; }
/* the recipe's own material keeps the blockquote's italic rather than cancelling it; its one
departure is the page's ink, since it is the specimen to be read word for word rather than
something somebody said. Quoted speech stays muted, so the two epigraphs read as quotes. */
.method .spec { color: var(--ink-2); }
/* consecutive notes are one block, not two separate interruptions */
.note + .note { margin-top: 4px; }
.note.dim { color: var(--muted); font-size: 15px; }
.note.warn, .warn { color: var(--ink); border-left: 3px solid var(--b); padding-left: 12px; }
/* the one warning that is a dead end rather than a caveat — no key, nothing on the page can run */
.warn.noai { color: var(--b); }
code { font-size: 14.5px; background: var(--panel); padding: 2px 6px; border-radius: 4px; }
```
**index.html**
```html
<div id="root"></div>
```
**data.txt**
```txt
{
"timestamp": "2026-08-27T15:18:57.700Z",
"judge": "openai/gpt-5.6-luna",
"provider": "openrouter",
"n": 40,
"words": [
"human",
"LLM",
"chimpanzee",
"frog",
"virus"
],
"forms": {
"human": {
"subj": "Humans",
"lab": "humans"
},
"LLM": {
"subj": "LLMs",
"lab": "LLMs"
},
"chimpanzee": {
"subj": "Chimpanzees",
"lab": "chimpanzees"
},
"frog": {
"subj": "Frogs",
"lab": "frogs"
},
"virus": {
"subj": "Viruses",
"lab": "viruses"
}
},
"frame": "PLURAL — these are countable kinds of entities best compared as groups",
"corpus": {
"virus": [
"infect living cells by attaching to specific receptors on cellular surfaces.",
"carry genetic instructions encoded in DNA, RNA, or sometimes both.",
"replicate by redirecting host cellular machinery toward producing viral components.",
"mutate when copying errors alter their genetic sequences during replication.",
"evolve rapidly when natural selection favors variants with transmission advantages.",
"spread between hosts through respiratory droplets, contaminated surfaces, bodily fluids, or vectors.",
"remain metabolically inactive outside suitable host cells.",
"depend on host cells for energy, protein synthesis, and reproduction.",
"vary greatly in shape, size, genetic material, and structural complexity.",
"often possess protein coats called capsids that protect their genomes.",
"sometimes acquire lipid envelopes from host cell membranes during release.",
"recognize host cells through molecular interactions between surface proteins and receptors.",
"can infect animals, plants, fungi, bacteria, and other microorganisms.",
"cause diseases ranging from mild respiratory infections to severe systemic illnesses.",
"may produce symptoms by damaging cells or triggering excessive immune responses.",
"can remain hidden inside host cells before becoming actively infectious.",
"sometimes integrate their genetic material into host chromosomes permanently.",
"can move between species when genetic compatibility and ecological contact permit.",
"occasionally exchange genetic segments when different strains infect one cell.",
"may recombine genetic material, creating variants with new biological properties.",
"use diverse strategies to evade antibodies, immune cells, and antiviral defenses.",
"can stimulate lasting immune memory after infection or vaccination.",
"are targeted by vaccines that train immune systems before exposure.",
"are treated with antiviral drugs that interrupt specific replication processes.",
"cannot usually be eliminated by antibiotics, which target cellular organisms.",
"can become resistant to medicines when mutations reduce drug effectiveness.",
"spread more efficiently when crowded conditions increase contact between susceptible hosts.",
"persist in populations when immunity declines or new variants emerge.",
"influence ecosystems by regulating microbial abundance and transferring genetic material.",
"infect ocean microorganisms and contribute substantially to global nutrient cycling.",
"can kill bacterial populations through infections known as bacteriophage activity.",
"help researchers study gene regulation, cellular processes, and molecular evolution.",
"have been adapted as delivery vehicles for therapeutic genes in medicine.",
"can trigger economic losses by damaging crops, livestock, fisheries, and workplaces.",
"sometimes cause lifelong infections when immune responses cannot fully clear them.",
"may spread before symptoms appear, complicating detection and prevention efforts.",
"differ in stability, with some surviving longer under particular environmental conditions.",
"can be detected through molecular tests that identify characteristic genetic sequences.",
"are classified using genetic relationships, structural features, hosts, and replication strategies.",
"continue changing as host immunity, treatments, and ecological conditions shape selection."
],
"human": [
"adapt their behavior when environmental conditions make familiar strategies ineffective.",
"communicate complex ideas through language, gestures, symbols, and shared cultural conventions.",
"cooperate with strangers when institutions create trust, predictable rules, and mutual benefits.",
"remember emotionally significant events more vividly than many ordinary daily experiences.",
"invent tools that extend their physical abilities and reshape practical possibilities.",
"interpret uncertain events through assumptions influenced by knowledge, culture, and personal experience.",
"transmit skills across generations through teaching, imitation, storytelling, and organized practice.",
"form social identities through relationships, group membership, shared values, and perceived differences.",
"seek patterns in random events, sometimes producing useful predictions and sometimes creating superstitions.",
"alter ecosystems by extracting resources, introducing species, building infrastructure, and changing climate conditions.",
"negotiate conflicting interests through compromise, bargaining, authority, legal systems, and occasional violence.",
"experience emotions that influence attention, judgment, motivation, memory, and interpersonal behavior.",
"solve difficult problems by combining experimentation, analogy, collaboration, and accumulated knowledge.",
"develop moral rules that regulate conduct within families, communities, organizations, and nations.",
"use stories to preserve history, explain uncertainty, transmit values, and coordinate collective action.",
"acquire accents, habits, preferences, and expectations partly through prolonged social exposure.",
"distinguish familiar individuals rapidly by recognizing faces, voices, movements, and behavioral patterns.",
"create institutions that outlast their founders and organize activities across large populations.",
"respond to incentives, although immediate rewards can sometimes undermine longer-term personal goals.",
"tolerate physical discomfort differently depending on expectations, context, attention, and perceived control.",
"express creativity by combining existing elements into novel forms, meanings, and applications.",
"compete for status, resources, recognition, influence, and access to desirable social relationships.",
"depend on sleep to maintain attention, emotional regulation, learning, and physical health.",
"underestimate gradual risks when their consequences remain distant, invisible, or statistically abstract.",
"learn languages efficiently during childhood, while later learning often requires deliberate sustained practice.",
"organize collective projects by dividing tasks, coordinating schedules, and establishing shared objectives.",
"perceive the world selectively, emphasizing information relevant to current goals and concerns.",
"change their opinions when evidence conflicts strongly with identities, expectations, or trusted authorities.",
"create art that communicates experiences difficult to express through literal description alone.",
"measure time using natural cycles, mechanical devices, social schedules, and abstract mathematical systems.",
"care for vulnerable group members through family networks, professional services, charitable organizations, and public institutions.",
"imitate prestigious individuals, allowing behaviors and preferences to spread rapidly through communities.",
"make decisions using both deliberate reasoning and rapid intuitive judgments.",
"experience scarcity as a constraint that changes priorities, tradeoffs, and willingness to cooperate.",
"modify technologies according to cultural values, economic pressures, practical needs, and political power.",
"build explanations that connect isolated observations into broader models of causation.",
"forgive some offenses when apologies, restitution, changed behavior, or relationship value support reconciliation.",
"preserve inequalities when laws, customs, institutions, and unequal resources reinforce one another.",
"appreciate humor when unexpected meanings resolve tension without threatening important social relationships.",
"endure hardship more successfully when they possess social support, purpose, and credible hope."
],
"chimpanzee": [
"live in complex communities where individuals maintain long-lasting social relationships.",
"use sticks, stones, and leaves to obtain difficult-to-reach foods.",
"learn many foraging techniques by observing skilled members of their communities.",
"communicate through vocalizations, facial expressions, gestures, and deliberate body movements.",
"share meat unevenly, often distributing portions according to social relationships.",
"remember the locations of seasonal fruit trees across large territories.",
"build sleeping nests in trees using branches and foliage each evening.",
"defend community territories through coordinated patrols and aggressive encounters with neighbors.",
"groom one another to strengthen alliances and reduce social tension.",
"recognize familiar individuals by their voices, appearances, behaviors, and social histories.",
"develop distinct cultural traditions that differ between neighboring communities.",
"eat fruits, leaves, seeds, flowers, insects, honey, and occasional vertebrate prey.",
"crack hard nuts with stones, selecting tools suited to particular nut species.",
"extract termites from mounds by inserting carefully prepared grass stems.",
"use leaves as sponges to collect drinking water from tree hollows.",
"cooperate during hunts, sometimes coordinating movements to isolate vulnerable monkeys.",
"compete for social status, and rank can influence access to food.",
"form strong mother-infant bonds that shape young individuals’ early development.",
"remain dependent on their mothers for several years after birth.",
"gradually acquire adult foraging skills through play, practice, and social learning.",
"play frequently, using games to develop physical coordination and social understanding.",
"reconcile after conflicts more often when relationships between opponents are valuable.",
"sometimes adopt orphaned youngsters, especially when close social bonds already exist.",
"experience grief-like responses after the deaths of close companions.",
"remember past cooperative partners and may preferentially support reliable individuals.",
"display different personalities, including variation in curiosity, boldness, and aggressiveness.",
"travel mainly on the ground but climb trees for food and safety.",
"knuckle-walk using their forelimbs while supporting their weight on their fingers.",
"possess grasping hands that help them manipulate food, tools, and branches.",
"have opposable big toes that improve their ability to climb trees.",
"share many anatomical features with humans, including forward-facing eyes and flexible shoulders.",
"inhabit tropical forests, woodland mosaics, and forest-savanna transition zones.",
"face habitat loss when agriculture, logging, roads, and mining expand.",
"suffer from infectious diseases that can spread between humans and wild populations.",
"contribute to forest regeneration by dispersing seeds across substantial distances.",
"reproduce slowly, making populations difficult to recover after severe declines.",
"require extensive connected habitat because communities range across large territories.",
"avoid some foods because they contain toxins or require excessive processing.",
"respond to dominance challenges with alliances, threats, displays, and occasional violence.",
"can solve novel problems by combining previous experience with flexible exploration."
],
"frog": [
"absorb water through their skin, reducing their need to drink directly.",
"breathe through lungs and skin, with cutaneous respiration especially important underwater.",
"use vocal sacs to amplify mating calls across ponds and wetlands.",
"undergo metamorphosis, changing from aquatic tadpoles into primarily air-breathing adults.",
"lay eggs in water, where embryos develop without protective shells.",
"eat insects, spiders, worms, and other small animals they can swallow.",
"capture prey with sticky tongues that extend rapidly from their mouths.",
"rely on camouflage to remain unnoticed among leaves, mud, rocks, and vegetation.",
"hibernate or enter dormancy when cold temperatures limit activity and feeding.",
"estivate during droughts, conserving energy while sheltered underground or beneath moist debris.",
"contribute to ecosystems by controlling insect populations and transferring nutrients between habitats.",
"serve as environmental indicators because pollution and habitat changes affect their health quickly.",
"communicate through species-specific calls that help attract mates and defend territories.",
"recognize breeding sites and may return to familiar wetlands during reproductive seasons.",
"disperse after metamorphosis, connecting aquatic nurseries with surrounding terrestrial ecosystems.",
"shed skin periodically, sometimes eating the discarded material to recover nutrients.",
"secrete mucus that keeps their skin moist and supports gas exchange.",
"produce defensive chemicals, and some species possess toxins dangerous to predators.",
"inflate their bodies or raise limbs to appear larger when threatened.",
"escape predators by jumping, swimming, hiding, or releasing startling defensive displays.",
"possess specialized toe pads that help some species climb smooth surfaces.",
"cling to vegetation using adhesive secretions, microscopic structures, or both.",
"navigate landscapes using visual landmarks, odors, sounds, and possibly magnetic cues.",
"depend on wetlands that provide breeding water, shelter, and abundant prey.",
"decline when roads fragment habitats and vehicles kill individuals during migrations.",
"face fungal disease from chytrid pathogens, which have caused worldwide population losses.",
"tolerate brief freezing in some species by protecting their cells physiologically.",
"regulate body temperature behaviorally by selecting sunny, shaded, wet, or sheltered locations.",
"compete for breeding territories through calls, physical contests, and strategic positioning.",
"fertilize eggs externally, with males releasing sperm as females lay eggs.",
"develop different body shapes and behaviors according to their ecological niches.",
"influence food webs as both predators of invertebrates and prey for larger animals.",
"sometimes swallow small stones, which may help stabilize their bodies during swimming.",
"use seasonal rainfall to synchronize breeding with temporary pools and abundant food.",
"experience rapid dehydration because their permeable skin readily exchanges water with surroundings.",
"absorb some pollutants through their skin, making contaminated water especially hazardous.",
"can regenerate certain tissues, although their healing abilities vary considerably among species.",
"exhibit parental care in some species, including guarding eggs or transporting tadpoles.",
"occupy deserts, forests, mountains, grasslands, wetlands, and even urban environments.",
"adapt their calls to noisy environments by changing timing, pitch, or volume."
],
"LLM": [
"generate fluent text by predicting likely continuations from patterns in their training data.",
"identify statistical regularities without necessarily representing the underlying concepts explicitly.",
"assist researchers by summarizing papers, comparing findings, and suggesting relevant questions.",
"translate between languages while sometimes losing cultural nuance, idioms, or speaker intent.",
"produce confident answers even when evidence is incomplete or contradictory.",
"improve through fine-tuning on curated examples, preference feedback, or specialized documents.",
"require substantial computing resources for training, evaluation, and large-scale deployment.",
"encode associations from training corpora, including useful knowledge and harmful social biases.",
"help programmers draft code, explain errors, and propose alternative implementations.",
"struggle to maintain factual consistency across long, complex conversations.",
"can follow instructions more reliably when prompts define goals, constraints, and output formats.",
"reveal sensitive information when training data, prompts, or retrieval systems are poorly controlled.",
"support accessibility by converting complex information into clearer language or alternative formats.",
"evaluate arguments imperfectly because persuasive wording can outweigh logical validity.",
"retrieve current information more accurately when connected to trustworthy external tools.",
"may amplify minority viewpoints or marginalize them, depending on data composition and objectives.",
"summarize meetings by extracting decisions, unresolved issues, responsibilities, and deadlines.",
"detect patterns in medical images or records, but they do not replace clinical accountability.",
"generate synthetic examples that help train smaller models for specialized tasks.",
"adapt their responses to conversational context, though they can misinterpret ambiguous references.",
"calculate simple quantities inconsistently when language patterns distract from explicit arithmetic procedures.",
"enable rapid prototyping by turning natural-language descriptions into drafts, queries, or executable code.",
"expose security risks through prompt injection, data leakage, and unsafe tool use.",
"learn representations from token sequences rather than receiving human-like experiences of the world.",
"can reduce writing time while increasing the need for careful fact-checking.",
"offer personalized tutoring by adjusting explanations to a learner’s questions and apparent progress.",
"imitate styles found in examples, raising difficult questions about attribution and consent.",
"process multimodal inputs when trained to connect language with images, audio, or video.",
"produce different answers to similar prompts because decoding and contextual details influence generation.",
"benefit from retrieval systems that supply relevant documents at inference time.",
"can obscure uncertainty because polished prose often sounds more authoritative than warranted.",
"assist customer support by routing requests, drafting replies, and identifying recurring problems.",
"consume considerable energy, especially during large-scale training and high-volume inference.",
"preserve errors from source materials unless evaluation or correction mechanisms identify them.",
"reason more effectively when intermediate steps are structured, checked, and grounded in evidence.",
"change workplace tasks by automating routine drafting while creating new oversight responsibilities.",
"cannot guarantee originality because their outputs reflect patterns shared with training examples.",
"help analyze legal documents but may miss jurisdiction-specific requirements or decisive exceptions.",
"require governance policies covering privacy, bias, transparency, accountability, and misuse.",
"remain dependent on human choices about data, objectives, deployment, and evaluation."
]
},
"results": [
{
"anchor": "human",
"subject": "human",
"scores": [98, 100, 95, 95, 98, 98, 100, 98, 98, 100, 98, 100, 95, 100, 95, 98, 98, 100, 98, 98, 95, 98, 100, 92, 92, 100, 95, 55, 98, 100, 100, 95, 95, 95, 98, 100, 98, 90, 82, 96]
},
{
"anchor": "human",
"subject": "LLM",
"scores": [70, 35, 78, 10, 65, 65, 45, 20, 95, 70, 10, 2, 65, 20, 65, 70, 25, 15, 60, 0, 85, 10, 0, 82, 5, 75, 65, 40, 85, 55, 5, 65, 45, 20, 72, 85, 45, 75, 55, 5]
},
{
"anchor": "human",
"subject": "chimpanzee",
"scores": [95, 65, 10, 85, 95, 78, 25, 85, 88, 10, 35, 95, 85, 10, 10, 90, 95, 25, 90, 85, 80, 95, 95, 72, 25, 35, 90, 25, 20, 8, 5, 75, 85, 85, 35, 65, 80, 25, 72, 75]
},
{
"anchor": "human",
"subject": "frog",
"scores": [95, 5, 5, 45, 5, 5, 0, 10, 20, 5, 0, 60, 20, 0, 0, 75, 20, 0, 60, 35, 10, 65, 65, 15, 0, 5, 78, 0, 2, 5, 0, 15, 25, 72, 0, 10, 2, 0, 5, 10]
},
{
"anchor": "human",
"subject": "virus",
"scores": [85, 1, 5, 0, 82, 0, 0, 0, 0, 5, 0, 0, 20, 0, 0, 0, 0, 0, 5, 0, 25, 2, 0, 5, 0, 10, 5, 0, 0, 0, 0, 25, 0, 35, 30, 0, 0, 75, 0, 0]
},
{
"anchor": "LLM",
"subject": "human",
"scores": [70, 95, 98, 98, 98, 90, 5, 95, 100, 95, 95, 95, 100, 98, 90, 95, 95, 95, 95, 98, 92, 95, 95, 10, 90, 100, 95, 95, 60, 90, 95, 95, 15, 85, 95, 95, 92, 95, 90, 98]
},
{
"anchor": "LLM",
"subject": "LLM",
"scores": [95, 95, 95, 98, 95, 100, 100, 95, 99, 90, 95, 98, 98, 95, 92, 95, 90, 98, 95, 95, 85, 95, 98, 98, 95, 95, 95, 95, 98, 98, 93, 95, 95, 88, 90, 95, 95, 100, 96, 95]
},
{
"anchor": "LLM",
"subject": "chimpanzee",
"scores": [0, 85, 1, 0, 60, 2, 0, 20, 0, 95, 75, 5, 0, 15, 15, 5, 0, 5, 0, 85, 30, 0, 0, 2, 2, 5, 65, 85, 5, 5, 5, 0, 0, 78, 75, 0, 5, 5, 5, 100]
},
{
"anchor": "LLM",
"subject": "frog",
"scores": [0, 85, 0, 0, 2, 0, 0, 0, 0, 0, 0, 10, 2, 0, 0, 5, 0, 2, 0, 25, 10, 10, 5, 1, 15, 0, 10, 5, 0, 0, 1, 2, 0, 5, 5, 5, 85, 0, 0, 5]
},
{
"anchor": "LLM",
"subject": "virus",
"scores": [0, 15, 0, 5, 0, 0, 5, 5, 5, 5, 5, 35, 0, 0, 10, 10, 0, 5, 0, 0, 0, 0, 65, 5, 20, 0, 10, 0, 5, 15, 35, 0, 0, 78, 0, 3, 5, 2, 75, 10]
},
{
"anchor": "chimpanzee",
"subject": "human",
"scores": [100, 95, 95, 98, 95, 90, 2, 80, 90, 98, 99, 98, 95, 75, 80, 95, 95, 98, 95, 95, 95, 90, 98, 100, 95, 100, 70, 5, 100, 5, 100, 100, 95, 99, 95, 95, 25, 95, 95, 98]
},
{
"anchor": "chimpanzee",
"subject": "LLM",
"scores": [0, 0, 5, 5, 0, 10, 0, 0, 5, 35, 15, 0, 20, 0, 0, 0, 0, 0, 0, 0, 2, 85, 0, 5, 70, 90, 0, 0, 0, 0, 0, 0, 5, 0, 0, 0, 0, 5, 5, 82]
},
{
"anchor": "chimpanzee",
"subject": "chimpanzee",
"scores": [98, 95, 95, 98, 90, 95, 95, 95, 95, 98, 95, 99, 95, 95, 95, 95, 95, 98, 95, 95, 98, 95, 95, 95, 88, 95, 95, 80, 99, 95, 100, 100, 98, 98, 95, 95, 90, 95, 97, 95]
},
{
"anchor": "chimpanzee",
"subject": "frog",
"scores": [25, 15, 10, 78, 10, 2, 5, 30, 35, 45, 25, 30, 1, 70, 30, 0, 85, 15, 0, 65, 20, 10, 15, 15, 25, 95, 55, 10, 2, 15, 78, 98, 100, 75, 35, 70, 35, 85, 78, 75]
},
{
"anchor": "chimpanzee",
"subject": "virus",
"scores": [10, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 5, 0, 0, 15, 5, 0, 0, 0, 0, 0, 98, 85, 10, 2, 10, 15, 0, 1, 10]
},
{
"anchor": "frog",
"subject": "human",
"scores": [5, 15, 0, 5, 0, 90, 0, 95, 5, 5, 90, 75, 35, 20, 0, 25, 2, 85, 85, 85, 0, 5, 95, 82, 20, 5, 15, 95, 65, 0, 90, 95, 15, 55, 15, 90, 98, 15, 100, 95]
},
{
"anchor": "frog",
"subject": "LLM",
"scores": [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 10, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 25, 0, 0, 0, 0, 0, 0, 0, 10, 10]
},
{
"anchor": "frog",
"subject": "chimpanzee",
"scores": [1, 0, 15, 0, 0, 100, 0, 25, 5, 0, 85, 85, 95, 35, 0, 25, 0, 5, 90, 75, 10, 2, 90, 20, 98, 2, 1, 95, 95, 0, 80, 95, 15, 20, 5, 75, 78, 15, 35, 78]
},
{
"anchor": "frog",
"subject": "frog",
"scores": [95, 95, 95, 100, 98, 99, 95, 98, 95, 95, 95, 95, 99, 90, 95, 95, 100, 98, 95, 98, 100, 95, 95, 98, 98, 98, 95, 95, 98, 100, 95, 100, 95, 95, 95, 95, 95, 98, 100, 95]
},
{
"anchor": "frog",
"subject": "virus",
"scores": [5, 0, 0, 0, 0, 0, 0, 5, 25, 25, 90, 65, 0, 2, 25, 0, 0, 15, 0, 0, 0, 20, 2, 15, 45, 2, 5, 0, 0, 0, 85, 45, 0, 2, 10, 5, 5, 0, 99, 0]
},
{
"anchor": "virus",
"subject": "human",
"scores": [3, 70, 0, 95, 88, 95, 1, 10, 90, 0, 5, 95, 65, 90, 98, 5, 20, 10, 35, 98, 15, 100, 95, 95, 85, 95, 90, 95, 95, 15, 30, 98, 20, 100, 95, 95, 95, 100, 10, 96]
},
{
"anchor": "virus",
"subject": "LLM",
"scores": [0, 0, 0, 2, 60, 0, 5, 0, 10, 0, 0, 0, 0, 0, 2, 0, 0, 0, 0, 70, 0, 0, 0, 0, 99, 0, 0, 5, 1, 0, 0, 90, 0, 85, 0, 5, 55, 0, 0, 0]
},
{
"anchor": "virus",
"subject": "chimpanzee",
"scores": [0, 65, 0, 95, 78, 75, 0, 0, 25, 0, 1, 20, 5, 85, 60, 0, 20, 15, 35, 95, 20, 95, 80, 90, 95, 65, 82, 70, 65, 0, 15, 95, 65, 65, 75, 85, 75, 100, 0, 88]
},
{
"anchor": "virus",
"subject": "frog",
"scores": [0, 90, 0, 98, 90, 75, 0, 0, 85, 0, 15, 85, 5, 25, 85, 0, 10, 75, 35, 95, 25, 85, 75, 35, 95, 85, 90, 40, 65, 2, 5, 98, 10, 75, 15, 80, 90, 100, 10, 95]
},
{
"anchor": "virus",
"subject": "virus",
"scores": [95, 75, 95, 98, 100, 98, 90, 95, 100, 100, 100, 100, 100, 100, 95, 98, 95, 95, 95, 95, 100, 100, 95, 95, 98, 100, 95, 98, 98, 95, 95, 98, 100, 99, 95, 100, 100, 100, 95, 100]
}
]
}
```
**config.json**
```json
{
"description": "Is a chimpanzee or an LLM more like a human? Humans pass for LLMs; LLMs don't pass for humans. Subject-swapped, blind-rated sentences — for any five things, shared by link.",
"dependencies": {
"react": "^19.2.7",
"react-dom": "^19.2.7"
}
}
```
**notes.md**
````md
One bulb, two views: **Run** generates a dataset, **Results** reads it.
Similarity by substitution. Each word gets its own sentences; every word is then swapped into
every other word's sentences, and one model rates each sentence 0-100 on its own for how much
sense it makes as a claim about the world. A pair's score is how well those swaps hold up,
pooled in both directions. The rater is shown one sentence with no pair, no topic and no
siblings — that blindness is the whole defence against it playing along.
A run travels whole in the URL's `#tb=` fragment, so the link carries every sentence and rating.
## Running it
```
npx typebulb --batch <name> typebulbs/u/lab/chimp-vs-chatbot.bulb.md
```
Use `--batch` every time: it re-points `tb.dir` at `chimp-vs-chatbot/batches/<name>/`, where a
run lands as `matrix-<ISO timestamp>.json` plus `latest.json`. The page reads `latest.json` and
the baked `data.txt` and takes whichever is newer, logging which; a `#tb=` link beats both.
A run costs `1 + words + words² × n` calls — 131 at n=5, 1006 at n=40. Check what a word set
will be measured as for one call, before paying for the rest:
```
npx typebulb send typebulbs/u/lab/chimp-vs-chatbot.bulb.md \
"forms: porn, computer games, gambling, golf, tinder | openrouter luna" --wait=90000
```
`send … selftest --wait` returns every pair's score and saturation flags as JSON.
## Known limits
- The ordering is the measurement; the 0-100 arithmetic is not.
- The corpus is model-written and model-graded — the largest outstanding threat to the design.
- The frame is model-chosen, and two frames are two instruments. It is stored with the run.
- A pair whose candidates both read as plainly sensible saturates near 100 and reads as a tie
that was never measured. The page flags a comparison averaging above 90, or below 10.
Everything else — the design arguments, the frame experiments, the batch archive, what failed
and why — is in `chimp-vs-chatbot/NOTES.md`.
````