donny.nyc/unprompted
1
0
Fork 0
Code Actions Releases Activity
Files
5f430432c8b33cab3f5872073a5c528ffd7242e4
unprompted/columns/2026-07-16-panel-of-judges.md
Stephen Donahue c6d7c8c211
Some checks failed
Deploy to S3 / deploy (push) Failing after 1m24s
Details
Swap published columns for 24 entries from random.md 
Retract the seven published columns into drafts/ and promote the 24 per-date
entries split out of Notes/random.md. Bodies are verbatim; titles, subjects
and slugs are new.

Make publication a matter of location. Notes/Notes.11tydata.js gives anything
under Notes/ permalink: false unless tagged `column`; drafts/drafts.11tydata.js
renders drafts at their real permalink under `yarn dev` and excludes them from
collections entirely in a build, so no template can surface one. Publishing is
now `git mv drafts/<file> columns/`, and moving it back retracts it.

Untagged notes were previously rendered into _site/ and deployed — including
Notes/random.md, which was live at /Notes/random/.
2026-08-02 22:45:19 +00:00

1.7 KiB
Raw Blame History

title, date, layout, tags, permalink
title date layout tags permalink
Panel of Judges 2026-07-16 column.njk
column
computing
/columns/2026/panel-of-judges/

Been a month - wild, unverified ideas

  1. How could quantum computing impact LLMs, or other probablistic, non-deterministic models?

  2. Can we put type safety, reliability features around LLM invocations? Can we formalize it, as its own kind of process: sequential or concurrent invocations of asynchronous processes.

We do that already with RPCs, web requests, etc.

what are some safety patterns around async await?

  • timeouts
    • can we explicitly prompt the LLM: You have 500ms to respond or we move on?
    • Like an ad network bid: "winning" response wins
    • How are they compared? Panel of judges
      • At first I thought you could single "identical" judges in front of each submission
        • that's counting on each to be trustworthy
      • Then I thought about having a panel of judges, quorum style
      • Then I thought about a "ranking funnel" a shrinking pool, maybe bracket style (is that a merge-sort elimination?)
        • or you grade pools of responses as a collection, instead of individually (can't tell if this would be a fidelity loss or win - would that matter?)
    • this sounds really expensive: time and resources
      • 1/N success rate, and what does it take to eliminate the other N-1?
    • it sounds much less efficient than developing / evolving a single model session (some kind of hybrid) as long as it survives (and maybe each session can determine that for itself).
    • LLM (lives) are not precious. They can off themselves.
  • exception handling // what's exceptional when prompting an LLM?
  • validations / sanitation