Breck Baldwin
Welcome! I work at the intersection of large language models (LLMs), linguistic theory, and practical AI evaluation — a space I call Cyber Linguistics.
This site serves as a hub for my publications, talks, newsletters, and public outreach with a focus on the new stuff.
Note that all data and software are released with the goal of reproducabilty. If you are having problems getting my code/data working then please reach out–it is a priority for me.
Featured Talk
Title: Cyber Linguistics: In an era when machines speak, language is no longer just human
Event: NIST AI Metrology Colloquium
Date: 12 noon EDT, August 14, 2025
Full Announcment:
Slides
I don’t have the video of the talk yet, I’ll post here when it is up. It was a lot of fun.
Resources referenced:
Radio Show on WIOX: AI in the Catskills: Where bots get their hands dirty.
I host a biweekly radio show on WIOX that explores how AI can serve rural communities, nonprofits, and small businesses.
Cyber Linguistics Newsletter
Cyber Linguistics is the missing connective tissue between language and system architecture — between what models say, and what systems do.
Resources/writeups broken down by topic–newsletter unless indicated otherwise:
Manefesto:
Pragmatics:
LLM Determinism:
- Paper: Non-Determinism of “Deterministic” LLM Settings
- The Long Road to AGI Begins with Control: The problem pt 1
- The Long Road to AGI Begins with Control: Why nobody cares pt2
- The Long Road to AGI Begins with Control: Mitigation strategy #1 pt 3
Jupyter Notebooks on small LLM experiments
Small LLM experiments can become big LLM experiments. This is my seedling area.
Relationship between logprobs and accuracy and determinism.
- Reddit: Experiment: Can determinism of LLM output be predicted with output probabilities? TL;DR Not that I could find
- Notebook: https://github.com/breckbaldwin/llm-stability/blob/main/experiments/logprob/analysis.ipynb
Schemas impact on LLM performance
- Reddit: Impact of schema directed prompts on LLM determinism, accuracy
- Notebook: https://github.com/breckbaldwin/llm-stability/blob/main/experiments/json_schema/analysis.ipynb
Get in Touch
© 2025 Breck Baldwin
“Science is the art of setups where reality is the punchline. Try to have a sense of humor.”