Active Inference is a theory of adaptive action selection for agents proposed by Karl Friston initially and now expanded upon by many authors and forms a small academic subfield of research. The core claims of the theory are that action selection and decision-making can be usefully understood as inference problems...
[Read More]
Right to Left (R2L) Integer Tokenization
This is a guest post by Max Buckley, a software engineer at Google and fellow AI researcher1.
Contributions: Max wrote a draft on this post and did the experiments, Beren provided editorial review. ↩
[Read More]
Alignment likely generalizes further than capabilities.
Recently, I was reading this paper which demonstrates how to do online RLHF for alignment of LLMs and a sentence stuck out to me:
[Read More]
The Unconditioned Distribution of Current Open LLMs
Last year, I wrote a quick post investigating the ‘unconditioned’ distribution of LLMs in the OpenAI API, where the ‘unconditioned distribution’ is simply the distribution of LLM outputs following the empty string – or beginning of sequence token. My intuition here was that this gives some idea of what the...
[Read More]
Capital Ownership Will Not Prevent Human Disempowerment
When discussing the future of AI, I semi-often hear an argument along the lines that in a slow takeoff world, despite AIs automating increasingly more of the economy, humanity will remain in the driving seat because of its ownership of capital. This world posits one where humanity effectively becomes a...
[Read More]