This technical note was originally written by me in early 2021 on ways to generalize the successor matrix and enable flexible generalization of reward functions across changing environments. It probably does not make much sense unless you are familiar with the successor representation in RL. I originally planned to explore...
[Read More]
Hedonic loops and taming RL
Everybody knows about the hedonic treadmill. Your hedonic state adjusts to your circumstances over time and quickly reverts to a mostly stable baseline. This is true of basic physiological needs – you feel hungry; you seek out food; you eat; you feel sated, and you no longer seek food. This...
[Read More]
Predictive coding networks can perform causal and counterfactual inference
This note was written around Christmastime 2021 (hence the Christmas theme) as my initial thoughts after figuring out that predictive coding networks could be straightforwardly adapted to perform causal inference. I intended to write this up into a proper paper and do more experiments verifying it works at scale as...
[Read More]
Scaffolded LLMs as natural language computers
Recently, LLM-based agents have been all the rage – with projects like AutoGPT showing how easy it is to wrap an LLM in a simple agentic loop and prompt it to achieve real-world tasks. More generally, we can think about the class of ‘scaffolded’ 1 LLM systems – which wrap...
[Read More]
The singularity as cognitive decoupling
An interesting way I like to think about the singularity is as the cognitive decoupling. Specifically, the singularity is the final industrial revolution that lets capital be converted directly into intellectual labour 1. The first industrial revolution occurred when capital became convertible into manual energy – i.e. humanity learned to...
[Read More]