Beren's Blog

Now that deep learning has been fully victorious and the reasons for its victory have started to be understood, it is interesting to revisit some old debates and return to the question of why GOFAI failed with a new perspective. [Read More]

Originally written in early 2021 while writing up my thesis. Never got around to publishing it on my blog then. I think it might be interesting to people wanting to see what a PhD looks like from the ‘inside’. Note that everyone’s PhD experience is highly personal to them and... [Read More]

Epistemic status: I owe a lot of my thoughts to Jacob Cannell’s work on both brains and deep learning. My thinking here comes from my experience in large-scale ML as well as neuroscience background and specifically experience in analog hardware for deep learning [Read More]

This is a short post meant to highlight something I do not yet understand and therefore a potential issue with my models. Why do vision (and audio) models work so well despite being so small? State of the art models like stable diffusion and midjourney work exceptionally well, generating near-photorealistic... [Read More]

I have been increasingly thinking about NN representations and slowly coming to the conclusion that they are (almost) completely secretly linear inside 1. This means that, theoretically, if we can understand their directions, we can very easily exert very powerful control on the internal representations, as well as compose and... [Read More]