Projects

Here are some of the projects I'm currently working on.

Platonic representation of Tic-Tac-Toe

The idea is to deeply study the representations of a transformer trained on Tic-Tac-Toe games. I use various techniques to interpret the transformer's layers, including probing and techniques developed in the Platonic Representation Hypothesis paper published in June 2024 by Philip Isola and his students at MIT. Learn more

Noisy chessGPT

Here, I define a notion of noise in a chess game as a uniform draw among legal moves. I train several LLMs on different noise rates (0%, 5%, 50%, and 80%) and observe better robustness in models trained on more noise (fewer illegal moves). Do these "noisy" models better understand the rules of chess?

Grokking Tic Tac Toe

Here, I attempt to train an LLM using the fastGrok algorithm and 1% of possible Tic Tac Toe games to verify if the model eventually understands the game exhaustively.