Treating LLM Latents as Continuous Actions
A proposal for RL fine-tuning LLMs in latent space instead of token space — making pathwise gradients (SAC-style) applicable and sidestepping the 150k-way discrete exploration problem.
Posts
Occasional writing on my projects and ideas.
Ladder style value transformers for LLM fine tuning
Tuning Qwen3 4b with a custom RL architecture from scratch
Memory and Implicit Communication
Emergent multi-agent signaling with spatial memory
Online Transformer RL
Training a transformer with PPO to solve POMDPs
Vanilla Actor Critic
Implement a actor critic from scratch in jax