ML Theory Learning Group
I used to co-lead the ML Theory Learning Group at Cohere for AI. We mixed invited talks, member-led pitches, and research-proposal sessions, with the goal of taking theoretical ideas seriously while keeping them close enough to modern models to shape how we build systems.
Speaker series and deep dives
Longer sessions, invited talks, and topic-focused presentations.
Lottery Ticket Hypothesis
Harsha presented the Lottery Ticket Hypothesis: sparse subnetworks inside dense models, why pruning can reveal useful structure, and how this changes the way we think about overparameterization.
Yaroslav Bulatov: generating functions for gradient descent
The talk explored a generating-function approach to gradient descent analysis: instead of only worst-case optimization theorems, ask what the average-case trajectory looks like.
Privacy of Noisy SGD
Ajinkya led a session on the privacy behavior of noisy SGD and how optimization noise, stability, and data leakage interact.
Resource
Neural Operators
I presented neural operators as models that learn maps between function spaces, with applications in PDEs, scientific computing, and fast surrogate simulation.
Resource
Topic pitches and proposal sessions
Shorter sessions where members proposed directions and voted on future talks.
Lottery tickets, neural operators, and why Adam still survives
A wide-ranging session that reopened the year: sparse subnetworks, neural operators, and the gap between optimizer theory and what people actually use for large models.
Notes
We discussed whether pruned subnetworks reveal reusable circuit structure, and why Adam remains hard to dislodge despite a long list of proposed alternatives.
One thread that kept coming back was whether regular structures inside lottery tickets can be treated almost like reusable computational pieces: not a complete AGI story, obviously, but a useful way to ask what pruning is really exposing.
Fourier neural operators and gradient descent proofs
A final session for the year with pitches on Fourier Neural Operators, gradient descent proofs, diffusion models, and an optimization-book reading plan.
Session notes
I gave a short preview of Fourier Neural Operators, Harsha pitched gradient descent proofs, Ashish suggested diffusion models, and the group discussed starting a more systematic optimization reading plan after the break.
CLIP and contrastive learning
Ahmad Mustafa presented CLIP, contrastive learning, language-guided recognition, and the core ideas behind pairing image and text representations.
Meta-learning and neural information retrieval
Sree Harsha Nelaturu and Max Marion spoke about meta-learning and neural information retrieval, respectively.
Proposal round: meta-learning, quantum probability, neural IR, representation learning
Four proposals set up the next talks. The group voted to prioritize meta-learning, neural information retrieval, representation learning, and quantum probability.
First session and mission statement
We started by discussing what members wanted from the group, what kinds of theory felt useful, and how to choose topics that invite both rigor and research taste.
