Andy Zeng
I'm Co-founder and Chief Scientist at Generalist.
Before that, I was a research scientist at Google DeepMind – and before that, a student at Princeton and UC Berkeley.
AGI is inevitable. It will be born from the physical world. My life's work is to make it universally accessible and useful, for the new Renaissance of builders.
Recent work:
GEN-1.5: Embodied Foundation Models are One-Shot Learners
We're starting to see in-context learning and few-gradient-step adaptation emerge from robot pretraining. GEN-1.5 can learn a new task in seconds from a single example placed in context, with zero training. It can also be fine-tuned for a new task in 1–10 gradient steps on just 1–5 minutes of data. Improvisational intelligence, which we first observed in GEN-1, now appears more often.
GEN-1: Scaling Embodied Foundation Models to Mastery
By further scaling our models, accelerated by algorithmic advances, we're starting to see mastery of simple tasks at 99% success rates. We think much of this comes from improvisational intelligence: emergent behaviors from pretraining that enable the robot to recover from unexpected situations.
GEN-0: Embodied Foundation Models That Scale with Physical Interaction
We pretrained robot models from scratch on orders of magnitude more physical interaction data than ever before, and found that more pretraining makes adaptation to new tasks faster, cheaper, more data efficient, and more general. While this may have been the first time the world saw scaling laws for robotics, we had actually already seen the same exponents appear three times internally! Each time gave us increasing confidence that this was going to work.
Some past publications:
For a more up-to-date list, visit my Google Scholar.
Implicit Behavioral Cloning
We realized that sensorimotor action distributions can be highly complex, discontinuous, and multimodal, and that the standard policy-learning machinery at the time (regression) struggled to capture them. Implicit BC instead represents actions with an energy-based model, and serves as an early stepping stone towards the generative action models widely used in robot learning today.
PaLM-E: An Embodied Multimodal Language Model
I've been fortunate to work with Pete and Danny on what eventually became Google's largest embodied multimodal model. It transfers to robot models the visual-semantic knowledge from large-scale foundation models pretrained on Internet data.
Grasping in the Wild: Learning 6DoF Closed-Loop Grasping from Low-Cost Demonstrations
We noticed people picking up beach trash with grabber-reacher tools and wondered: with a camera and some 3D vision, could these become robot data-collection devices? So we built a low-cost handheld gripper for collecting grasping demonstrations in everyday environments, an early precursor to wearable/handheld data-collection interfaces like UMI (Chi et al., RSS 2024).