Feature	Vision-Language Models (VLM)	Vision-Language-Action (VLAs)	Video World Models	World Action Models (WAMs)
Core function	Maps text visual data (images, videos) to text (descriptions, answering questions about an image, ...)	Maps visual observations and language instructions directly to physical robot actions	Simulates and predicts what will happen next in a sequence	Jointly simulates future world states (video) & predicts the actions required to achieve them
Math formulation	$y \sim p(\cdot \mid o, \ell)$ (observation $o$, language prompt $\ell$)	$a_t \sim \pi(\cdot \mid o_t, \ell)$ (language prompt $\ell$)	$o_{t+1:T} \sim p(\cdot \mid o_{1:t}, a_{1:t}, \ell)$ (predicts future video frames $o_{t+1:T}$ given a history of past observations $o_{1:t}$, past actions $a_{1:t}$, and language instruction $\ell$)	$(o_{t+1:T}, a_{t:T}) \sim p(\cdot \mid o_{1:t}, \ell)$
Primary output	Text (descriptions, labels, high-level code)	Low-level control commands (e.g., joint torques, end-effector poses, gripper states)	Video frames (pixels showing predicted future states)	Both video frames & low-level control commands
Training objective	Aligning visual data with text semantics	Behavioral cloning/imitation learning (predicting the correct action tokens from expert demonstrations)	Next-frame prediction (learning forward dynamics)	Jointly modeling video and action via generative backbones
Grasp of physics	Weak, relies on statistical patterns in text.	Grounded but reactive - knows "what to do" based on training, but doesn't explicitly know "what happens next"	Stronger, implicitly learns geometry, collisions, and spatial dynamics by predicting motion.	Strong & applied - inherits the physical dynamics of a video model + directly maps them to physical execution
Examples	DeepMind Flamingo, https://llava-vl.github.io	https://robotics-transformer2.github.io	https://deepmind.google/models/genie/	https://dreamzero0.github.io

Forward dynamics: Computation of motions from forces. Given the configuration $q \in \mathcal{C}$, tangent velocity $v$, torques $\tau$, and contact forces $f$ acting on our robot, forward dynamics is the calculation of joint accelerations $\dot{v}$ consistent with the constrained equations of motion. TL;DR: FD is used to simulate robot motion.

• 💡 How are VLAs different from video world models?
  • VLAs are reactive policies: they look at the current state and output the next best action based on the distribution of their training data.
    • If a VLA makes a mistake, it cannot foresee the consequence of that mistake until the next observation comes in. → this is where world models come in!
  • World models learn the environment's transition dynamics - so coupling a policy with a world model allows the robot to perform closed-loop rollouts internally (simulate a trajectory, realize an action will cause a collision/failed grasp, and adjust its plan before sending commands to the physical hardware)

Humanoids

• Recent deployments:
  • BMW: High-voltage battery production for EVs, where workers currently wear heavy protective gear
• Recent startups:
  • RLWRLD: https://www.rlwrld.ai (foundation model for robots trained using data collected directly inside of real factories, warehouses, and service environments)

Research

World action model (VLA alternative)

• DreamZero: World Action Models are Zero-shot Policies
  • Chris Paxton blogpost: https://itcanthink.substack.com/p/why-is-dreamzero-so-good-at-robotics
  • https://dreamzero0.github.io

DIY tutorials

• How we finetuned Pi05 for our custom task and environment with <$10K: https://x.com/bytedunks/status/2026942838085452184

Yap

Robotics startup lessons (YC)

(2.10.2026 post: https://ruixu.us/posts/six-things-robotics-startup, startup: https://kscale.ai)

TL;DR: We got the robots built and shipped ( The Information,Hacker News). The hardware worked. What we massively underestimated was everything else: sim-to-real transfer, locomotion tuning, the long tail of making a robot actually walk. We were perpetually “one week away” from a working gait. The team was young and fearless, which is great for morale and terrible for schedule estimates. I was buried in supply chain and couldn’t pull the software side forward. The gap between the pitch deck and reality kept growing.

1. Beware "Large model chauvinism": the belief that AI is getting so good that hardware can afford to be dumb.
   • Even Tesla still puts brakes on the car!
2. Stop using over-simplified analogies: they're for fundraising, not for building.
   • Throwing away information by compressing ideas down to simple analogies can kill you in engineering.
3. Hardware supply chain is not a task:
   • Manufacturing is not a service you buy, it's a capability you build.
   • Not: "Find someone who speaks Chinese, point them at a factory, check the box. This is one of the most common ways hardware startups get into trouble."
4. Robotics hardware has yet to become "commodity," assembled from off-the-shelf parts by Chinese manufacturers, with the real value sitting in the AI layer.
   • ⚠️ There's no standard BOM for a humanoid. No off-the-shelf actuators that just work for walking. Every team building a legged robot right now is designing custom hardware.
5. In a race, bad R&D decisions kill faster than bad luck
   • Author says their biggest mistake was getting stuck on locomotion - months burned, the robot still wasn't walking right, and the fundraising window closed / competitors shipped demos. → The entire team underestimated how hard the problem was and how long it would take.
   • GitHub is full of repos: but repos don't ship! Demos ship. Products ship!
   • R&D velocity isn't repos or commits or hours logged. It's how fast you converge on something that actually works.
6. 欲速则不达
   • It was always the robot walks next week. Every week.
   • When that's the culture^ -- people start cutting corners to hit impossible dates. Engineers write code with AI tools without reviewing it properly, sensors go into the product without calibration.
   • ^ The damage goes beyond engineering: when you make promises to your contract manufacturer based on fantasy timelines, you burn the relationship.
     • "Move fast, break things" works in software, does not work when your manufacturer is allocating factory floor time based on commitments you can't keep.

Video models vs. latent models for robotics world modeling

(Original X post: https://x.com/Majumdar_Ani/status/2021242532517040560)

Currently (early 2026), world modeling in robotics is dominated by video world models, i.e. generative models that are trained to predict future video frames conditioned on text (and potentially actions).

Why video models might be useful to robotics

• 💡 Core idea: Build off video modeling's progress in producing videos that embed complex physical interactions
  • Proof of concept: video models fine-tuned on robotics data can perform:
    • Policy evaluation: https://veo-robotics.github.io, https://ctrl-world.github.io, https://world-model-eval.github.io/abstract
    • Data generation: https://research.nvidia.com/labs/gear/dreamgen/
    • Inference-time plan generation: https://www.boyuan.space/large-video-planner/, https://mimic-video.github.io
  • Survey: https://arxiv.org/pdf/2601.07823v1

Challenges with video models

• Hallucinations:
  • Objects duplicating, appearing from nowhere, disappearing, morphing...
• Long-horizon generation:
  • Current video models in robotics struggle to produce high-quality generations beyond 20-30 seconds at most

Latent world models

• 💡 New idea: Don't predict pixels!
  • Intuition: Video modeling is overkill for a world model - e.g., predicting the motion of leaves in the background or the precise facial expressions of the person is unnecessary for learning the semantics of the world necessary to do robotics tasks that we care about.
  • ^ Ideally: Focus representation power on "things that matter"
• Yann LeCun: JEPAs (joint-embedding predictive architectures) that learn latent representations of observations that can predict representations of other (e.g., future) observations - https://openreview.net/pdf?id=BZ5a1r-kVsf
  • e.g., https://ai.meta.com/research/vjepa/ shows that...
    • Video features can emerge from this kind of self-supervised learning
    • Action-conditioned version of the model can enable robot planning by optimizing action sequences at inference time
• 💡 Latent world models could be good for long-horizon tasks:
  • Predicting pixel-level details of how the world will evolve in the next 10-20 seconds seems plausible, but doing this at the scale of minutes or hours seems exceedingly challenging & unnecessary.

Author's take on how the future will unroll

• 🔑 Prediction: Video models will win in the short term, because...
  • Conceptual simplicity:
  • Clear evaluation metrics:
  • Inference-time scaling with verifiers:
  • Video models enable policy evaluation:
  • Video models compose naturally with image editors:
  • Huge commercial incentives: There is massive commercial pressure to develop good video generation models (independent of robotics!) - social media apps, movie making, ...
    • e.g., depth cameras for gaming, IMUs for smart phones, LLMs for NLP - technologies developed for independent commercial reasons have time and time again revolutionized robotics.

Author prediction: In the next 2-3 terms, they expect video models to continue as the dominant force in world modeling for robotics, especially for short-horizon manipulation tasks.

• 🔑 But will JEPAs win in the long term?
• Still largely unresolved if the "predictable features" learned with a JEPA are useful features for robotics.
  • But DINO provides proof of concept: https://github.com/facebookresearch/dinov2

Next step reading list

List of things I want to read/deep dive into next!

• https://github.com/facebookresearch/dinov2
• https://ai.meta.com/research/vjepa/
• https://dreamzero0.github.io