Notes

Passive vs. Active Intelligence

🔑 Key claim: We need to progress from passive to active intelligence for robots to become high throughput and general
- Currently, there is a divide: Industry robots are high throughput and highly specific to the task; academia robots are low throughput and attempting to be highly general ("generalist policies," etc.)
- We need to close this divide to obtain embodied intelligence and have general robots roaming around in the world!

Paradigm shifts that are necessary in modern robotics

Position intelligence vs. force intelligence

💡 Key idea: During manipulation tasks, humans interact with the world through force intelligence and not position intelligence.
- We don't screw in a screw by finding the perfect position to place the screw-head such that the threads match up perfectly, and then start screwing. We place the screw in a generally correct position, and softly fiddle with it until the threads click → force intelligence!

^ Because force intelligence allows more margins of error in robot motion during execution, it needs vastly less training data than position intelligence → force intelligence can enable better scaling laws!
- Within the position intelligence paradigm: increasing accuracy by a few mm requires 10K-100K++ more data points
🔑 Position intelligence paradigm generally focuses on avoiding contact with the world: but think of a human grabbing keys on a table top! We are fast because we slam our hand on the table at a reasonable force, feel for the keys, and then grab them. We do not exhibit conflict avoidance when we manipulate the world!
- ^ So we should program robots that do the same.

Why manipulation is hard

Humanoids interact with the world: Earth, largely speaking, allows a high margin of error (nothing bad will happen if you step too hard on the ground) -- humanoids can be off by cm's in their policies and still walk properly. Manipulation is much more delicate! (And usually, with how heavy the robot arm is, vs. the manipulation task it completes, we end up with a situation similar to having the truck below move the chair.)

💡 2 parts of robotic manipulation:
1. Unlocking RL: By overcoming the sim2real gap
2. Hardware that enables force reasoning: Forces are the language of physical interactions

We need better hardware

To achieve better robot manipulation, we need better hardware.

Robot wrists

🔑 Research shows that DOFs in the wrist of a robot arm are more effective than DOFs in the hand:

^ Inspired by this, their lab developed DexWrist: https://dexwrist.csail.mit.edu

Design principles for hardware to allow high throughput

design principles for hardware the supports high throughput.png

Tactile sensing

🔑 Vision + tactile policy performs better than vision-only policy!
- e.g., if you open a ketchup bottle, you rarely turn the bottle s.t. the latch is facing you before you open it. As humans, we grab the bottle, feel for the latch, and then pop it open. Can we get robots to do the same? Yes!