When the GPT-4V acts as a robot brain, you may not even be able to plan as well as the AI

The Institute of Cross-Information Studies (ICIS) at Tsinghua University has proposed an algorithm called "ViLa" (Robotic Vision-Language Planning) to plan and control robotic tasks in complex environments.The ViLa algorithm realizes GPT-4V to go into the physical world to provide planning for robotic life tasks. ViLa's ability to do joint reasoning with vision and language to allow robots to understand the common sense of the physical world is a concept that fills a gap in algorithms for task planning for robots with large language models.