Microsoft researchers are using ChatGPT to instruct robots and drones (with video)

[Renew Deal]

OpenAI's ChatGPT isn't just good at generating coherent text responses to natural language prompts -- it can also play a role in human-to-robot interactions and use sensor feedback to write code for robot actions.

Microsoft recently conducted research to "see if ChatGPT can think beyond text, and reason about the physical world to help with robotics tasks." The aim was to see if people can use ChatGPT to instruct robots without learning programming languages or understanding robotic systems.


"The key challenge here is teaching ChatGPT how to solve problems considering the laws of physics, the context of the operating environment, and how the robot's physical actions can change the state of the world," a team from Microsoft Autonomous Systems and Robotics Research note in a blogpost.


The researchers note in their blog post: "ChatGPT asked clarification questions when the user's instructions were ambiguous, and wrote complex code structures for the drone such as a zig-zag pattern to visually inspect shelves."

Microsoft tested ChatGPT to use a robotic arm to move blocks around to form the Microsoft logo. The researchers also tasked ChatGPT with writing an algorithm for a drone to reach a point without crashing into obstacles. They also tested whether ChatGPT can decide where a robot should go based on sensor feedback in real time.

https://www.zdnet.com/article/microsoft-researchers-are-using-chatgpt-to-instruct-robots-and-drones/

Here is a related video of ChatGPT responding verbally via a humanoid robot. This is different from ChatGPT because it responds verbally. ChatGPT was released as text only.

In the video, ChatGPT says that humanoid robots will be a multibillion-dollar industry by 2030.

[edisdead]

1. I have used it to give me example code

It has been extremely useful in that sense. Rather than searching through stackoverflow questions and responses I may ask how to look over a key value pair in php or some other language just to use as an illustration or even more complex coding examples.

[Renew Deal]

2. I've heard about people asking it to make their own code more efficient.

And it supposedly does well at that.

[edisdead]

3. it does!

A friend of mine has used it like that as well. I just like to tinker with my coding a little more.

[highplainsdem]

4. Sigh. That Engineered Arts robot isn't a Microsoft project.

And I posted about the research Microsoft is doing two days ago and included the correct video for the story.

That post: https://www.democraticunderground.com/100217671422

You were misled by a retitled/mistitled video posted in the last 24 hours and getting the facts wrong, scrambling that new headline with a video from Engineered Arts several months ago.

They did use GPT3 to have ths robot, Ameca, respond to questions. But they weren't working with Microsoft.

Here's the original video:

Article about Engineered Arts from The Verge:

https://www.theverge.com/23054881/engineered-arts-robotic-humanoid-ai-ameca-artificial-intelligence

Microsoft's blog post on their own research:

https://www.microsoft.com/en-us/research/group/autonomous-systems-group-robotics/articles/chatgpt-for-robotics/

[Pachamama]

5. This must be how it started before Terminator....

I actually find this terrifying…