RoboticsHackathonDemo

Using social signals to make robots more human-aware

Controlling robots shouldn't require joysticks. WaveArm uses a webcam to turn gestures and behaviors into real-time robot commands, enabling natural, safe human-robot interaction.

·5 min read

In assistive settings, such as supporting elderly users or people with residual motor function, controlling a robotic arm in a natural and intuitive way remains a significant challenge. Everyday actions like picking up a glass, pressing a button, or interacting with objects in a human environment require more than traditional joystick-based control and might not be feasible when motor function is limited. But safety and awareness matter just as much as control.

Author: Paula Petcu

WaveArm logo

WaveArm uses gesture-based input to control the robot arm and combines that with Interhuman AI as a safety or decision layer. In the system architecture, the robot should not only detect that a person is making a movement, but also assess whether the person appears engaged and capable of controlling the arm safely. The solution includes a simple interface on the screen, with immediate visual feedback, using green/red signals to indicate whether movement is viable.

Gesture-based control with immediate visual feedback on screen

WaveArm system architecture: hand-tracking camera, user-attention camera, feedback screen, and robotic arm

The project was born at the Robotic Agents Hackathon in Milan in June 2026, where Interhuman AI sponsored a challenge around a simple but important question: how can social signals be used to make robots more human-aware?

A Robot Arm That Pays Attention

In robotics, "stop" is one of the most important commands.

But in real human-robot interaction, people do not always say "stop" clearly or in time. They might hesitate. They might look uncertain. They might become tense. They might show discomfort before they verbalize it.

The team integrated social signal analysis into their robot arm demo, using human signals as part of the decision layer for the robot arm's movements.

This is the kind of use case we are excited about at Interhuman AI: giving robots access to the social context around their actions.

Why This Matters for Robotics

Robots are increasingly expected to work around people: in labs, hospitals, classrooms, homes, factories, and public spaces.

In those settings, physical intelligence is not enough. A robot can know where an object is, how to move an arm, or how to follow a task sequence. But it also needs to understand the person in the interaction.

Are they confident?

Are they uncertain?

Are they paying attention?

Are they uncomfortable?

Should the robot continue, pause, slow down, or stop?

That is where social signal analysis becomes useful.

With Interhuman AI, developers can start adding a human-awareness layer to robotic systems. Instead of relying only on explicit commands, the robot can also respond to signals in the interaction.

The Team Behind WaveArm

As with many robotics challenges, making the physical robot arm behave reliably is already a big challenge. We were impressed by how much the team achieved in a short amount of time, and that they treated social intelligence not as a nice-to-have feature, but as part of the robot's control logic.

The WaveArm team receiving their prize at the Robotic Agents Hackathon in Milan

The WaveArm project was built by Gianluca Morotti, Matteo Magnani, Andrea Montalbano, and Elia Coppiardi.

The WaveArm team brought together robotics, software, and AI integration under tight time pressure. Their project showed the kind of fast experimentation that hackathons are best at: taking a big idea and making it tangible.

Elia: "We started with the idea of controlling a robot without a joystick, just by tracking arm gestures. While building it, we came up with the idea to mount it on a wheelchair for people with limited motor function."

Once they thought of that, they couldn't let go of building for a good purpose: "This technology is designed for elderly individuals and people with minimal residual motor function who can no longer perform basic everyday tasks. A critical factor in its application, however, is the user's cognitive state. Operating a powerful device like a robotic arm carries inherent risks, requiring the user to be fully aware of their actions. Therefore, a robust control system is fundamental; if the user becomes distracted or confused, it could lead to severe safety risks."

The toughest part of the implementation? Transitioning from the simulation to the real hardware at the last minute, under stress, on a local area network where 50 other people were trying to connect to the physical robots. On the other hand, integrating Interhuman AI was remarkably smooth thanks to its accessible design. Gianluca shares: "Implementing it was straightforward because the system is so easy to work with. This allowed us to really focus on the GUI, which was crucial. For the user, it was important to see immediate visual feedback, like a green or red light, to know if the movement was viable."

The WaveArm team presenting their demo with the robot arm

Next for the project is making the robot movements smoother, with less acceleration, and explore the integration into a wheelchair or a hospital bed. And as it is with early stage products, iterate and pivot until the product market fit is clear.

Elia is excited about the future: "Right now, the robotics field is exploding from both a technological and a market perspective. Since we are going to interact more and more with robots, it will be crucial to embed Interhuman AI, or a similar system, capable of understanding the humans interacting with them. Technology like yours will be essential for our relationship with robots and humanoids in this rapidly emerging field."

A Glimpse of Human-Aware Robots

WaveArm started as a hackathon project, but the underlying idea is serious.

As robots become more present in human environments, they need better ways to interpret what is happening socially. A person's reaction can be just as important as a command.

For Interhuman AI, WaveArm is a strong example of how social signals can become an actionable input for robotics.

That is a direction we are excited to support. The WaveArm team has received our first Interhuman AI robotics kit, plus 12-months in Interhuman AI credits, and our support for bringing this project forward. This is exactly the kind of robotics application we believe will become increasingly important as AI systems move from screens into the physical world.

Demo video: https://www.youtube.com/watch?v=32ebCxhgnBs

GitHub: https://github.com/AndreaMontalbano/Hackathon-Robotica

DevPost: https://devpost.com/software/wavearm