Category: breakthrough

This rapid growth has raised concerns about sustainability. In response, researchers at a School of Engineering have created a proof-of-concept AI system designed to be far more efficient. Their approach could reduce energy use by up to 100 times while also improving performance on tasks.

A Hybrid Approach Called Neuro-Symbolic AI

The research comes from the laboratory of Matthias Scheutz, Karol Family Applied Technology Professor. His team is developing neuro-symbolic AI, which combines traditional neural networks with symbolic reasoning. This method mirrors how people approach problems by breaking them into steps and categories.

The work will be presented at the International Conference of Robotics and Automation in Vienna in May and will appear in the conference proceedings.

Teaching Robots to See, Understand, and Act

Unlike familiar large language models (LLMs) such as ChatGPT and Gemini, the team focuses on AI systems used in robotics. These systems are known as visual-language-action (VLA) models. They extend LLM capabilities by incorporating vision and physical movement.

VLA models take in visual data from cameras and instructions from language, then translate that information into real-world actions. For example, they can control a robot’s wheels, arms, or fingers to complete a task.

Why Traditional AI Struggles With Simple Tasks

Conventional VLA systems rely heavily on data and trial-and-error learning. If a robot is asked to stack blocks into a tower, it must first analyze the scene, identify each block, and determine how to place them correctly.

This process often leads to mistakes. Shadows may confuse the system about a block’s shape, or the robot may place pieces incorrectly, causing the structure to collapse.

These errors are similar to the problems seen in LLMs. Just as robots can misplace blocks, chatbots can generate false or misleading outputs. Examples include fabricating legal cases or producing images with unrealistic details such as extra fingers.

How Symbolic Reasoning Improves Accuracy and Efficiency

Symbolic reasoning offers a different strategy. Instead of relying only on patterns from data, it uses rules and abstract concepts such as shape and balance. This allows the system to plan more effectively and avoid unnecessary trial and error.

“Like an LLM, VLA models act on statistical results from large training sets of similar scenarios, but that can lead to errors,” said Scheutz. “A neuro-symbolic VLA can apply rules that limit the amount of trial and error during learning and get to a solution much faster. Not only does it complete the task much faster, but the time spent on training the system is significantly reduced.”

Strong Results in Puzzle Tests

The researchers tested their system using the Tower of Hanoi puzzle, a classic problem that requires careful planning.

The neuro-symbolic VLA achieved a 95% success rate, compared with just 34% for standard systems. When given a more complex version of the puzzle that it had not encountered before, the hybrid system still succeeded 78% of the time. Traditional models failed every attempt.

Training time also dropped sharply. The new system learned the task in only 34 minutes, while conventional models required more than a day and a half.

Massive Energy Savings in Training and Use

Energy consumption was reduced dramatically as well. Training the neuro-symbolic model required only 1% of the energy used by a standard VLA system. During operation, it used just 5% of the energy needed by conventional approaches.

Scheutz compared this inefficiency to everyday AI tools. “These systems are just trying to predict the next word or action in a sequence, but that can be imperfect, and they can come up with inaccurate results or hallucinations. Their energy expense is often disproportionate to the task. For example, when you search on Google, the AI summary at the top of the page consumes up to 100 times more energy than the generation of the website listings.”

The Growing Strain of AI on Power Infrastructure

As AI adoption accelerates across industries, demand for computing power continues to climb. Companies are building increasingly large data centers, some of which require hundreds of megawatts of electricity. That level of consumption can exceed the needs of entire small cities.

This trend has sparked a race to expand infrastructure, raising concerns about long-term energy limits.

A More Sustainable Path for AI

The researchers suggest that current approaches based on LLMs and VLAs may not be sustainable in the long run. While these systems are powerful, they consume large amounts of energy and can still produce unreliable results.

In contrast, neuro-symbolic AI offers a different direction. By combining learning with structured reasoning, it may provide a more efficient and dependable foundation for future AI systems.

Humans place enormous importance on facial expression, especially subtle movements of the lips. While awkward walking or clumsy hand gestures can be forgiven, even small mistakes in facial motion tend to stand out immediately. This sensitivity contributes to what scientists call the “Uncanny Valley,” a phenomenon where robots appear unsettling rather than lifelike. Poor lip movement is a major reason robots can seem eerie or emotionally flat, but researchers say that may soon change.

A Robot That Learns to Move Its Lips

On January 15, a team from Columbia Engineering announced a major advance in humanoid robotics. For the first time, researchers have built a robot that can learn facial lip movements for speaking and singing. Their findings, published in Science Robotics, show the robot forming words in multiple languages and even performing a song from its AI-generated debut album “hello world_.”

Rather than relying on preset rules, the robot learned through observation. It began by discovering how to control its own face using 26 separate facial motors. To do this, it watched its reflection in a mirror, then later studied hours of human speech and singing videos on YouTube to understand how people move their lips.

“The more it interacts with humans, the better it will get,” said Hod Lipson, James and Sally Scapa Professor of Innovation in the Department of Mechanical Engineering and director of Columbia’s Creative Machines Lab, where the research took place.

See link to “Lip Syncing Robot” video below.

Robot Watches Itself Talking

Creating natural-looking lip motion in robots is especially difficult for two main reasons. First, it requires advanced hardware, including flexible facial material and many small motors that must operate quietly and in perfect coordination. Second, lip movement is closely tied to speech sounds, which change rapidly and depend on complex sequences of phonemes.

Human faces are controlled by dozens of muscles located beneath soft skin, allowing movements to flow naturally with speech. Most humanoid robots, however, have rigid faces with limited motion. Their lip movements are typically dictated by fixed rules, which leads to mechanical, unnatural expressions that feel unsettling.

To address these challenges, the Columbia team designed a flexible robotic face with a high number of motors and allowed the robot to learn facial control on its own. The robot was placed in front of a mirror and began experimenting with thousands of random facial expressions. Much like a child exploring their reflection, it gradually learned which motor movements produced specific facial shapes. This process relied on what researchers call a “vision-to-action” language model (VLA).

Learning From Human Speech and Song

After understanding how its own face worked, the robot was shown videos of people talking and singing. The AI system observed how mouth shapes changed with different sounds, allowing it to associate audio input directly with motor movement. With this combination of self-learning and human observation, the robot could convert sound into synchronized lip motion.

The research team tested the system across multiple languages, speech styles, and musical examples. Even without understanding the meaning of the audio, the robot was able to move its lips in time with the sounds it heard.

The researchers acknowledge that the results are not flawless. “We had particular difficulties with hard sounds like ‘B’ and with sounds involving lip puckering, such as ‘W’. But these abilities will likely improve with time and practice,” Lipson said.

Beyond Lip Sync to Real Communication

The researchers stress that lip synchronization is only one part of a broader goal. Their aim is to give robots richer, more natural ways to communicate with people.

“When the lip sync ability is combined with conversational AI such as ChatGPT or Gemini, the effect adds a whole new depth to the connection the robot forms with the human,” said Yuhang Hu, who led the study as part of his PhD work. “The more the robot watches humans conversing, the better it will get at imitating the nuanced facial gestures we can emotionally connect with.”

“The longer the context window of the conversation, the more context-sensitive these gestures will become,” Hu added.

Facial Expression as the Missing Link

The research team believes that emotional expression through the face represents a major gap in current robotics.

“Much of humanoid robotics today is focused on leg and hand motion, for activities like walking and grasping,” Lipson said. “But facial affection is equally important for any robotic application involving human interaction.”

Lipson and Hu expect realistic facial expressions to become increasingly important as humanoid robots are introduced into entertainment, education, healthcare, and elder care. Some economists estimate that more than one billion humanoid robots could be produced over the next decade.

“There is no future where all these humanoid robots don’t have a face. And when they finally have a face, they will need to move their eyes and lips properly, or they will forever remain uncanny,” Lipson said.

“We humans are just wired that way, and we can’t help it. We are close to crossing the uncanny valley,” Hu added.

Risks and Responsible Progress

This work builds on Lipson’s long-running effort to help robots form more natural connections with people by learning facial behaviors such as smiling, eye contact, and speech. He argues that these skills must be learned through observation rather than programmed through rigid instructions.

“Something magical happens when a robot learns to smile or speak just by watching and listening to humans,” he said. “I’m a jaded roboticist, but I can’t help but smile back at a robot that spontaneously smiles at me.”

Hu emphasized that the human face remains one of the most powerful tools for communication, and scientists are only beginning to understand how it works.

“Robots with this ability will clearly have a much better ability to connect with humans because such a significant portion of our communication involves facial body language, and that entire channel is still untapped,” Hu said.

The researchers also acknowledge the ethical concerns that come with creating machines that can emotionally engage with humans.

“This will be a powerful technology. We have to go slowly and carefully, so we can reap the benefits while minimizing the risks,” Lipson said.