AI can write essays, generate images, and explain how the physical world works. But explaining how the world works is different from predicting what will happen in a real situation.
Imagine a humanoid robot helping with household chores. Its job is to tidy up the living room, pick up objects, and put them where they belong.
The robot can recognize a cup on the table. But what happens if the cup is near the edge? What if the robot accidentally knocks it? Or what if someone walks in front of it while it is carrying the cup?
To handle these situations, the robot needs more than object recognition. It needs to predict how objects and the environment might change when it takes an action.
This is the idea behind a World Model: an AI model that learns how the world changes over time and predicts what might happen next.
Credit: Unitree General-Purpose Humanoid Foundation Model Fully Upgrade Major Open Source (YouTube)
How Do World Models Work?
A World Model learns patterns from observations and experience, such as how objects move, how they interact, and how the environment changes when an action is taken.
Let’s return to our household robot.
Imagine the robot needs to pick up a cup from a table. Before acting, it can use its World Model to predict what might happen in different situations.
Picking up the cup: The robot predicts how the cup will move as it lifts it.
Moving around furniture: It predicts whether its path is clear or if it might bump into something.
Putting the cup down: It predicts where the cup will land and whether the surface is stable.
These predictions can help the robot compare different actions and choose a better way to complete its task.
World Models can also help robots reason about things they cannot currently see. For example, if someone walks behind a sofa and disappears from view, the model can use previous observations to estimate where that person might be.
Of course, these predictions are not always correct. A World Model can make mistakes, especially in unfamiliar situations. But even imperfect predictions can help a robot make better decisions.
Credit: Unitree General-Purpose Humanoid Foundation Model Fully Upgrade Major Open Source (YouTube)
Why Do We Need World Models?
Large Language Models (LLMs) can explain how the world works. They can tell you that a cup dropped on the floor may break. But predicting exactly how that cup will fall, where it will land, and whether it will hit something else is a different challenge.
For robots, this difference matters. They need to do more than recognize objects or describe situations. They need to predict possible outcomes and use those predictions to guide their actions.
World Models offer a way to help AI systems learn patterns about movement, physical interactions, and cause and effect. They can also act as internal simulators, allowing AI to evaluate possible actions before trying them in the real world.
This could help robots adapt to different environments and handle situations they have not encountered before. Instead of manually programming a response for every possible situation, developers can train models to learn useful patterns from data and experience.
World Models are still an active area of research, but they could play an important role in building more capable Physical AI—AI that can perceive, reason about, and interact with the physical world.
Ready to Build More Intelligent Robots?
Understanding World Models is one step toward understanding how AI can help robots make better decisions in the physical world. To put these ideas into practice, you also need to understand the technologies that help robots perceive, learn, and act.
The Physical AI Robotics Masterclass by The Construct takes you from AI foundations to developing full-stack robotics expertise, equipping you with the skills to design, train, and deploy robots with embodied intelligence.
Reinforcement learning (RL) is a way to train AI to make decisions by learning from experience. Instead of following a fixed set of instructions, an AI tries different actions, observes what happens, and learns which choices lead to better results.
Imagine teaching a robot to walk without telling it exactly how to move. It tries, falls, adjusts, and tries again. Over time, it can learn to move more effectively.
That’s the basic idea behind reinforcement learning: learn by doing and improve through feedback.
1. How Does Reinforcement Learning Work?
Let’s meet Bob, a robot with a simple mission: reach a battery across the room.
To complete this task, Bob needs four things:
Agent: The learner making decisions—in this case, Bob.
Environment: The world Bob interacts with, including the room, walls, and battery.
Actions: What Bob can do, such as moving forward or turning left.
Rewards: Feedback that tells Bob whether an outcome is desirable.
We can give Bob +10 points for reaching the battery, −5 for hitting a wall, and −0.1 for every second he wastes.
At first, Bob will probably make plenty of mistakes. He might turn the wrong way or run straight into a wall. But each attempt gives him experience, helping him discover which actions work better.
Over time, Bob can learn a strategy for choosing what to do in different situations. In reinforcement learning, this strategy is called a policy.
In simple terms, a policy answers one question: Given what I know right now, what should I do next?
2. Why Not Just Program the AI?
For simple tasks, we can give a robot explicit instructions. But real-world environments are unpredictable. Furniture moves, floors get slippery, and unexpected obstacles appear.
Trying to program a rule for every possible situation quickly becomes complicated.
Reinforcement learning offers another approach: define a goal, provide feedback through rewards, and let the AI learn a strategy through experience.
This doesn’t mean the AI will automatically figure everything out. Its success depends on the learning method, the quality of the reward system, and the experiences it receives. But for tasks involving many possible actions and changing conditions, it can be a powerful alternative to manually programming every decision.
There’s one catch, though: learning through trial and error can be expensive and potentially dangerous when using real robots.
That’s why researchers often use simulation. A virtual environment lets robots practise, fail, and try again without damaging physical hardware. Researchers can also reset the environment quickly and test different conditions. Once a strategy is learned, they can evaluate it on a real robot, although some adjustments may still be needed.
3. Where Is Reinforcement Learning Used?
Reinforcement learning is useful when an AI needs to make decisions over time and its actions affect what happens next.
Some common applications include:
Robotics: Learning to grasp objects, manipulate items, and control movement.
Navigation and walking: Finding routes, avoiding obstacles, and learning how to move.
Games: Developing strategies by playing repeatedly and learning from the results.
Decision-making: Choosing actions that help achieve longer-term goals.
The key difference is that recognizing a situation and deciding what to do are two different problems.
For example, identifying a wall is one thing. Learning how to move around it efficiently is another. Reinforcement learning can help AI tackle the second challenge.
4. So, Why Does AI Need Reinforcement Learning?
AI doesn’t just need to recognize patterns or make predictions. In many applications, it also needs to take actions and learn from their consequences.
Reinforcement learning provides a way to develop that decision-making ability through experience. Rather than relying entirely on fixed instructions, an agent learns which actions tend to lead to better outcomes.
It’s not the right tool for every AI problem, and training can require many attempts. Reward systems also need to be designed carefully, because an AI may learn unexpected behaviors if the rewards don’t reflect the real goal.
But for robotics and other tasks where decisions have consequences, reinforcement learning offers a valuable way to teach AI how to act.
In a nutshell: An agent takes actions, receives rewards, and learns a policy that helps it make better decisions over time.
Understanding reinforcement learning is a starting point, but applying these principles to physical systems introduces new challenges. This is where Physical AI comes in, combining AI with the ability to perceive, interact with, and act in the real world.
The Physical AI Robotics Masterclass takes you from AI foundations to developing full-stack robotics expertise, equipping you with the skills to design, train, and deploy robots with embodied intelligence.
In the 1980s, companies spent billions of dollars on software that promised to capture and reproduce human expertise. Businesses invested heavily in expert systems, governments funded AI initiatives, and specialized hardware was built around the idea that machines could reason like human experts.
Then, within a few years, much of that industry collapsed.
It became known as the AI Winter. And if you look at what is happening with large language models today, the comparison is surprisingly interesting.
The technology is completely different. But underneath the buzzwords, there is a similar idea: take human knowledge, encode it into a machine, and build a system that can use that knowledge to answer questions and make decisions.
So what exactly were expert systems, and what do they have to do with LLMs?
What Were Expert Systems?
The idea behind an expert system was relatively simple. Instead of trying to build a machine that could think about everything, researchers focused on something narrower: capture the knowledge of one specific human expert and turn it into software.
Imagine a doctor who specializes in blood infections. A knowledge engineer would interview that doctor over weeks or months, trying to understand how they make decisions.
The doctor might explain something like: if a patient has a fever, the bacteria is gram-negative, and the infection is in the bloodstream, then certain antibiotics should be considered.
The knowledge engineer would turn this kind of reasoning into explicit if-then rules. Thousands of these rules could be written down and stored in a knowledge base.
An inference engine would then connect those rules together and use them to answer new questions.
The basic architecture was:
Knowledge Base + Inference Engine = Expert System
There was no neural network learning from data and no statistical model discovering patterns on its own. The system was essentially a very large, carefully constructed rulebook built from human expertise.
And for a while, it looked like magic.
The Success Stories That Sold the Dream
Several early systems demonstrated that expert systems could do surprisingly sophisticated work.
DENDRAL, developed at Stanford, could analyze chemical data and help determine molecular structures — a task that traditionally required an expert organic chemist.
Then came MYCIN, a system designed to diagnose blood infections and recommend antibiotics. In controlled tests, it performed as well as, and sometimes better than, junior doctors. MYCIN was never actually deployed in hospitals, largely because of the legal and ethical questions surrounding machine-assisted medical diagnosis.
Then there was PROSPECTOR, developed at SRI International to capture the reasoning of expert geologists.
PROSPECTOR could take observations about rock formations, mineral samples, and geological structures and reason through whether a site was worth investigating. In 1980, it analyzed a site at Mount Tolman in Washington State and identified a molybdenum deposit that human geologists had missed. The deposit was later estimated to be worth around $100 million.
That was the kind of result that made the promise of expert systems hard to ignore: a machine using hand-written rules had produced a real-world result with significant economic value.
The industry took off. Companies started selling expert-system software, specialized Lisp machines were developed to run it, business schools taught the technology, and governments funded AI initiatives at a massive scale.
For a while, it seemed like machines that could reproduce human expertise were finally becoming a reality.
DENDRAL was developed in 1965 by Edward Feigenbaum, also referred to as the “father of expert systems”, and by Joshua Lederberg at Stanford University in California.
Why Did Expert Systems Fall Apart?
The problem was not that expert systems were useless. The problem was that building and maintaining them at scale was extremely difficult.
The first challenge was knowledge acquisition. Every rule had to come from somewhere, and that usually meant extracting it from a human expert. Interviewing experts for weeks or months just to build one system was slow and expensive.
The second problem was brittleness. Expert systems could be extremely good within the situations their rules covered, but they struggled when they encountered something outside those rules. A system might perform impressively on the cases it was designed for and fail badly when the situation changed.
Then came maintenance. Thousands of rules were connected to one another, so changing one rule could have unexpected effects somewhere else. As systems grew, keeping everything consistent became increasingly difficult and expensive.
Finally, the industry made a major bet on specialized hardware. Many expert systems ran on expensive Lisp machines. As general-purpose computers became powerful enough to perform similar tasks at much lower cost, much of the hardware industry built around expert systems disappeared.
By the early 1990s, funding had largely dried up. Expert systems did not disappear completely — some continued to be used for practical, narrow applications — but the larger vision of expert systems as the foundation for thinking machines had largely collapsed.
Then, decades later, something familiar appeared.
Enter the LLMs
Strip away the modern terminology, and a large language model can be viewed as doing something conceptually similar: compressing human knowledge into a form that a machine can use to answer questions.
The difference is how that knowledge gets collected and represented.
Think back to the doctor example.
An expert system might have a knowledge engineer ask a doctor: What do you do when you see these five symptoms together? The doctor’s answer becomes a rule.
An LLM approaches the problem at a completely different scale. Instead of interviewing one doctor, its training data can contain medical textbooks, articles, online discussions, and countless other examples of human-written knowledge. Instead of a person explicitly writing “if X, then Y,” the model learns statistical patterns from the data.
That is the “on steroids” part.
It is not the same technology as an expert system. An LLM does not contain a giant collection of human-written if-then rules. But the broader idea — taking human knowledge and encoding it into a machine that can be queried — has a meaningful conceptual connection.
And there is another important similarity.
Neither system is a human mind.
An expert system can chain together rules without knowing what a “blood infection” actually is. An LLM generates text by predicting likely sequences based on patterns learned during training. Both can produce remarkably useful results without thinking about the world in the same way a person does.
Will LLMs Crash Like Expert Systems Did?
This is where the comparison becomes more complicated.
It would be too simple to assume that LLMs will simply repeat the history of expert systems.
Expert systems mainly created value inside specific organizations and industries. PROSPECTOR could help a mining company find a valuable mineral deposit, but ordinary people had little reason to interact with it.
LLMs are different because language is a universal interface. You don’t need to be a chemist, geologist, or engineer to get value from a system that can write, summarize, explain, or generate code.
That gives LLMs a much broader reach than expert systems ever had.
But the historical comparison is still useful.
The lesson from expert systems is not that AI systems are destined to fail. It is that we should distinguish between a system that produces valuable results and a system that actually thinks or understands like a human.
LLMs are extraordinarily powerful and useful. But their usefulness does not automatically mean they are minds.
From AI That Knows to AI That Acts
The story of expert systems and LLMs is ultimately a story about how we turn human knowledge into something machines can use.
But AI doesn’t stop at language. The same advances in perception, learning, reasoning, and foundation models are now being applied to robotics, enabling machines to learn from data, understand their surroundings, and perform physical tasks.
This is Physical AI — bringing AI into embodied systems that allow robots to perceive, learn, and perform advanced tasks in the real world. And that is exactly what the new Physical AI Robotics Masterclassfrom The Construct Robotics Institute is designed to teach: from AI foundations to becoming a full-stack AI robotics engineer, designing, training, and deploying robots with embodied intelligence.
This Masterclass is delivered 100% online, combining structured online courses, hands-on practice, and independent real robotics projects to help you build the knowledge, skills, and practical experience needed for a career in AI robotics.
Artificial intelligence is increasingly moving beyond software environments and into the physical world.
Robots need to do more than execute predefined commands. They need to perceive their surroundings, understand instructions, learn from data, make decisions, and take physical actions. This shift is driving the development of Physical AI—AI systems designed to operate and learn in the real world.
To help robotics developers build these capabilities, The Construct is launching the New Physical AI Robotics Masterclass, an online, hands-on, self-paced program that takes learners from AI foundations to a full-stack AI robotics engineer, designing, training, and deploying robots with embodied intelligence.
What Does the Physical AI Robotics Masterclass Include?
The Physical AI Robotics Masterclass is a career-focused learning path designed to take you from AI foundations to full-stack AI robotics engineering.
Becoming an AI Robotics Engineer requires more than learning individual AI models or robotics frameworks. It requires a combination of AI knowledge, robotics skills, software development, model training, robot control, and hands-on experience with real robotic systems.
The Masterclass brings these areas together in one structured program, with each stage building the knowledge and practical skills needed for the next.
Phase 1: AI Foundations for Robotics
The first phase builds a strong foundation in AI for robotics, developing practical skills in machine learning, deep learning, PyTorch, and reinforcement learning.
Topics include:
AI Foundations for Robotics
Data Augmentation
Basic Machine Learning for Robotics
Deep Learning Basics
PyTorch Essentials for Robotics
Mastering RL for Robotics
The objective is to provide a practical understanding of the core AI techniques that are later applied to robotic systems.
Rather than treating AI as a separate discipline from robotics, this phase establishes the connection between AI methods and robotic applications.
Phase 2: Intermediate AI for Robotics
Once the foundational concepts are established, this phase develops practical skills in building, training, and deploying AI models for robotic perception and language understanding.
This phase covers:
On-Device AI for Robotics
Generative AI for Robotics
Intermediate Generative AI for Robotics
AI Agents
These technologies introduce new approaches to how robots can process information, interact with their environment, and respond to higher-level instructions.
The focus moves from understanding individual AI techniques toward combining them into systems that can be applied to robotics.
Phase 3: Advanced AI for Robotics
The final phase focuses on advanced Physical AI techniques for robot learning and control, building practical skills in reinforcement learning, autonomous robot learning, and complex manipulation tasks.
Topics include:
MuJoCoLab for Robotics AI
Perception with modern AI
RL for Robot Control
Imitation Learning for Robotics
Vision-Language-Action Model (VLA)
Mobile ALOHA
This phase brings together perception, learning, and control to address more complex robotics tasks.
You’ll work with both simulated environments and real robotic systems, providing a path from experimentation in simulation to practical deployment.
PLUS: A Real Robotic Arm Kit
During Phase 3, you’ll receive a real Embodied Intelligence Open-Source Robotic Arm Kit for your final project.
You’ll build your own robotic system and implement Vision-Language-Action (VLA) models on the robot to complete real-world tasks.
From Simulation to Real Robots
Throughout the Masterclass, You’ll apply what you learn through hands-on practice.
The program combines simulation, remote robot labs, and real robotic systems, allowing you to work in realistic robotics environments, gain practical experience, and build the skills needed to solve real-world robotics problems.
Remote robotics lab environments in the Physical AI Robotics Masterclass:
1. FASTBOT City Lab
Train a mobile robot to navigate and perform tasks in a real miniature city.
2. Warehouse Lab
Create real automation tasks with collaborative warehouse robots.
3. Cyberworld Lab
Design autonomy algorithms for a wheeled robot in a cyber-physical arena.
4. Robot Coffee Lab
Deploy AI in a real café with a humanoid and collaborative arm.
What You Will Achieve
By completing the program, you will be able to:
Design Physical AI systems for robotics
Apply reinforcement learning to robot control
Build vision–language–action systems
Develop agent-based robotic autonomy
Deploy on-device AI for real robots
Prepare for a Career in AI Robotics
The Physical AI Robotics Masterclass is delivered 100% online and combines structured online courses, hands-on practice, and independent robotics projects to help you build the knowledge, skills, and practical experience needed for a career in AI robotics.
Self-Paced Learning: The flexible learning format allows you to balance your studies with work and personal commitments while progressing through the program at your own pace.
1-on-1 Tutor Support: Throughout the Masterclass, you’ll receive 1-on-1 guidance from industry experts, with support throughout your learning process and practical projects.
Certificate: Upon successful completion of the Masterclass, you’ll receive a Physical AI Robotics Masterclass Certificate, which can be shared on LinkedIn as part of your professional profile.
Portfolio: Complete the Masterclass with a portfolio of real-world projects that showcases your AI robotics skills and practical experience.
If you want to become a full-stack AI robotics engineer, the Physical AI Robotics Masterclass provides a structured learning path from AI foundations to advanced Physical AI, combining simulation, remote robot labs, and hands-on work with real robotic systems.
From understanding how AI works to deploying intelligence on real machines, you’ll build the knowledge, technical skills, and practical experience needed to develop AI-powered robots!
