How Morphology Shapes Perception, Action, and Learning

Welcome back. In the previous lesson, we expressed embodied decision-making as a POMDP: the robot acts in a true physical state it cannot fully observe, receives sensor observations, and maintains uncertainty through a belief state. That formulation is necessary—but it is not yet enough. We also need to ask a prior question: what kind of body is making the observations and producing the actions?

This lesson examines morphology: the robot’s physical form, geometry, joints, materials, actuators, sensors, and contact surfaces. Morphology is not just packaging around an AI policy. It determines what information reaches the policy, which commands can cause meaningful changes in the world, and which useful behaviors can be learned reliably. By the end, you should be able to analyze a robot body as a set of constraints and opportunities for perception, action, and learning.

---

Morphology belongs inside the agent–environment loop

For physical AI, morphology includes at least:

• Geometry: body size, link lengths, shape, mass distribution, and reachable workspace.
• Kinematics: joints, degrees of freedom, joint limits, and mechanical couplings.
• Actuation: motors, tendons, pneumatic chambers, torque and speed limits, compliance.
• Sensing: sensor types, placement, field of view, resolution, noise, and latency.
• Materials and contact surfaces: stiffness, friction, damping, elasticity, texture, and deformation.
• Mechanical constraints: wheels that do not slide sideways, tendon routing, passive springs, or an underactuated hand’s coupled fingers.

We can represent a morphology abstractly by $m$. In the POMDP language from last lesson, morphology changes both the observation model and the transition model:

$$o_t\sim O_m(\cdot \mid s_t)$$ $$s_{t+1}\sim T_m(\cdot \mid s_t,u_t)$$

Here, $u_t$ is the command issued by a controller: motor torque, wheel speed, valve pressure, or another actuator-level input. The subscript $m$ matters. The same world state can generate very different observations for a forward-facing camera, a wrist camera, and a tactile sensor. The same nominal motor command can also have different effects for a rigid arm, a wheeled base, and a soft pneumatic gripper.

A useful discipline is to distinguish three questions:

Question	What morphology determines
What can the agent observe?	Which task-relevant state variables leave detectable traces in its sensors
What can the agent do?	Which state changes are physically feasible, safe, and repeatable
What can it learn practically?	Which policies can be discovered with available data, exploration, and model capacity

These are related, but not identical. A robot may be physically capable of a maneuver that it cannot learn reliably because it cannot sense the conditions under which that maneuver succeeds. Conversely, a compliant gripper may have only a low-dimensional action input, yet learn robust grasps because its body absorbs geometric uncertainty.

---

Morphology constrains observations

A sensor does not observe “the environment.” It measures a physical quantity from a particular place, orientation, and mechanism.

A fixed monocular camera, for example, provides a perspective projection of visible surfaces. It cannot directly determine depth from one image without learned or assumed structure; it cannot see behind occluders; and it loses information outside its field of view. Adding a second camera, moving the camera to the wrist, or adding a depth sensor changes the observation process itself.

Likewise, a joint encoder offers relatively direct information about a joint angle, but not necessarily about the end-effector’s contact state. A force sensor at the wrist detects aggregate wrench information, but may not tell which fingertip is slipping. A tactile array can localize contact over a finger surface, but typically only after the finger has made contact.

The key idea is observational distinguishability. Suppose two physical states $s^{(1)}$ and $s^{(2)}$ generate nearly identical sensor readings:

$$O_m(o\mid s^{(1)})\approx O_m(o\mid s^{(2)})$$

Then a reactive policy receiving only $o$ has little basis for choosing different actions in those two states. It may need memory, a new viewpoint, an active probing motion, or an additional sensor.

Sensor placement is a design decision

The same sensor technology can be far more or less useful depending on placement.

• A camera above a workbench gives broad scene coverage but can suffer from hand-induced occlusion during grasping.
• A wrist camera can inspect an object closely and change viewpoint through arm motion, but its view moves during action and may be blocked by the gripper.
• A force sensor at the wrist may detect that contact occurred, while fingertip tactile sensors can reveal asymmetric contact and incipient slip.
• A microphone on a mobile robot may detect a sound, but an array with spatial separation can estimate a sound direction.

Thus, an observation space should be based not on what would be convenient for a policy, but on what the body and sensors can physically produce.

Mechanical structure can improve sensing

Morphology may amplify or filter a signal before electronics or software process it. Fingertip ridges, for example, can change both contact friction and the strain patterns that tactile receptors receive. Whiskers convert small distant contacts or air disturbances into larger bending signals at their base.

This does not mean a clever body eliminates the need for perception. It means perception is distributed across the sensing material, geometry, and electronics. For a physical-AI system, the observation pipeline begins at contact geometry and material deformation—not only at the neural-network input tensor.

---

Morphology constrains actions and reachable behavior

An action space lists the commands that a controller may issue. Morphology determines whether those commands provide enough control authority to achieve a goal.

A useful distinction:

• Command space: inputs the software may send, such as two wheel angular velocities or one pneumatic pressure.
• Physical capability: motions and forces the whole robot can actually produce, given its geometry, dynamics, limits, and environment.

A differential-drive robot can command left and right wheel speeds. Yet, because its wheels are designed not to slide sideways, it cannot instantaneously move laterally. It must reorient and follow a curved path. A car can reach a parking spot that is beside it, but not by translating sideways in a single step.

Such constraints are often expressed as nonholonomic constraints: they restrict how configurations can be reached, rather than merely forbidding configurations outright. This is why a rectangular free-space map is not enough to plan for a wheeled robot. A path must respect the robot’s turning and motion constraints.

Actuator count and limits matter as well. Consider a mechanical system with configuration $q$, velocity $\dot{q}$, and control $u$:

$$\ddot{q}=f_1(q,\dot{q})+f_2(q,\dot{q})u$$

If the agent has fewer independent control directions than configuration directions, it is underactuated. Formally, at a given state, this occurs when

$$\mathrm{rank}(f_2)<\dim (q)$$

A fully actuated industrial arm can often command torque at each joint and counter disturbances within its operating range. A legged robot in flight cannot directly change the ballistic trajectory of its center of mass; it can reposition limbs, but it cannot produce a ground reaction force until contact returns. A passive pendulum has even less authority: its motion comes primarily from gravity, inertia, and its mechanical design.

Bodies can make a behavior easier—or impossible

A body constrains behavior in at least four ways:

1. Reachability.
   Link lengths and joint limits determine whether an arm can reach around an obstacle or grasp an object at the back of a shelf.

2. Force and speed.
   Torque limits, gear ratios, pneumatic flow limits, and structural stiffness determine the forces and accelerations available. A policy cannot compensate for an actuator that cannot brake a moving link fast enough.

3. Contact affordances.
   A rigid two-finger gripper can pinch surfaces within its aperture. A suction cup needs a sealable surface. A soft hand can envelop irregular objects but may struggle to manipulate them with millimeter-level positional precision.

4. Natural dynamics.
   Springs, compliant feet, and mass distribution can stabilize some motions passively. Poorly chosen stiffness or inertia can make those same motions energy-intensive or fragile.

The consequence is important: “more actions” is not automatically better. More joints and motors can expand capability, but they also expand the control problem, create coordination demands, increase sensing requirements, and enlarge the search space a learning algorithm must explore.

---

Case study: a soft pneumatic gripper

The supplied Soft robotic gripper interacting with an environment image shows this interaction between structure, sensing, and behavior particularly clearly.

The gripper’s controller may issue a simple command such as chamber pressure. But pressure alone is not the action that matters at the task level. The actual effect depends on the finger’s elastomer, chamber geometry, fiber reinforcement, current deformation, and contacts with the environment.

Observation constraints

The embedded strain gauge makes finger deformation measurable. That can support feedback such as “the finger has bent farther than expected” or “contact changed the bending response.” But a strain gauge does not directly reveal every task-relevant variable:

• It may not identify the object’s full six-dimensional pose.
• It may not uniquely identify contact location or contact force.
• Similar strains can arise from different combinations of pressure, bending, object size, and contact geometry.
• It gives local proprioceptive information, not a global map of the scene.

A camera or tactile array could reduce some of these ambiguities. Alternatively, the gripper can use active sensing: apply a gentle pressure change, observe the deformation response, and infer whether it has contacted a rigid object, a soft object, or empty space.

Action constraints

The air chamber supplies distributed deformation rather than independently controlled motion at every point on the finger. The inextensible fibers constrain how the elastomer expands, shaping the deformation that pressure can produce. This is a feature: reinforcement can bias motion toward a useful grasping curve instead of allowing uncontrolled ballooning.

At the same time, one chamber cannot generally command arbitrary fingertip position and force independently. A rigid multi-joint hand with many motors might permit finer motion control, but that comes with much greater coordination complexity.

Learnable behavior

For an irregular object family, compliance can make grasping easier to learn. The fingers conform to modest errors in estimated object pose, reducing the precision needed in the approach trajectory. The body absorbs variation that a rigid gripper would force the policy to model explicitly.

However, soft bodies introduce their own difficulties:

• deformation can be nonlinear and history-dependent;
• pressure may not map to the same shape under different loads;
• contacts can be distributed and hard to infer from sparse sensing;
• response can be slower than a rigid motorized mechanism due to fluid flow and material dynamics.

So morphology does not simply shift intelligence from software to hardware. It changes the problem. A soft gripper may make robust acquisition easier while making precise in-hand manipulation harder.

---

Passive dynamics: behavior embodied in mechanics

A passive dynamic walker provides an even stronger example. It can walk down a shallow slope without motors, batteries, or onboard computation. Gravity supplies energy; impacts dissipate energy; leg geometry and mass distribution produce a repeating gait.

The passive walker is not “intelligent” in the broad sense. It cannot choose a destination, interpret a scene, or adapt to arbitrary terrain. But it demonstrates an essential physical-AI principle:

A well-designed body can turn a difficult control objective into a stable physical tendency.

This helps explain why morphology changes learnability. If a desired behavior lies near a passive stable pattern, a learned controller can focus on starting, steering, regulating, and recovering that pattern. If the behavior continually fights the system’s natural dynamics, learning must discover and maintain precise compensations at every moment.

In practice, good embodied design often combines both:

• morphology handles local compliance, energy storage, or passive stability;
• feedback corrects variation and disturbances;
• learning chooses context-sensitive actions and adapts behavior across tasks.

---

From physical constraints to learning consequences

“Learnable” should not be read as a guarantee that a policy-gradient algorithm will converge. Here it means: can a policy with realistic observations, actions, data, and safety limits acquire behavior that works across relevant conditions?

Morphology affects this through several mechanisms.

1. It determines the dimensionality of control

A soft gripper with two chamber-pressure commands has a smaller raw action space than a five-finger hand with twenty independently driven joints. Fewer commands can make exploration and policy fitting easier.

But reduced dimensionality also limits dexterity. The soft gripper may excel at enveloping a broad object class while lacking the independent motions required to rotate a key in a lock.

2. It determines sensitivity to error

A rigid pinch grasp can fail if approach position, object pose, or gripper width is slightly wrong. A compliant finger may tolerate such errors by deforming around the object.

Lower sensitivity often improves data efficiency: successful experiences occur over a wider region of state space. In control terms, the successful behavior has a larger basin of attraction. This does not eliminate uncertainty, but it can make recovery and generalization more attainable.

3. It determines whether relevant state is observable

A robot cannot learn a policy conditioned on an unmeasured variable unless that variable can be inferred from history or correlated measurements. If a grasp’s success depends on slip but the robot receives only a binary “fingers closed” signal, it may not reliably distinguish a stable hold from an empty close.

Better tactile sensing might solve the issue. So might adding recurrent memory, if temporal changes in joint position or strain reveal the hidden contact condition. We will return to the memory question later in this module.

4. It shapes the data distribution created by exploration

Actions change not only the world but also what the robot gets to observe. A wrist camera can move to disambiguate an occluded object. A soft finger can probe compliance. A wheeled robot can rotate in place to acquire a more informative visual view.

Thus, morphology determines which informative experiments are even available to the agent.

---

A morphology audit for a physical-AI task

Before selecting a policy architecture or training algorithm, write a short morphology audit.

Audit item	Example question
Task-relevant state	What must be known to succeed: object pose, contact state, slope, friction, battery level?
Observable variables	Which of those variables are directly measured, indirectly inferable, or hidden?
Available commands	Are actions torques, velocities, positions, pressures, or high-level skills?
Feasible motions	What motions are impossible or unreliable because of joint limits, non-slip wheels, collisions, or weak actuators?
Useful passive behavior	Does compliance, gravity, spring energy, or shape naturally help the task?
Failure sensitivities	Which small errors cause failure: pose error, timing error, friction variation, occlusion, delay?
Learning implication	Does the policy need memory, active sensing, a dynamics model, demonstrations, or a redesigned body?

For example, consider a mobile robot with a forward camera and differential drive, asked to navigate a cluttered hallway:

• Its camera may not see obstacles around a blind corner.
• Its wheels can drive and turn, but cannot slide sideways to escape a near-wall configuration.
• Its body width determines whether a gap is traversable.
• Its turning radius changes which local plans are feasible.
• Its policy may need memory because the current image might not reveal whether it has just passed an intersection.

The task may look like “vision plus reinforcement learning,” but its difficulty is inseparable from camera placement, wheel geometry, robot width, and turning constraint.

---

Key takeaways

Morphology is the physical structure through which an embodied agent senses, acts, and interacts. It includes geometry, joints, actuators, materials, sensors, and contact surfaces.

It constrains:

• observations by determining which physical signals are visible, measurable, amplified, occluded, or ambiguous;
• actions by limiting reachability, forces, speeds, contact modes, and feasible trajectories;
• learnable behaviors by changing control dimensionality, robustness to error, observability of relevant state, and the distribution of experiences produced during exploration.

Morphological computation is valuable but limited: compliant materials and passive dynamics can simplify a control problem, yet broad autonomy still requires sensing, feedback, and decision-making.

Next, we will make this concrete in simulation by specifying observation and action spaces for an embodied task—turning the body-level analysis here into an interface that a controller or learning algorithm can actually use.