Modular vs. End-to-End Driving: From Sensors to Actions

Welcome back. In the previous lesson, you established that an automated-driving claim becomes meaningful only when we specify who performs the dynamic driving task, the operational design domain, and the fallback when the system reaches a limit.

This lesson looks inside the vehicle’s software. You will be able to explain—without getting lost in engineering detail—two broad ways a driving system can turn sensor data into vehicle motion:

• a modular architecture, which separates perception, prediction, planning, and control; and
• a learned end-to-end architecture, which trains a unified model to map sensor inputs to a planned trajectory or driving commands.

For a conference audience, the crucial point is not that one approach is “traditional” and the other “AI.” Both can use advanced AI. The real contrast is where the system draws its internal boundaries, what it makes explicit, and how learning is used to produce driving behaviour.

---

Start with the shared problem: from sensing to motion

Regardless of architecture, an automated vehicle must accomplish a connected set of tasks:

1. Sense the surrounding world through cameras, radar, lidar, ultrasonic sensors, and vehicle-motion sensors.
2. Interpret what matters: road geometry, vehicles, cyclists, pedestrians, traffic lights, and other relevant conditions.
3. Decide how to move safely and lawfully in light of a route, road rules, and predicted behaviour of other road users.
4. Actuate the decision through steering, acceleration, and braking.

The question is how explicitly software divides those tasks.

The diagram is deliberately high level. In real vehicles, the boundary between modules is often less clean than the boxes imply. Still, it captures a useful conference-level distinction: modular systems pass human-designed intermediate representations between components, while end-to-end systems seek to learn more of the transformation jointly from data.

---

Modular architecture: explicit stages and visible handoffs

A modular driving architecture divides the driving problem into specialised components. Each component receives an output from the preceding stage and produces a defined output for the next.

A typical trace looks like this:

Stage	Typical input	Typical output	Driving purpose
Sensing	Camera images, radar returns, lidar points, vehicle sensors	Time-synchronised sensor data	Capture evidence about vehicle and environment
Perception and localisation	Sensor data, maps, motion estimates	Lanes, objects, positions, road features, ego-vehicle location	Build an interpretable account of the scene
Prediction and scene understanding	Detected objects and road context	Possible future movements of other road users	Anticipate conflicts and uncertainty
Planning	World model, predictions, route, traffic rules	Candidate trajectories and a selected trajectory	Decide where and how the vehicle should travel
Control	Selected trajectory and current vehicle state	Steering, throttle, and brake commands	Make the physical vehicle follow the intended path

The exact modules differ across organisations. Some systems combine prediction and planning; some put mapping or fusion in a separate layer. The important feature is that the system exposes meaningful intermediate outputs: for example, an object track, a lane boundary, a predicted pedestrian path, or a planned lane change.

Trace a modular system through one road event

Imagine the vehicle approaches roadworks on a city street. A cyclist is ahead, slightly left of centre, while a temporary sign narrows the lane.

In a modular system:

1. Cameras, radar, and lidar gather different forms of evidence about the cyclist, sign, barriers, lane markings, and road edge.
2. Perception identifies or estimates objects and road structure. Localisation estimates where the vehicle is relative to its route and map.
3. Prediction estimates plausible cyclist movements, including whether the cyclist may move further into the narrowed lane.
4. Planning considers legal, safe, and comfortable options: slow behind the cyclist, change position within the lane, or wait. It selects a trajectory.
5. Control calculates steering, acceleration, and braking commands needed to follow that trajectory.

This trace is useful because it lets engineers ask targeted questions. Did the sensors fail to detect the cyclist? Was the cyclist detected but poorly tracked? Was the prediction inadequate? Did the planner select an unsafe trajectory? Or did the controller fail to execute a safe plan?

That diagnostic value is a major attraction of modular design. It supports focused testing, monitoring, and iteration. However, explicit stages also create a risk: an early error can affect every later stage, and a component optimised in isolation may not optimise the driving outcome as a whole.

A modular system is not necessarily “hand-coded.” Modern modular stacks commonly use machine learning, especially for perception and prediction. The defining feature is not the absence of neural networks; it is the use of separate functions with explicit interfaces.

---

Learned end-to-end architecture: optimise the driving task jointly

A learned end-to-end driving system trains a model to transform sensor observations, usually alongside relevant vehicle state and navigation context, into a driving output.

That output can take different forms:

• a direct control command, such as a steering value;
• a sequence of future positions;
• a planned trajectory, later converted into steering, acceleration, and braking by a control layer.

The last option is important. “End-to-end” does not always mean that raw camera pixels directly control the steering wheel with no other software involved. Many modern systems use a learned model to output a trajectory, then use conventional vehicle control to follow it safely and smoothly.

The key idea is that the model learns useful internal representations from driving data and training objectives, rather than requiring designers to specify every intermediate concept as a separate output. Internally, the model may learn features corresponding to lanes, moving objects, road boundaries, or a vehicle’s likely future motion. But those representations need not be exposed as discrete modules such as “cyclist detector” or “lane-marker detector.”

Trace the same event through an end-to-end system

Return to the narrowed roadworks lane and cyclist. An end-to-end system can receive a sequence of camera, radar, lidar, and vehicle-motion inputs, potentially with route information. A unified learned model processes the scene and produces a proposed future trajectory: for example, a deceleration and a path that remains safely behind the cyclist.

The vehicle’s control function then follows that trajectory with steering, braking, and acceleration commands.

What is different is not the physical reality of the driving task. The system still has to respond appropriately to the cyclist, roadworks, and narrowing lane. What changes is the internal route from sensing to decision:

• The modular stack makes its intermediate scene description explicit: “cyclist here,” “barrier there,” “lane boundary here,” then “safe trajectory.”
• The end-to-end stack learns much more of the connection between observed sensor patterns and the desired trajectory jointly.

This joint optimisation is the central technical argument for end-to-end learning. If the ultimate objective is safe, lawful, smooth driving, a unified model can in principle learn representations that are useful for the overall behaviour rather than for intermediate tasks selected primarily because people find them easy to inspect.

---

“End-to-end” is not a synonym for “one black box”

The phrase can mislead an audience if it is used too casually. Treat it as a description of the scope of learning and optimisation, not a guarantee that every function is collapsed into one opaque component.

A practical system may retain:

• sensor calibration and timing;
• health monitoring for sensors and compute;
• vehicle-dynamics control;
• route information and maps;
• rule-based constraints or independent safety checks;
• logging, diagnostics, and mechanisms for degraded operation.

Likewise, a modular system may contain many neural networks. The useful contrast is therefore:

Question	Modular architecture	Learned end-to-end architecture
How are driving functions organised?	Separate, named components	More functions learned jointly in a unified model
What moves between stages?	Explicit objects, maps, predictions, and planned paths	Learned internal features and a final trajectory or control output
Where can machine learning appear?	Often perception and prediction, sometimes planning	Across much or all of perception-to-planning, potentially control
Main engineering attraction	Inspectable stages and targeted diagnosis	Joint optimisation and fewer hand-designed interfaces
Central assurance challenge	Demonstrate each component and their integration work safely	Demonstrate reliable behaviour despite less explicit intermediate reasoning

Neither column establishes that a vehicle is legally self-driving, safe in a particular ODD, or ready for large-scale deployment. Those claims require evidence beyond an architecture diagram.

The middle category in the image is worth mentioning in a presentation because it prevents a false binary. Many systems are hybrids. A company may learn perception and planning jointly, yet retain a conventional control layer and additional safety mechanisms. The sector’s practical question is usually not “modular or end-to-end?” but which functions are learned together, what remains explicit, and how is the whole system assured?

---

What a Wayve-style explanation contributes—and what it does not prove

One useful public articulation of the end-to-end position comes from Wayve’s CEO, Alex Kendall. He contrasts earlier robotics approaches that separated perception, planning, mapping, and control with an approach centred on a large end-to-end neural network.

The careful way to report this is:

“Wayve describes its approach as learning from sensor data to motion outputs through a unified neural network, rather than relying on a fully hand-engineered sequence of perception, planning, mapping, and control modules.”

Avoid turning that into stronger claims such as “a single network makes driving safe,” “end-to-end systems need no maps or safety engineering,” or “end-to-end learning has solved general autonomous driving.” Architecture is a method; safety and deployment are evidence-based outcomes.

---

A concise explanation for your keynote

For an insurance and policy audience, a useful explanation should connect the technical design to the questions the room needs answered.

You might say:

“A modular vehicle software stack resembles a chain of specialist teams. One component builds a picture of the road, another predicts what road users may do, a planner chooses a path, and a controller turns that path into steering and braking. End-to-end systems try to learn more of that chain jointly from sensor data to a driving trajectory. The trade-off is not simply old technology against new technology. It is between more explicit intermediate decisions and more joint, data-driven optimisation. In either case, the essential test remains the same: can the system demonstrate safe behaviour in its defined operating domain and manage its limits?”

That wording does four things:

• distinguishes the architectures accurately;
• avoids claiming that modular systems do not use AI;
• avoids presenting learned systems as inherently unsafe or inherently superior;
• returns the discussion to ODD, fallback, assurance, and evidence.

If you show only one architecture slide, use the first diagram and narrate one concrete road event. A cyclist near roadworks is more memorable than a list of acronyms. Explain what the modular system makes visible, then explain that an end-to-end system learns more of the same connection from observation to intended motion.

---

Key takeaways

A modular architecture separates sensing, perception, prediction, planning, and control. It passes explicit intermediate outputs—such as object tracks, road geometry, predicted movements, and trajectories—between specialised components.

A learned end-to-end architecture trains a unified model to map sensor data, often with navigation and vehicle context, to a driving trajectory or control output. It learns more of the intermediate processing jointly, though it may still include conventional control, monitoring, and safety functions.

Do not frame the choice as AI versus non-AI, or as a simple choice between transparent and opaque systems. Modern modular systems use machine learning, while end-to-end systems can retain explicit safeguards. Most importantly, neither architecture alone demonstrates safe operation, legal self-driving status, or readiness for deployment.

Next, the course moves from how systems are built to how claims about their progress should be assessed: testing, supervised operation, driverless service, commercial operation, and deployment at scale are not interchangeable milestones.