Understanding Serialization: How it Works and RCE Risks

Hello! Welcome to the first lesson of our module on advanced server-side vulnerabilities.

In the previous module, we concluded by learning how to turn a file upload vulnerability into persistent Remote Code Execution (RCE) by uploading and leveraging web shells. You saw how a simple upload can become a full system compromise. Today, we pivot to another, often more subtle, but equally devastating class of vulnerability that can also lead to RCE: Insecure Deserialization.

Your background in computer science means you've likely encountered serialization before—perhaps saving an object to a file with Python's pickle or converting data to JSON for an API. This lesson is designed to reframe that familiar programming concept through a security lens. We'll explore how this everyday process can become a critical vulnerability.

Our goal is to explain how data serialization works and the RCE risks associated with insecure deserialization. We will focus on the "what" and the "why," building the fundamental knowledge you'll need to start exploiting these flaws in our next lesson.

1. What Are Serialization and Deserialization?

At its core, serialization is the process of converting an in-memory object (like a class instance in Python or Java) into a "flat" format, like a stream of bytes or a string. This allows the object and its state (the values of its properties) to be easily stored in a file, sent across a network, or saved in a database. Deserialization is the reverse process: taking that byte stream and reconstructing the original object in memory.

A fantastic way to visualize this is by thinking about a video game save file.

As the video mentions, web applications use serialization for various purposes, most commonly to store complex session objects in cookies. Instead of querying a database on every request to get user preferences, roles, and other session data, the server can serialize this information into a cookie. When the user makes a new request, the server simply deserializes the cookie to restore the session object.

There are two main categories of serialization formats:

• Platform-agnostic (human-readable): Formats like JSON and XML are language-independent and designed primarily for data interchange. They represent data, but not typically executable logic.
• Platform-specific (often binary): Formats like Python's pickle, Java's Serializable, and PHP's serialize() are specific to a programming language. These are powerful because they can serialize almost any object, including its methods and internal logic. This power is also what makes them dangerous.

To get a clearer picture of these concepts, the PortSwigger Web Security Academy provides a solid textual definition.

2. How Deserialization Becomes Insecure

The vulnerability isn't in serialization itself, but in the deserialization of untrusted data. Insecure deserialization occurs when an application deserializes data provided by a user (e.g., from a cookie or a POST request body) without proper validation.

Why is this so dangerous? Because many language-specific serialization formats don't just store data; they can also include references to code that should be executed during the reconstruction of the object. An attacker can craft a malicious serialized object that, when deserialized, forces the application to perform unintended actions.

This is a "pre-logic" vulnerability. The attack executes during the deserialization process itself, often before the application's own business logic has a chance to inspect or validate the resulting object.

This is why simply checking the object's properties after it has been deserialized is often too late. The damage may have already been done.

3. The Path to RCE: Magic Methods and Gadgets

The most critical impact of insecure deserialization is Remote Code Execution. Let's examine how this works using Python's pickle module, as you are already familiar with Python.

When pickle deserializes an object, it looks for special methods that define how that object should be reconstructed. One such "magic method" is __reduce__(). This method is intended to let a developer define custom serialization/deserialization logic. However, an attacker can abuse it.

The __reduce__() method should return a tuple. The first element of the tuple is a callable (like a function), and the second is a tuple of arguments for that callable. During deserialization, pickle will automatically execute this callable with the provided arguments.

An attacker can create a class with a malicious __reduce__() method that returns (os.system, ('some_command',)). When an application deserializes an instance of this malicious class, it will effectively run os.system('some_command'), giving the attacker RCE.

The following resources demonstrate this exact process.

To see this in code, the Semgrep article on insecure deserialization provides a very clear and minimal example.

Test your understanding!

A developer on your team suggests using Python's pickle to serialize a user's preferences object and store it in a browser cookie. They argue it's safe because after deserializing the cookie, their code explicitly checks that the restored object is of the expected Preferences class before using it.

Based on what you've learned, explain why this check is not sufficient to prevent an RCE attack.

Show answer

The check is insufficient because insecure deserialization vulnerabilities often execute during the deserialization process itself, before any application logic can run. An attacker could provide a malicious pickle string that defines a class with a harmful __reduce__() method. When the server calls pickle.loads() on this string, Python will execute the code in the __reduce__ method (e.g., os.system('...')) as part of the object reconstruction. The application will crash with an error when it tries to perform the class type check (because the object isn't a Preferences object), but by then, the malicious command has already been executed. The validation happens too late.

4. The Broader Impact

While RCE is the most severe outcome, insecure deserialization can lead to a range of other attacks:

• Privilege Escalation: If a serialized object contains a user's role (e.g., role: "user"), an attacker might be able to modify the serialized data to role: "admin" and deserialize it to gain administrative privileges.
• Arbitrary File Access: Payloads can be crafted to read or write arbitrary files on the server, leading to information disclosure or a web shell upload.
• Denial-of-Service (DoS): A malicious object can be crafted to consume excessive memory or CPU when deserialized, causing the application to crash. This is sometimes called a "billion laughs" attack.

Conclusion

In this lesson, we demystified the concept of insecure deserialization. You have learned that this vulnerability transforms a standard programming convenience into a powerful vector for attack.

Key Takeaways:

• Serialization converts objects into byte streams for storage or transport, while deserialization restores them.
• The vulnerability arises when an application deserializes untrusted, user-controlled data.
• Language-specific formats like Python's pickle are especially dangerous because they can execute code during the deserialization process via magic methods (e.g., __reduce__).
• The primary risk is Remote Code Execution (RCE), but other impacts like privilege escalation and DoS are also possible.
• This is a "pre-logic" flaw; the attack happens before the application has a chance to validate the restored object, making it difficult to mitigate with simple checks.

Next Lesson Preview:

Now that you understand the theory, it's time to get practical. In the next lesson, "Identify and exploit insecure deserialization vulnerabilities in a target application (e.g., Python Pickle)," we will take these concepts and apply them to a real-world scenario. You will learn how to spot, craft, and execute a pickle deserialization attack to gain a shell on a vulnerable machine.