Why Python’s deepcopy Can Be So Slow (and How to Avoid It)

The Cost of Using deepcopy

Python’s copy.deepcopy() creates a fully independent clone of an object, traversing every nested element of the object graph. This comprehensive copying is convenient but expensive. Deepcopy must handle arbitrary Python objects, check for duplicates (using a memo dict to avoid copying shared objects), and even call custom __deepcopy__ methods if defined. In practice, this means a lot of Python-level operations and memory allocation. As one developer noted, deep-copying a game state (~1000 objects) consumed about 25% of the program’s runtime. In that case, deepcopy took several ms per call (versus microseconds for a shallow copy), which was clearly a bottleneck.

This overhead often makes deepcopy the slowest approach to duplicating data.

In a simple benchmark I created (see this gist), replicating complex objects, deepcopy was much slower than alternatives like serialization – using json/orjson or pickle to dump and load the object.

Relative performance:
  shallow_copy   :  1.00x slower
  orjson_copy    : 100.19x slower
  pickle_copy    : 146.97x slower
  json_copy      : 504.14x slower
  deepcopy       : 664.24x slower

Real-World Examples of Deepcopy Slowness

Pydantic-AI Agent Example: In the Pydantic-AI framework (an extension of Pydantic for AI Agents workflows), an expensive use of deepcopy was recently identified and fixed by Codeflash. The code is provided below. The agent kept a message history (a list of message objects). Originally, when the agent needed to modify the last message (for example, inserting the LLM’s final answer), it deep-copied the entire message history to avoid mutating the original. For long histories, this was costly. The optimization discovered by Codeflash changed this to copy only what’s necessary: they now shallow-copy the list and deep-copy just the last message object that needs modification. The code comment says it all: “Only do deepcopy when we have to modify”. By avoiding an unnecessary deep clone of all messages, this change greatly reduced the overhead of producing the final output. This is a prime example of how a small tweak (avoiding a full deepcopy of a large structure) yielded a big performance win. The new code is now 180x faster!

The slow original code with deepcopy:

    def _set_output_tool_return(self, return_content: str) -> list[_messages.ModelMessage]:
2        """Set return content for the output tool.
3        Useful if you want to continue the conversation and want to set the response to the output tool call.
4        """
5        if not self._output_tool_name:
6            raise ValueError('Cannot set output tool return content when the return type is `str`.')
7        messages = deepcopy(self._state.message_history) # This is expensive
8        last_message = messages[-1]
9        for part in last_message.parts:
10            if isinstance(part, _messages.ToolReturnPart) and part.tool_name == self._output_tool_name:
11                part.content = return_content
12                return messages
13        raise LookupError(f'No tool call found with tool name {self._output_tool_name!r}.')

The optimized code that only deepcopies the last message:

    def _set_output_tool_return(self, return_content: str) -> list[_messages.ModelMessage]:
2        """Set return content for the output tool.
3        Useful if you want to continue the conversation and want to set the response to the output tool call.
4        """
5        if not self._output_tool_name:
6            raise ValueError('Cannot set output tool return content when the return type is `str`.')
7
8        messages = self._state.message_history
9        last_message = messages[-1]
10        for idx, part in enumerate(last_message.parts):
11            if isinstance(part, _messages.ToolReturnPart) and part.tool_name == self._output_tool_name:
12                # Only do deepcopy when we have to modify
13                copied_messages = list(messages)
14                copied_last = deepcopy(last_message) #Only deepcopy the last message
15                copied_last.parts[idx].content = return_content  # type: ignore[misc]
16                copied_messages[-1] = copied_last
17                return copied_messages
18
19        raise LookupError(f'No tool call found with tool name {self._output_tool_name!r}.')

Pydantic Models (Deepcopy vs.Rebuilding): Pydantic, a popular data validation library, provides a model_copy() method to duplicate model instances. Under the hood, this uses Python’s copy mechanisms. Users discovered that using model_copy() was significantly slower than rebuilding a new instance from the same fields. In one experiment, creating a new model bypassing the original’s fields to the constructor was ~25% faster than model_copy(). Updating a field via copy was even worse: m2 = m1.model_copy(); m2.bar= 5 took 2.819 s, whereas simply making a new model with the changed field took ~0.873s. The Pydantic team explained why: model_copy() endsup calling __copy__/__deepcopy__ internally, and due to Pydantic’s internal structure (privateattributes, extra fields, etc.), this adds overhead beyond a plain dataclass copy.In other words, the “safe” deep copy path in Pydantic incurred extra Python logic, making it slower than expected. The takeaway: even in high-level libraries, a deep clone of an object can be noticeably slower than creating afresh one, especially if validation or other logic is involved in copying.

‍

When (and How) to Avoid Deepcopy

Given the above, it’s clear that you should avoid deepcopy unless it’s truly necessary. Here are some strategies and cases to consider:

• Prefer Shallow Copies or New Object Creation: If you only need to copy a container’s top-level structure, a shallow copy (e.g. using the .copy() method of a list/dict or copy.copy()) is much cheaper. Often, you may not need a true deep clone – you might just need to copy a list of references or make a new object with some of the same data. For example, instead of m2 = deepcopy(m1) and then changing m2, it can be faster to explicitly create a new object with the desired updates (as seen with Pydantic models above). This lets you bypass all the internal traversal that deepcopy would do.
• Copy Only What You Must: A recurring theme in optimizations is making the copy scope smaller. The Pydantic-AI example showed that if you only need to tweak one part of a large structure, it’s far more efficient to copy that part alone. You can append that part back into the larger structure without cloning everything. In your own code, analyze whether you truly need every nested object duplicated. Perhaps only a specific sub-object is about to be mutated. If so, copy just that sub-object (or create a modified version of it) and reuse the rest by reference.
• Avoid Copies for Immutables or Read-Only Data: Many objects in Python are effectively immutable (numbers, strings, etc.). Copying them is wasteful because you gain nothing – they can be safely shared. Deepcopy is smart enough to return the same object for immutables, but the function call overhead still occurs. If you know certain data won’t be modified, you can share it instead of copying. For instance, if you have a cached configuration or a large tuple of constants, just pass a reference around. In custom classes, you could implement __deepcopy__ to skip copying expensive attributes that don’t need duplication (e.g. large caches, read-only structures). This kind of custom logic can prevent deepcopy from doing needless work. (One caution: only do this for truly immutable or safe-to-share parts, otherwise you reintroduce the very shared-state bugs deepcopy is meant to avoid.)
• Rethink Data Management (Diffs or Lazy Copies): If you find yourself copying a huge object to “see what would happen if…”, consider alternative designs. For example, in a Reddit thread about slow portfolio copies, a commenter suggested using a “diff” approach or storing state in a database. Instead of copying the entire in-memory state, one could apply transformations and record the differences or simulate changes. Another approach is copy-on-write: delay the copy until you actually need to modify something, and even then only copy the part being modified. These patterns avoid the upfront cost of cloning everything. While they add complexity, they can be worthwhile for performance-critical code handling large data.
• Use Serialization or Other Libraries Cautiously: In some cases, you can serialize an object to clone it. As shown, json.dumps/loads or using pickle can outperform deepcopy for pure data structures. However, this only works for data that is serializable (and pickle can bring its own overhead or security issues). If you have a complex object (with file handles, network connections, etc.), serialization might fail or produce only a shallow copy of those resources. So this method is context-dependent. It can be an effective hack for pure data (e.g. copying a large nested dict by converting to JSON and back), but ensure that the structure has no non-serializable parts and that you don’t need object identity preserved.
• Leverage Efficient Copying in Libraries: Some libraries provide faster copy mechanisms. For example, NumPy arrays can be copied very quickly at the C level (since it’s just memory buffer duplication). If you’re dealing with large numeric data, copying via NumPy (arr.copy()) is far faster than Python-level loops or deepcopies of lists. Similarly, pandas DataFrames have methods to copy or slice data efficiently. The key is to use domain-specific optimizations rather than copying Python objects element-by-element.

‍