内容表

tracemalloc — 跟踪内存分配
- 范例
- API
  - 函数
  - DomainFilter
  - 过滤
  - Frame
  - Snapshot
  - Statistic
  - StatisticDiff
  - Trace
  - 回溯

上一话题

`tracemalloc` — 跟踪内存分配 ¶

Added in version 3.4.

tracemalloc 模块是跟踪 Python 分配内存块的调试工具。它提供下列信息：

回溯分配对象位置
有关每文件名和每行号所分配的内存块统计信息：分配内存块的总大小、数量和平均大小
计算 2 快照间差异以检测内存泄漏

要跟踪 Python 分配的大部分内存块，应尽早启动模块通过设置 PYTHONTRACEMALLOC 环境变量到 1 ，或通过使用 -X tracemalloc 命令行选项。 tracemalloc.start() 函数可以在运行时被调用以启动跟踪 Python 内存分配。

默认情况下，仅存储分配内存块跟踪最近的 1 帧。要在启动时存储 25 帧：设置 PYTHONTRACEMALLOC 环境变量到 25 ，或使用 -X tracemalloc=25 命令行选项。

范例 ¶

显示前 10 ¶

显示分配最多内存的 10 个文件：

import tracemalloc
tracemalloc.start()
# ... run your application ...
snapshot = tracemalloc.take_snapshot()
top_stats = snapshot.statistics('lineno')
print("[ Top 10 ]")
for stat in top_stats[:10]:
    print(stat)

Python 测试套件的输出范例：

[ Top 10 ]
<frozen importlib._bootstrap>:716: size=4855 KiB, count=39328, average=126 B
<frozen importlib._bootstrap>:284: size=521 KiB, count=3199, average=167 B
/usr/lib/python3.4/collections/__init__.py:368: size=244 KiB, count=2315, average=108 B
/usr/lib/python3.4/unittest/case.py:381: size=185 KiB, count=779, average=243 B
/usr/lib/python3.4/unittest/case.py:402: size=154 KiB, count=378, average=416 B
/usr/lib/python3.4/abc.py:133: size=88.7 KiB, count=347, average=262 B
<frozen importlib._bootstrap>:1446: size=70.4 KiB, count=911, average=79 B
<frozen importlib._bootstrap>:1454: size=52.0 KiB, count=25, average=2131 B
<string>:5: size=49.7 KiB, count=148, average=344 B
/usr/lib/python3.4/sysconfig.py:411: size=48.0 KiB, count=1, average=48.0 KiB

可以看到 Python 加载了 4855 KiB 数据 (字节码和常量) 从模块而 collections 模块分配了 244 KiB 以构建 namedtuple 类型。

见 Snapshot.statistics() 了解更多选项。

计算差异 ¶

获取 2 快照并显示差异：

import tracemalloc
tracemalloc.start()
# ... start your application ...
snapshot1 = tracemalloc.take_snapshot()
# ... call the function leaking memory ...
snapshot2 = tracemalloc.take_snapshot()
top_stats = snapshot2.compare_to(snapshot1, 'lineno')
print("[ Top 10 differences ]")
for stat in top_stats[:10]:
    print(stat)

运行 Python 测试套件的一些测试之前/之后的输出范例：

[ Top 10 differences ]
<frozen importlib._bootstrap>:716: size=8173 KiB (+4428 KiB), count=71332 (+39369), average=117 B
/usr/lib/python3.4/linecache.py:127: size=940 KiB (+940 KiB), count=8106 (+8106), average=119 B
/usr/lib/python3.4/unittest/case.py:571: size=298 KiB (+298 KiB), count=589 (+589), average=519 B
<frozen importlib._bootstrap>:284: size=1005 KiB (+166 KiB), count=7423 (+1526), average=139 B
/usr/lib/python3.4/mimetypes.py:217: size=112 KiB (+112 KiB), count=1334 (+1334), average=86 B
/usr/lib/python3.4/http/server.py:848: size=96.0 KiB (+96.0 KiB), count=1 (+1), average=96.0 KiB
/usr/lib/python3.4/inspect.py:1465: size=83.5 KiB (+83.5 KiB), count=109 (+109), average=784 B
/usr/lib/python3.4/unittest/mock.py:491: size=77.7 KiB (+77.7 KiB), count=143 (+143), average=557 B
/usr/lib/python3.4/urllib/parse.py:476: size=71.8 KiB (+71.8 KiB), count=969 (+969), average=76 B
/usr/lib/python3.4/contextlib.py:38: size=67.2 KiB (+67.2 KiB), count=126 (+126), average=546 B

We can see that Python has loaded 8173 KiB of module data (bytecode and constants), and that this is 4428 KiB more than had been loaded before the tests, when the previous snapshot was taken. Similarly, the linecache module has cached 940 KiB of Python source code to format tracebacks, all of it since the previous snapshot.

If the system has little free memory, snapshots can be written on disk using the Snapshot.dump() method to analyze the snapshot offline. Then use the Snapshot.load() 方法重新加载快照。

获取内存块的回溯 ¶

显示最大内存块回溯的代码：

import tracemalloc
# Store 25 frames
tracemalloc.start(25)
# ... run your application ...
snapshot = tracemalloc.take_snapshot()
top_stats = snapshot.statistics('traceback')
# pick the biggest memory block
stat = top_stats[0]
print("%s memory blocks: %.1f KiB" % (stat.count, stat.size / 1024))
for line in stat.traceback.format():
    print(line)

Example of output of the Python test suite (traceback limited to 25 frames):

903 memory blocks: 870.1 KiB
  File "<frozen importlib._bootstrap>", line 716
  File "<frozen importlib._bootstrap>", line 1036
  File "<frozen importlib._bootstrap>", line 934
  File "<frozen importlib._bootstrap>", line 1068
  File "<frozen importlib._bootstrap>", line 619
  File "<frozen importlib._bootstrap>", line 1581
  File "<frozen importlib._bootstrap>", line 1614
  File "/usr/lib/python3.4/doctest.py", line 101
    import pdb
  File "<frozen importlib._bootstrap>", line 284
  File "<frozen importlib._bootstrap>", line 938
  File "<frozen importlib._bootstrap>", line 1068
  File "<frozen importlib._bootstrap>", line 619
  File "<frozen importlib._bootstrap>", line 1581
  File "<frozen importlib._bootstrap>", line 1614
  File "/usr/lib/python3.4/test/support/__init__.py", line 1728
    import doctest
  File "/usr/lib/python3.4/test/test_pickletools.py", line 21
    support.run_doctest(pickletools)
  File "/usr/lib/python3.4/test/regrtest.py", line 1276
    test_runner()
  File "/usr/lib/python3.4/test/regrtest.py", line 976
    display_failure=not verbose)
  File "/usr/lib/python3.4/test/regrtest.py", line 761
    match_tests=ns.match_tests)
  File "/usr/lib/python3.4/test/regrtest.py", line 1563
    main()
  File "/usr/lib/python3.4/test/__main__.py", line 3
    regrtest.main_in_temp_cwd()
  File "/usr/lib/python3.4/runpy.py", line 73
    exec(code, run_globals)
  File "/usr/lib/python3.4/runpy.py", line 160
    "__main__", fname, loader, pkg_name)

We can see that the most memory was allocated in the importlib module to load data (bytecode and constants) from modules: 870.1 KiB . The traceback is where the importlib loaded data most recently: on the import pdb line of the doctest module. The traceback may change if a new module is loaded.

美化顶部 ¶

采用美化输出显示分配最多内存的 10 行代码，忽略 <frozen importlib._bootstrap> and <unknown> 文件：

import linecache
import os
import tracemalloc
def display_top(snapshot, key_type='lineno', limit=10):
    snapshot = snapshot.filter_traces((
        tracemalloc.Filter(False, "<frozen importlib._bootstrap>"),
        tracemalloc.Filter(False, "<unknown>"),
    ))
    top_stats = snapshot.statistics(key_type)
    print("Top %s lines" % limit)
    for index, stat in enumerate(top_stats[:limit], 1):
        frame = stat.traceback[0]
        print("#%s: %s:%s: %.1f KiB"
              % (index, frame.filename, frame.lineno, stat.size / 1024))
        line = linecache.getline(frame.filename, frame.lineno).strip()
        if line:
            print('    %s' % line)
    other = top_stats[limit:]
    if other:
        size = sum(stat.size for stat in other)
        print("%s other: %.1f KiB" % (len(other), size / 1024))
    total = sum(stat.size for stat in top_stats)
    print("Total allocated size: %.1f KiB" % (total / 1024))
tracemalloc.start()
# ... run your application ...
snapshot = tracemalloc.take_snapshot()
display_top(snapshot)

Python 测试套件的输出范例：

Top 10 lines
#1: Lib/base64.py:414: 419.8 KiB
    _b85chars2 = [(a + b) for a in _b85chars for b in _b85chars]
#2: Lib/base64.py:306: 419.8 KiB
    _a85chars2 = [(a + b) for a in _a85chars for b in _a85chars]
#3: collections/__init__.py:368: 293.6 KiB
    exec(class_definition, namespace)
#4: Lib/abc.py:133: 115.2 KiB
    cls = super().__new__(mcls, name, bases, namespace)
#5: unittest/case.py:574: 103.1 KiB
    testMethod()
#6: Lib/linecache.py:127: 95.4 KiB
    lines = fp.readlines()
#7: urllib/parse.py:476: 71.8 KiB
    for a in _hexdig for b in _hexdig}
#8: <string>:5: 62.0 KiB
#9: Lib/_weakrefset.py:37: 60.0 KiB
    self.data = set()
#10: Lib/base64.py:142: 59.8 KiB
    _b32tab2 = [a + b for a in _b32tab for b in _b32tab]
6220 other: 3602.8 KiB
Total allocated size: 5303.1 KiB

见 Snapshot.statistics() 了解更多选项。

记录所有跟踪内存块的当前和峰值大小 ¶

The following code computes two sums like 0 + 1 + 2 + ... inefficiently, by creating a list of those numbers. This list consumes a lot of memory temporarily. We can use get_traced_memory() and reset_peak() to observe the small memory usage after the sum is computed as well as the peak memory usage during the computations:

import tracemalloc
tracemalloc.start()
# Example code: compute a sum with a large temporary list
large_sum = sum(list(range(100000)))
first_size, first_peak = tracemalloc.get_traced_memory()
tracemalloc.reset_peak()
# Example code: compute a sum with a small temporary list
small_sum = sum(list(range(1000)))
second_size, second_peak = tracemalloc.get_traced_memory()
print(f"{first_size=}, {first_peak=}")
print(f"{second_size=}, {second_peak=}")

输出：

first_size=664, first_peak=3592984
second_size=804, second_peak=29704

使用 reset_peak() ensured we could accurately record the peak during the computation of small_sum , even though it is much smaller than the overall peak size of memory blocks since the start() call. Without the call to reset_peak() , second_peak would still be the peak from the computation large_sum (that is, equal to first_peak ). In this case, both peaks are much higher than the final memory usage, and which suggests we could optimise (by removing the unnecessary call to list , and writing sum(range(...)) ).

API ¶

函数 ¶

tracemalloc. clear_traces ( ) ¶

清零由 Python 分配的内存块跟踪。

另请参阅 stop() .

tracemalloc. get_object_traceback ( obj ) ¶

Get the traceback where the Python object obj was allocated. Return a Traceback 实例，或 None 若 tracemalloc module is not tracing memory allocations or did not trace the allocation of the object.

另请参阅 gc.get_referrers() and sys.getsizeof() 函数。

tracemalloc. get_traceback_limit ( ) ¶

Get the maximum number of frames stored in the traceback of a trace.

The tracemalloc module must be tracing memory allocations to get the limit, otherwise an exception is raised.

The limit is set by the start() 函数。

tracemalloc. get_traced_memory ( ) ¶: Get the current size and peak size of memory blocks traced by the tracemalloc module as a tuple: (current: int, peak: int) .

tracemalloc. reset_peak ( ) ¶

Set the peak size of memory blocks traced by the tracemalloc module to the current size.

什么都不做若 tracemalloc 模块不跟踪内存分配。

This function only modifies the recorded peak size, and does not modify or clear any traces, unlike clear_traces() . Snapshots taken with take_snapshot() before a call to reset_peak() can be meaningfully compared to snapshots taken after the call.

另请参阅 get_traced_memory() .

Added in version 3.9.

tracemalloc. get_tracemalloc_memory ( ) ¶: Get the memory usage in bytes of the tracemalloc module used to store traces of memory blocks. Return an int .

tracemalloc. is_tracing ( ) ¶

True 若 tracemalloc 模块正在跟踪 Python 内存分配， False 否则。

另请参阅 start() and stop() 函数。

tracemalloc. start ( nframe : int = 1 ) ¶

Start tracing Python memory allocations: install hooks on Python memory allocators. Collected tracebacks of traces will be limited to nframe frames. By default, a trace of a memory block only stores the most recent frame: the limit is 1 . nframe 必须大于或等于 1 .

You can still read the original number of total frames that composed the traceback by looking at the Traceback.total_nframe 属性。

Storing more than 1 frame is only useful to compute statistics grouped by 'traceback' or to compute cumulative statistics: see the Snapshot.compare_to() and Snapshot.statistics() 方法。

Storing more frames increases the memory and CPU overhead of the tracemalloc 模块。使用 get_tracemalloc_memory() function to measure how much memory is used by the tracemalloc 模块。

The PYTHONTRACEMALLOC 环境变量 ( PYTHONTRACEMALLOC=NFRAME ) 和 -X tracemalloc=NFRAME command line option can be used to start tracing at startup.

另请参阅 stop() , is_tracing() and get_traceback_limit() 函数。

tracemalloc. stop ( ) ¶

Stop tracing Python memory allocations: uninstall hooks on Python memory allocators. Also clears all previously collected traces of memory blocks allocated by Python.

调用 take_snapshot() function to take a snapshot of traces before clearing them.

另请参阅 start() , is_tracing() and clear_traces() 函数。

tracemalloc. take_snapshot ( ) ¶

Take a snapshot of traces of memory blocks allocated by Python. Return a new Snapshot 实例。

The snapshot does not include memory blocks allocated before the tracemalloc module started to trace memory allocations.

Tracebacks of traces are limited to get_traceback_limit() frames. Use the nframe 参数为 start() function to store more frames.

The tracemalloc module must be tracing memory allocations to take a snapshot, see the start() 函数。

另请参阅 get_object_traceback() 函数。

DomainFilter ¶

class tracemalloc. DomainFilter ( inclusive : bool , domain : int ) ¶

Filter traces of memory blocks by their address space (domain).

Added in version 3.6.

inclusive ¶

若 inclusive is True (include), match memory blocks allocated in the address space domain .

若 inclusive is False (exclude), match memory blocks not allocated in the address space domain .

domain ¶: 内存块的地址空间 ( int )。只读特性。

过滤 ¶

class tracemalloc. 过滤 ( inclusive : bool , filename_pattern : str , lineno : int = None , all_frames : bool = False , domain : int = None ) ¶

过滤内存块跟踪。

见 fnmatch.fnmatch() function for the syntax of filename_pattern 。 '.pyc' 文件扩展名被替换为 '.py' .

范例：

Filter(True, subprocess.__file__) only includes traces of the subprocess 模块
Filter(False, tracemalloc.__file__) excludes traces of the tracemalloc 模块
Filter(False, "<unknown>") 排除空回溯

3.5 版改变： The '.pyo' 文件扩展名不再替换为 '.py' .

3.6 版改变：添加 domain 属性。

domain ¶

内存块的地址空间 ( int or None ).

tracemalloc 使用域 0 to trace memory allocations made by Python. C extensions can use other domains to trace other resources.

inclusive ¶

若 inclusive is True (include), only match memory blocks allocated in a file with a name matching filename_pattern 在行号 lineno .

若 inclusive is False (exclude), ignore memory blocks allocated in a file with a name matching filename_pattern 在行号 lineno .

lineno ¶: 行号 ( int ) of the filter. If lineno is None ，过滤匹配任何行号。

filename_pattern ¶: Filename pattern of the filter ( str )。只读特性。

all_frames ¶

若 all_frames is True , all frames of the traceback are checked. If all_frames is False , only the most recent frame is checked.

此属性无效若回溯限制为 1 。见 get_traceback_limit() 函数和 Snapshot.traceback_limit 属性。

Frame ¶

class tracemalloc. Frame ¶

Snapshot ¶

class tracemalloc. Snapshot ¶

Python 分配的内存块跟踪快照。

The take_snapshot() 函数创建快照实例。

compare_to ( old_snapshot : Snapshot , key_type : str , cumulative : bool = False ) ¶

dump ( filename ) ¶

filter_traces ( 过滤 ) ¶

classmethod load ( filename ) ¶

statistics ( key_type : str , cumulative : bool = False ) ¶

traceback_limit ¶

traces ¶

Statistic ¶

class tracemalloc. Statistic ¶

内存分配统计。

Snapshot.statistics() returns a list of Statistic 实例。

另请参阅 StatisticDiff 类。

count ¶: Number of memory blocks ( int ).

size ¶: Total size of memory blocks in bytes ( int ).

traceback ¶: Traceback where the memory block was allocated, Traceback 实例。

StatisticDiff ¶

class tracemalloc. StatisticDiff ¶

Statistic difference on memory allocations between an old and a new Snapshot 实例。

Snapshot.compare_to() returns a list of StatisticDiff instances. See also the Statistic 类。

count ¶

count_diff ¶

size ¶

size_diff ¶

traceback ¶

Trace ¶

class tracemalloc. Trace ¶

内存块的跟踪。

The Snapshot.traces 属性是序列对于 Trace 实例。

3.6 版改变：添加 domain 属性。

domain ¶

内存块的地址空间 ( int )。只读特性。

tracemalloc 使用域 0 to trace memory allocations made by Python. C extensions can use other domains to trace other resources.

size ¶

traceback ¶

回溯 ¶

class tracemalloc. 回溯 ¶

Sequence of Frame instances sorted from the oldest frame to the most recent frame.

回溯包含至少 1 帧。若 tracemalloc 模块无法获取帧，文件名 "<unknown>" 在行号 0 被使用。

When a snapshot is taken, tracebacks of traces are limited to get_traceback_limit() frames. See the take_snapshot() function. The original number of frames of the traceback is stored in the Traceback.total_nframe attribute. That allows to know if a traceback has been truncated by the traceback limit.

The Trace.traceback 属性是实例化的 Traceback 实例。

3.7 版改变： Frames are now sorted from the oldest to the most recent, instead of most recent to oldest.

total_nframe ¶: Total number of frames that composed the traceback before truncation. This attribute can be set to None 若信息不可用。

3.9 版改变： The Traceback.total_nframe 属性被添加。

format ( limit = None , most_recent_first = False ) ¶

key_type	description
`'filename'`	filename
`'lineno'`	文件名和行号
`'traceback'`	traceback