`tracemalloc` — 跟踪内存分配 ¶

3.4 版新增。

tracemalloc 模块是跟踪 Python 分配内存块的调试工具。它提供下列信息：

回溯分配对象位置
有关每文件名和每行号所分配的内存块统计信息：分配内存块的总大小、数量和平均大小
计算 2 快照间差异以检测内存泄漏

要跟踪 Python 分配的大部分内存块，应尽早启动模块通过设置 PYTHONTRACEMALLOC 环境变量到 1 ，或通过使用 -X tracemalloc 命令行选项。 tracemalloc.start() 函数可以在运行时被调用以启动跟踪 Python 内存分配。

默认情况下，仅存储分配内存块跟踪最近的 1 帧。要在启动时存储 25 帧：设置 PYTHONTRACEMALLOC 环境变量到 25 ，或使用 -X tracemalloc=25 命令行选项。

范例 ¶

显示前 10 ¶

显示分配最多内存的 10 个文件：

									import tracemalloc
tracemalloc.start()
# ... run your application ...
snapshot = tracemalloc.take_snapshot()
top_stats = snapshot.statistics('lineno')
print("[ Top 10 ]")
for stat in top_stats[:10]:
    print(stat)
									
								

Python 测试套件的输出范例：

									[ Top 10 ]
<frozen importlib._bootstrap>:716: size=4855 KiB, count=39328, average=126 B
<frozen importlib._bootstrap>:284: size=521 KiB, count=3199, average=167 B
/usr/lib/python3.4/collections/__init__.py:368: size=244 KiB, count=2315, average=108 B
/usr/lib/python3.4/unittest/case.py:381: size=185 KiB, count=779, average=243 B
/usr/lib/python3.4/unittest/case.py:402: size=154 KiB, count=378, average=416 B
/usr/lib/python3.4/abc.py:133: size=88.7 KiB, count=347, average=262 B
<frozen importlib._bootstrap>:1446: size=70.4 KiB, count=911, average=79 B
<frozen importlib._bootstrap>:1454: size=52.0 KiB, count=25, average=2131 B
<string>:5: size=49.7 KiB, count=148, average=344 B
/usr/lib/python3.4/sysconfig.py:411: size=48.0 KiB, count=1, average=48.0 KiB
									
								

可以看到 Python 加载了 4855 KiB 数据 (字节码和常量) 从模块而 collections 模块分配了 244 KiB 以构建 namedtuple 类型。

见 Snapshot.statistics() 了解更多选项。

计算差异 ¶

获取 2 快照并显示差异：

									import tracemalloc
tracemalloc.start()
# ... start your application ...
snapshot1 = tracemalloc.take_snapshot()
# ... call the function leaking memory ...
snapshot2 = tracemalloc.take_snapshot()
top_stats = snapshot2.compare_to(snapshot1, 'lineno')
print("[ Top 10 differences ]")
for stat in top_stats[:10]:
    print(stat)
									
								

运行 Python 测试套件的一些测试之前/之后的输出范例：

									[ Top 10 differences ]
<frozen importlib._bootstrap>:716: size=8173 KiB (+4428 KiB), count=71332 (+39369), average=117 B
/usr/lib/python3.4/linecache.py:127: size=940 KiB (+940 KiB), count=8106 (+8106), average=119 B
/usr/lib/python3.4/unittest/case.py:571: size=298 KiB (+298 KiB), count=589 (+589), average=519 B
<frozen importlib._bootstrap>:284: size=1005 KiB (+166 KiB), count=7423 (+1526), average=139 B
/usr/lib/python3.4/mimetypes.py:217: size=112 KiB (+112 KiB), count=1334 (+1334), average=86 B
/usr/lib/python3.4/http/server.py:848: size=96.0 KiB (+96.0 KiB), count=1 (+1), average=96.0 KiB
/usr/lib/python3.4/inspect.py:1465: size=83.5 KiB (+83.5 KiB), count=109 (+109), average=784 B
/usr/lib/python3.4/unittest/mock.py:491: size=77.7 KiB (+77.7 KiB), count=143 (+143), average=557 B
/usr/lib/python3.4/urllib/parse.py:476: size=71.8 KiB (+71.8 KiB), count=969 (+969), average=76 B
/usr/lib/python3.4/contextlib.py:38: size=67.2 KiB (+67.2 KiB), count=126 (+126), average=546 B
									
								

We can see that Python has loaded 8173 KiB of module data (bytecode and constants), and that this is 4428 KiB more than had been loaded before the tests, when the previous snapshot was taken. Similarly, the linecache module has cached 940 KiB of Python source code to format tracebacks, all of it since the previous snapshot.

If the system has little free memory, snapshots can be written on disk using the Snapshot.dump() method to analyze the snapshot offline. Then use the Snapshot.load() 方法重新加载快照。

获取内存块的回溯 ¶

显示最大内存块回溯的代码：

									import tracemalloc
# Store 25 frames
tracemalloc.start(25)
# ... run your application ...
snapshot = tracemalloc.take_snapshot()
top_stats = snapshot.statistics('traceback')
# pick the biggest memory block
stat = top_stats[0]
print("%s memory blocks: %.1f KiB" % (stat.count, stat.size / 1024))
for line in stat.traceback.format():
    print(line)
									
								

Example of output of the Python test suite (traceback limited to 25 frames):

									903 memory blocks: 870.1 KiB
  File "<frozen importlib._bootstrap>", line 716
  File "<frozen importlib._bootstrap>", line 1036
  File "<frozen importlib._bootstrap>", line 934
  File "<frozen importlib._bootstrap>", line 1068
  File "<frozen importlib._bootstrap>", line 619
  File "<frozen importlib._bootstrap>", line 1581
  File "<frozen importlib._bootstrap>", line 1614
  File "/usr/lib/python3.4/doctest.py", line 101
    import pdb
  File "<frozen importlib._bootstrap>", line 284
  File "<frozen importlib._bootstrap>", line 938
  File "<frozen importlib._bootstrap>", line 1068
  File "<frozen importlib._bootstrap>", line 619
  File "<frozen importlib._bootstrap>", line 1581
  File "<frozen importlib._bootstrap>", line 1614
  File "/usr/lib/python3.4/test/support/__init__.py", line 1728
    import doctest
  File "/usr/lib/python3.4/test/test_pickletools.py", line 21
    support.run_doctest(pickletools)
  File "/usr/lib/python3.4/test/regrtest.py", line 1276
    test_runner()
  File "/usr/lib/python3.4/test/regrtest.py", line 976
    display_failure=not verbose)
  File "/usr/lib/python3.4/test/regrtest.py", line 761
    match_tests=ns.match_tests)
  File "/usr/lib/python3.4/test/regrtest.py", line 1563
    main()
  File "/usr/lib/python3.4/test/__main__.py", line 3
    regrtest.main_in_temp_cwd()
  File "/usr/lib/python3.4/runpy.py", line 73
    exec(code, run_globals)
  File "/usr/lib/python3.4/runpy.py", line 160
    "__main__", fname, loader, pkg_name)
									
								

We can see that the most memory was allocated in the importlib module to load data (bytecode and constants) from modules: 870.1 KiB . The traceback is where the importlib loaded data most recently: on the import pdb line of the doctest module. The traceback may change if a new module is loaded.

美化顶部 ¶

采用美化输出显示分配最多内存的 10 行代码，忽略 <frozen importlib._bootstrap> and <unknown> 文件：

									import linecache
import os
import tracemalloc
def display_top(snapshot, key_type='lineno', limit=10):
    snapshot = snapshot.filter_traces((
        tracemalloc.Filter(False, "<frozen importlib._bootstrap>"),
        tracemalloc.Filter(False, "<unknown>"),
    ))
    top_stats = snapshot.statistics(key_type)
    print("Top %s lines" % limit)
    for index, stat in enumerate(top_stats[:limit], 1):
        frame = stat.traceback[0]
        print("#%s: %s:%s: %.1f KiB"
              % (index, frame.filename, frame.lineno, stat.size / 1024))
        line = linecache.getline(frame.filename, frame.lineno).strip()
        if line:
            print('    %s' % line)
    other = top_stats[limit:]
    if other:
        size = sum(stat.size for stat in other)
        print("%s other: %.1f KiB" % (len(other), size / 1024))
    total = sum(stat.size for stat in top_stats)
    print("Total allocated size: %.1f KiB" % (total / 1024))
tracemalloc.start()
# ... run your application ...
snapshot = tracemalloc.take_snapshot()
display_top(snapshot)
									
								

Python 测试套件的输出范例：

									Top 10 lines
#1: Lib/base64.py:414: 419.8 KiB
    _b85chars2 = [(a + b) for a in _b85chars for b in _b85chars]
#2: Lib/base64.py:306: 419.8 KiB
    _a85chars2 = [(a + b) for a in _a85chars for b in _a85chars]
#3: collections/__init__.py:368: 293.6 KiB
    exec(class_definition, namespace)
#4: Lib/abc.py:133: 115.2 KiB
    cls = super().__new__(mcls, name, bases, namespace)
#5: unittest/case.py:574: 103.1 KiB
    testMethod()
#6: Lib/linecache.py:127: 95.4 KiB
    lines = fp.readlines()
#7: urllib/parse.py:476: 71.8 KiB
    for a in _hexdig for b in _hexdig}
#8: <string>:5: 62.0 KiB
#9: Lib/_weakrefset.py:37: 60.0 KiB
    self.data = set()
#10: Lib/base64.py:142: 59.8 KiB
    _b32tab2 = [a + b for a in _b32tab for b in _b32tab]
6220 other: 3602.8 KiB
Total allocated size: 5303.1 KiB
									
								

见 Snapshot.statistics() 了解更多选项。

记录所有跟踪内存块的当前和峰值大小 ¶

The following code computes two sums like 0 + 1 + 2 + ... inefficiently, by creating a list of those numbers. This list consumes a lot of memory temporarily. We can use get_traced_memory() and reset_peak() to observe the small memory usage after the sum is computed as well as the peak memory usage during the computations:

										import tracemalloc
tracemalloc.start()
# Example code: compute a sum with a large temporary list
large_sum = sum(list(range(100000)))
first_size, first_peak = tracemalloc.get_traced_memory()
tracemalloc.reset_peak()
# Example code: compute a sum with a small temporary list
small_sum = sum(list(range(1000)))
second_size, second_peak = tracemalloc.get_traced_memory()
print(f"{first_size=}, {first_peak=}")
print(f"{second_size=}, {second_peak=}")
										
									

输出：

										first_size=664, first_peak=3592984
second_size=804, second_peak=29704

使用 reset_peak() ensured we could accurately record the peak during the computation of small_sum , even though it is much smaller than the overall peak size of memory blocks since the start() call. Without the call to reset_peak() , second_peak would still be the peak from the computation large_sum (that is, equal to first_peak ). In this case, both peaks are much higher than the final memory usage, and which suggests we could optimise (by removing the unnecessary call to list , and writing sum(range(...)) ).

API ¶

函数 ¶

tracemalloc. clear_traces ( ) ¶

清零由 Python 分配的内存块跟踪。

另请参阅 stop() .

tracemalloc. get_object_traceback ( obj ) ¶

Get the traceback where the Python object obj was allocated. Return a Traceback 实例，或 None 若 tracemalloc module is not tracing memory allocations or did not trace the allocation of the object.

另请参阅 gc.get_referrers() and sys.getsizeof() 函数。

tracemalloc. get_traceback_limit ( ) ¶

Get the maximum number of frames stored in the traceback of a trace.

tracemalloc module must be tracing memory allocations to get the limit, otherwise an exception is raised.

The limit is set by the start() 函数。

tracemalloc. get_traced_memory ( ) ¶: Get the current size and peak size of memory blocks traced by the tracemalloc module as a tuple: (current: int, peak: int) .

tracemalloc. reset_peak ( ) ¶

Set the peak size of memory blocks traced by the tracemalloc module to the current size.

什么都不做若 tracemalloc 模块不跟踪内存分配。

This function only modifies the recorded peak size, and does not modify or clear any traces, unlike clear_traces() . Snapshots taken with take_snapshot() before a call to reset_peak() can be meaningfully compared to snapshots taken after the call.

另请参阅 get_traced_memory() .

3.9 版新增。

tracemalloc. get_tracemalloc_memory ( ) ¶: Get the memory usage in bytes of the tracemalloc module used to store traces of memory blocks. Return an int .

tracemalloc. is_tracing ( ) ¶

True 若 tracemalloc 模块正在跟踪 Python 内存分配， False 否则。

另请参阅 start() and stop() 函数。

tracemalloc. start ( nframe: int=1 ) ¶

Start tracing Python memory allocations: install hooks on Python memory allocators. Collected tracebacks of traces will be limited to nframe frames. By default, a trace of a memory block only stores the most recent frame: the limit is 1 . nframe 必须大于或等于 1 .

You can still read the original number of total frames that composed the traceback by looking at the Traceback.total_nframe 属性。

Storing more than 1 frame is only useful to compute statistics grouped by 'traceback' or to compute cumulative statistics: see the Snapshot.compare_to() and Snapshot.statistics() 方法。

Storing more frames increases the memory and CPU overhead of the tracemalloc 模块。使用 get_tracemalloc_memory() function to measure how much memory is used by the tracemalloc 模块。

PYTHONTRACEMALLOC 环境变量 ( PYTHONTRACEMALLOC=NFRAME ) 和 -X tracemalloc=NFRAME command line option can be used to start tracing at startup.

另请参阅 stop() , is_tracing() and get_traceback_limit() 函数。

tracemalloc. stop ( ) ¶

Stop tracing Python memory allocations: uninstall hooks on Python memory allocators. Also clears all previously collected traces of memory blocks allocated by Python.

调用 take_snapshot() function to take a snapshot of traces before clearing them.

另请参阅 start() , is_tracing() and clear_traces() 函数。

tracemalloc. take_snapshot ( ) ¶

Take a snapshot of traces of memory blocks allocated by Python. Return a new Snapshot 实例。

The snapshot does not include memory blocks allocated before the tracemalloc module started to trace memory allocations.

Tracebacks of traces are limited to get_traceback_limit() frames. Use the nframe parameter of the start() function to store more frames.

tracemalloc module must be tracing memory allocations to take a snapshot, see the start() 函数。

另请参阅 get_object_traceback() 函数。

DomainFilter ¶

class tracemalloc. DomainFilter ( inclusive: bool , domain: int ) ¶

Filter traces of memory blocks by their address space (domain).

3.6 版新增。

inclusive ¶

若 inclusive is True (include), match memory blocks allocated in the address space domain .

若 inclusive is False (exclude), match memory blocks not allocated in the address space domain .

domain ¶: 内存块的地址空间 ( int )。只读特性。

Filter ¶

class tracemalloc. Filter ( inclusive: bool , filename_pattern: str , lineno: int=None , all_frames: bool=False , domain: int=None ) ¶

过滤内存块跟踪。

见 fnmatch.fnmatch() function for the syntax of filename_pattern 。 '.pyc' 文件扩展名被替换为 '.py' .

范例：

Filter(True, subprocess.__file__) only includes traces of the subprocess 模块
Filter(False, tracemalloc.__file__) excludes traces of the tracemalloc 模块
Filter(False, "<unknown>") 排除空回溯

3.5 版改变： '.pyo' 文件扩展名不再替换为 '.py' .

3.6 版改变：添加 domain 属性。

domain ¶

内存块的地址空间 ( int or None ).

tracemalloc 使用域 0 to trace memory allocations made by Python. C extensions can use other domains to trace other resources.

inclusive ¶

若 inclusive is True (include), only match memory blocks allocated in a file with a name matching filename_pattern 在行号 lineno .

若 inclusive is False (exclude), ignore memory blocks allocated in a file with a name matching filename_pattern 在行号 lineno .

lineno ¶: 行号 ( int ) of the filter. If lineno is None ，过滤匹配任何行号。

filename_pattern ¶: Filename pattern of the filter ( str )。只读特性。

all_frames ¶

若 all_frames is True , all frames of the traceback are checked. If all_frames is False , only the most recent frame is checked.

此属性无效若回溯限制为 1 。见 get_traceback_limit() 函数和 Snapshot.traceback_limit 属性。

Frame ¶

class tracemalloc. Frame ¶

回溯的帧。

Traceback 类是序列对于 Frame 实例。

filename ¶: 文件名 ( str ).

lineno ¶: 行号 ( int ).

Snapshot ¶

class tracemalloc. Snapshot ¶

Python 分配的内存块跟踪快照。

take_snapshot() 函数创建快照实例。

compare_to ( old_snapshot: Snapshot , key_type: str , cumulative: bool=False ) ¶

Compute the differences with an old snapshot. Get statistics as a sorted list of StatisticDiff instances grouped by key_type .

见 Snapshot.statistics() 方法对于 key_type and cumulative 参数。

结果按从最大到最小次序：绝对值的 StatisticDiff.size_diff , StatisticDiff.size ，绝对值的 StatisticDiff.count_diff , Statistic.count ，然后按 StatisticDiff.traceback .

dump ( filename ) ¶

将快照写入文件。

使用 load() 以重新加载快照。

filter_traces ( filters ) ¶

创建新的 Snapshot instance with a filtered traces 序列， filters is a list of DomainFilter and Filter 实例。若 filters is an empty list, return a new Snapshot instance with a copy of the traces.

All inclusive filters are applied at once, a trace is ignored if no inclusive filters match it. A trace is ignored if at least one exclusive filter matches it.

3.6 版改变： DomainFilter instances are now also accepted in filters .

classmethod load ( filename ) ¶

从文件加载快照。

另请参阅 dump() .

statistics ( key_type: str , cumulative: bool=False ) ¶

Get statistics as a sorted list of Statistic instances grouped by key_type :

key_type	description
`'filename'`	filename
`'lineno'`	文件名和行号
`'traceback'`	traceback

若 cumulative is True , cumulate size and count of memory blocks of all frames of the traceback of a trace, not only the most recent frame. The cumulative mode can only be used with key_type equals to 'filename' and 'lineno' .

结果按从最大到最小次序： Statistic.size , Statistic.count ，然后按 Statistic.traceback .

traceback_limit ¶: Maximum number of frames stored in the traceback of traces : result of the get_traceback_limit() when the snapshot was taken.

traces ¶

Traces of all memory blocks allocated by Python: sequence of Trace 实例。

The sequence has an undefined order. Use the Snapshot.statistics() method to get a sorted list of statistics.

Statistic ¶

class tracemalloc. Statistic ¶

内存分配统计。

Snapshot.statistics() returns a list of Statistic 实例。

另请参阅 StatisticDiff 类。

count ¶: Number of memory blocks ( int ).

size ¶: Total size of memory blocks in bytes ( int ).

traceback ¶: Traceback where the memory block was allocated, Traceback 实例。

StatisticDiff ¶

class tracemalloc. StatisticDiff ¶

Statistic difference on memory allocations between an old and a new Snapshot 实例。

Snapshot.compare_to() returns a list of StatisticDiff instances. See also the Statistic 类。

count ¶: Number of memory blocks in the new snapshot ( int ): 0 if the memory blocks have been released in the new snapshot.

count_diff ¶: Difference of number of memory blocks between the old and the new snapshots ( int ): 0 if the memory blocks have been allocated in the new snapshot.

size ¶: Total size of memory blocks in bytes in the new snapshot ( int ): 0 if the memory blocks have been released in the new snapshot.

size_diff ¶: Difference of total size of memory blocks in bytes between the old and the new snapshots ( int ): 0 if the memory blocks have been allocated in the new snapshot.

traceback ¶: Traceback where the memory blocks were allocated, Traceback 实例。

Trace ¶

class tracemalloc. Trace ¶

内存块的跟踪。

Snapshot.traces 属性是序列对于 Trace 实例。

3.6 版改变：添加 domain 属性。

domain ¶

内存块的地址空间 ( int )。只读特性。

tracemalloc 使用域 0 to trace memory allocations made by Python. C extensions can use other domains to trace other resources.

size ¶: 内存块大小以字节为单位 ( int ).

traceback ¶: Traceback where the memory block was allocated, Traceback 实例。

回溯 ¶

class tracemalloc. 回溯 ¶

Sequence of Frame instances sorted from the oldest frame to the most recent frame.

回溯包含至少 1 帧。若 tracemalloc 模块无法获取帧，文件名 "<unknown>" 在行号 0 被使用。

When a snapshot is taken, tracebacks of traces are limited to get_traceback_limit() frames. See the take_snapshot() function. The original number of frames of the traceback is stored in the Traceback.total_nframe attribute. That allows to know if a traceback has been truncated by the traceback limit.

Trace.traceback 属性是实例化的 Traceback 实例。

3.7 版改变： Frames are now sorted from the oldest to the most recent, instead of most recent to oldest.

total_nframe ¶: Total number of frames that composed the traceback before truncation. This attribute can be set to None 若信息不可用。

3.9 版改变： Traceback.total_nframe 属性被添加。

format ( limit=None , most_recent_first=False ) ¶

Format the traceback as a list of lines with newlines. Use the linecache module to retrieve lines from the source code. If limit is set, format the limit most recent frames if limit is positive. Otherwise, format the abs(limit) oldest frames. If most_recent_first is True , the order of the formatted frames is reversed, returning the most recent frame first instead of last.

类似 traceback.format_tb() 函数，除了 format() 不包括换行符。

范例：

													print("Traceback (most recent call first):")
for line in traceback:
    print(line)
													
												

输出：

													Traceback (most recent call first):
  File "test.py", line 9
    obj = Object()
  File "test.py", line 12
    tb = tracemalloc.get_object_traceback(f())
													
												

`tracemalloc` — 跟踪内存分配 ¶

范例 ¶

显示前 10 ¶

计算差异 ¶

获取内存块的回溯 ¶

美化顶部 ¶

记录所有跟踪内存块的当前和峰值大小 ¶

API ¶

函数 ¶

DomainFilter ¶

Filter ¶

Frame ¶

Snapshot ¶

Statistic ¶

StatisticDiff ¶

Trace ¶

回溯 ¶

内容表

上一话题

下一话题

本页

快速搜索

tracemalloc — 跟踪内存分配 ¶

范例 ¶

显示前 10 ¶

计算差异 ¶

获取内存块的回溯 ¶

美化顶部 ¶

记录所有跟踪内存块的当前和峰值大小 ¶

API ¶

函数 ¶

DomainFilter ¶

Filter ¶

Frame ¶

Snapshot ¶

Statistic ¶

StatisticDiff ¶

Trace ¶

回溯 ¶

内容表

上一话题

下一话题

本页

快速搜索

`tracemalloc` — 跟踪内存分配 ¶