27.7. `tracemalloc` — 跟踪内存分配 ¶

3.4 版新增。

tracemalloc 模块是跟踪 Python 分配内存块的调试工具。它提供下列信息：

回溯分配对象位置
Statistics on allocated memory blocks per filename and per line number: total size, number and average size of allocated memory blocks
计算 2 快照间差异以检测内存泄漏

要跟踪 Python 分配的大部分内存块，应尽早启动模块通过设置 PYTHONTRACEMALLOC 环境变量到 1 ，或通过使用 -X tracemalloc 命令行选项。 tracemalloc.start() 函数可以在运行时被调用以启动跟踪 Python 内存分配。

默认情况下，仅存储分配内存块跟踪最近的 1 帧。要在启动时存储 25 帧：设置 PYTHONTRACEMALLOC 环境变量到 25 ，或使用 -X tracemalloc=25 命令行选项。

27.7.1. 范例 ¶

27.7.1.1. Display the top 10 ¶

显示分配最多内存的 10 个文件：

import tracemalloc
tracemalloc.start()
# ... run your application ...
snapshot = tracemalloc.take_snapshot()
top_stats = snapshot.statistics('lineno')
print("[ Top 10 ]")
for stat in top_stats[:10]:
    print(stat)

Python 测试套件的输出范例：

[ Top 10 ]
<frozen importlib._bootstrap>:716: size=4855 KiB, count=39328, average=126 B
<frozen importlib._bootstrap>:284: size=521 KiB, count=3199, average=167 B
/usr/lib/python3.4/collections/__init__.py:368: size=244 KiB, count=2315, average=108 B
/usr/lib/python3.4/unittest/case.py:381: size=185 KiB, count=779, average=243 B
/usr/lib/python3.4/unittest/case.py:402: size=154 KiB, count=378, average=416 B
/usr/lib/python3.4/abc.py:133: size=88.7 KiB, count=347, average=262 B
<frozen importlib._bootstrap>:1446: size=70.4 KiB, count=911, average=79 B
<frozen importlib._bootstrap>:1454: size=52.0 KiB, count=25, average=2131 B
<string>:5: size=49.7 KiB, count=148, average=344 B
/usr/lib/python3.4/sysconfig.py:411: size=48.0 KiB, count=1, average=48.0 KiB

可以看到 Python 加载了 4.8 MiB 数据 (字节码和常量) 从模块而 collections 模块分配了 244 KiB 以构建 namedtuple 类型。

见 Snapshot.statistics() 了解更多选项。

27.7.1.2. Compute differences ¶

获取 2 快照并显示差异：

import tracemalloc
tracemalloc.start()
# ... start your application ...
snapshot1 = tracemalloc.take_snapshot()
# ... call the function leaking memory ...
snapshot2 = tracemalloc.take_snapshot()
top_stats = snapshot2.compare_to(snapshot1, 'lineno')
print("[ Top 10 differences ]")
for stat in top_stats[:10]:
    print(stat)

运行 Python 测试套件的一些测试之前/之后的输出范例：

[ Top 10 differences ]
<frozen importlib._bootstrap>:716: size=8173 KiB (+4428 KiB), count=71332 (+39369), average=117 B
/usr/lib/python3.4/linecache.py:127: size=940 KiB (+940 KiB), count=8106 (+8106), average=119 B
/usr/lib/python3.4/unittest/case.py:571: size=298 KiB (+298 KiB), count=589 (+589), average=519 B
<frozen importlib._bootstrap>:284: size=1005 KiB (+166 KiB), count=7423 (+1526), average=139 B
/usr/lib/python3.4/mimetypes.py:217: size=112 KiB (+112 KiB), count=1334 (+1334), average=86 B
/usr/lib/python3.4/http/server.py:848: size=96.0 KiB (+96.0 KiB), count=1 (+1), average=96.0 KiB
/usr/lib/python3.4/inspect.py:1465: size=83.5 KiB (+83.5 KiB), count=109 (+109), average=784 B
/usr/lib/python3.4/unittest/mock.py:491: size=77.7 KiB (+77.7 KiB), count=143 (+143), average=557 B
/usr/lib/python3.4/urllib/parse.py:476: size=71.8 KiB (+71.8 KiB), count=969 (+969), average=76 B
/usr/lib/python3.4/contextlib.py:38: size=67.2 KiB (+67.2 KiB), count=126 (+126), average=546 B

We can see that Python has loaded 8.2 MiB of module data (bytecode and constants), and that this is 4.4 MiB more than had been loaded before the tests, when the previous snapshot was taken. Similarly, the linecache module has cached 940 KiB of Python source code to format tracebacks, all of it since the previous snapshot.

If the system has little free memory, snapshots can be written on disk using the Snapshot.dump() method to analyze the snapshot offline. Then use the Snapshot.load() 方法重新加载快照。

27.7.1.3. Get the traceback of a memory block ¶

显示最大内存块回溯的代码：

import tracemalloc
# Store 25 frames
tracemalloc.start(25)
# ... run your application ...
snapshot = tracemalloc.take_snapshot()
top_stats = snapshot.statistics('traceback')
# pick the biggest memory block
stat = top_stats[0]
print("%s memory blocks: %.1f KiB" % (stat.count, stat.size / 1024))
for line in stat.traceback.format():
    print(line)

Example of output of the Python test suite (traceback limited to 25 frames):

903 memory blocks: 870.1 KiB
  File "<frozen importlib._bootstrap>", line 716
  File "<frozen importlib._bootstrap>", line 1036
  File "<frozen importlib._bootstrap>", line 934
  File "<frozen importlib._bootstrap>", line 1068
  File "<frozen importlib._bootstrap>", line 619
  File "<frozen importlib._bootstrap>", line 1581
  File "<frozen importlib._bootstrap>", line 1614
  File "/usr/lib/python3.4/doctest.py", line 101
    import pdb
  File "<frozen importlib._bootstrap>", line 284
  File "<frozen importlib._bootstrap>", line 938
  File "<frozen importlib._bootstrap>", line 1068
  File "<frozen importlib._bootstrap>", line 619
  File "<frozen importlib._bootstrap>", line 1581
  File "<frozen importlib._bootstrap>", line 1614
  File "/usr/lib/python3.4/test/support/__init__.py", line 1728
    import doctest
  File "/usr/lib/python3.4/test/test_pickletools.py", line 21
    support.run_doctest(pickletools)
  File "/usr/lib/python3.4/test/regrtest.py", line 1276
    test_runner()
  File "/usr/lib/python3.4/test/regrtest.py", line 976
    display_failure=not verbose)
  File "/usr/lib/python3.4/test/regrtest.py", line 761
    match_tests=ns.match_tests)
  File "/usr/lib/python3.4/test/regrtest.py", line 1563
    main()
  File "/usr/lib/python3.4/test/__main__.py", line 3
    regrtest.main_in_temp_cwd()
  File "/usr/lib/python3.4/runpy.py", line 73
    exec(code, run_globals)
  File "/usr/lib/python3.4/runpy.py", line 160
    "__main__", fname, loader, pkg_name)

We can see that the most memory was allocated in the importlib module to load data (bytecode and constants) from modules: 870 KiB . The traceback is where the importlib loaded data most recently: on the import pdb line of the doctest module. The traceback may change if a new module is loaded.

27.7.1.4. Pretty top ¶

采用美化输出显示分配最多内存的 10 行代码，忽略 <frozen importlib._bootstrap> and <unknown> 文件：

import linecache
import os
import tracemalloc
def display_top(snapshot, group_by='lineno', limit=10):
    snapshot = snapshot.filter_traces((
        tracemalloc.Filter(False, "<frozen importlib._bootstrap>"),
        tracemalloc.Filter(False, "<unknown>"),
    ))
    top_stats = snapshot.statistics(group_by)
    print("Top %s lines" % limit)
    for index, stat in enumerate(top_stats[:limit], 1):
        frame = stat.traceback[0]
        # replace "/path/to/module/file.py" with "module/file.py"
        filename = os.sep.join(frame.filename.split(os.sep)[-2:])
        print("#%s: %s:%s: %.1f KiB"
              % (index, filename, frame.lineno, stat.size / 1024))
        line = linecache.getline(frame.filename, frame.lineno).strip()
        if line:
            print('    %s' % line)
    other = top_stats[limit:]
    if other:
        size = sum(stat.size for stat in other)
        print("%s other: %.1f KiB" % (len(other), size / 1024))
    total = sum(stat.size for stat in top_stats)
    print("Total allocated size: %.1f KiB" % (total / 1024))
tracemalloc.start()
# ... run your application ...
snapshot = tracemalloc.take_snapshot()
display_top(snapshot)

Python 测试套件的输出范例：

Top 10 lines
#1: Lib/base64.py:414: 419.8 KiB
    _b85chars2 = [(a + b) for a in _b85chars for b in _b85chars]
#2: Lib/base64.py:306: 419.8 KiB
    _a85chars2 = [(a + b) for a in _a85chars for b in _a85chars]
#3: collections/__init__.py:368: 293.6 KiB
    exec(class_definition, namespace)
#4: Lib/abc.py:133: 115.2 KiB
    cls = super().__new__(mcls, name, bases, namespace)
#5: unittest/case.py:574: 103.1 KiB
    testMethod()
#6: Lib/linecache.py:127: 95.4 KiB
    lines = fp.readlines()
#7: urllib/parse.py:476: 71.8 KiB
    for a in _hexdig for b in _hexdig}
#8: <string>:5: 62.0 KiB
#9: Lib/_weakrefset.py:37: 60.0 KiB
    self.data = set()
#10: Lib/base64.py:142: 59.8 KiB
    _b32tab2 = [a + b for a in _b32tab for b in _b32tab]
6220 other: 3602.8 KiB
Total allocated size: 5303.1 KiB

见 Snapshot.statistics() 了解更多选项。

27.7.2. API ¶

27.7.2.1. Functions ¶

tracemalloc. clear_traces ( ) ¶

tracemalloc. get_object_traceback ( obj ) ¶

tracemalloc. get_traceback_limit ( ) ¶

tracemalloc. get_traced_memory ( ) ¶

tracemalloc. get_tracemalloc_memory ( ) ¶

tracemalloc. is_tracing ( ) ¶

tracemalloc. start ( nframe: int=1 ) ¶

tracemalloc. stop ( ) ¶

tracemalloc. take_snapshot ( ) ¶

27.7.2.2. Filter ¶

class tracemalloc. 过滤 ( inclusive: bool , filename_pattern: str , lineno: int=None , all_frames: bool=False ) ¶

27.7.2.3. Frame ¶

class tracemalloc. Frame ¶

filename ¶

lineno ¶

27.7.2.4. Snapshot ¶

class tracemalloc. Snapshot ¶

compare_to ( old_snapshot: Snapshot , group_by: str , cumulative: bool=False ) ¶

dump ( filename ) ¶

filter_traces ( 过滤 ) ¶

classmethod load ( filename ) ¶

statistics ( group_by: str , cumulative: bool=False ) ¶

group_by	description
`'filename'`	filename
`'lineno'`	文件名和行号
`'traceback'`	traceback

traceback_limit ¶

traces ¶

Traces of all memory blocks allocated by Python: sequence of Trace 实例。

The sequence has an undefined order. Use the Snapshot.statistics() method to get a sorted list of statistics.

27.7.2.5. Statistic ¶

class tracemalloc. Statistic ¶

27.7.2.6. StatisticDiff ¶

class tracemalloc. StatisticDiff ¶

27.7.2.7. Trace ¶

class tracemalloc. Trace ¶

27.7.2.8. Traceback ¶

class tracemalloc. 回溯 ¶

27.7. `tracemalloc` — 跟踪内存分配 ¶

27.7.1. 范例 ¶

27.7.1.1. Display the top 10 ¶

27.7.1.2. Compute differences ¶

27.7.1.3. Get the traceback of a memory block ¶

27.7.1.4. Pretty top ¶

27.7.2. API ¶

27.7.2.1. Functions ¶

27.7.2.2. Filter ¶

27.7.2.3. Frame ¶

27.7.2.4. Snapshot ¶

27.7.2.5. Statistic ¶

27.7.2.6. StatisticDiff ¶

27.7.2.7. Trace ¶

27.7.2.8. Traceback ¶

内容表

上一话题

下一话题

本页

快速搜索

27.7. tracemalloc — 跟踪内存分配 ¶

27.7.1. 范例 ¶

27.7.1.1. Display the top 10 ¶

27.7.1.2. Compute differences ¶

27.7.1.3. Get the traceback of a memory block ¶

27.7.1.4. Pretty top ¶

27.7.2. API ¶

27.7.2.1. Functions ¶

27.7.2.2. Filter ¶

27.7.2.3. Frame ¶

27.7.2.4. Snapshot ¶

27.7.2.5. Statistic ¶

27.7.2.6. StatisticDiff ¶

27.7.2.7. Trace ¶

27.7.2.8. Traceback ¶

内容表

上一话题

下一话题

本页

快速搜索

27.7. `tracemalloc` — 跟踪内存分配 ¶