queue
sched — 事件调度器
sched
源代码: Lib/queue.py
The queue 模块实现多生产者、多消费者队列。在线程化编程中尤其有用,当必须在多个线程之间安全交换信息时。 Queue 类在此模块中实现所有要求的锁定语义。
Queue
模块实现了 3 种队列类型,唯一不同之处是检索条目的次序。在 FIFO 队列, 首先检索最先添加的任务。在 LIFO 队列,首先检索最近添加的条目 (操作像堆栈)。采用优先级队列,条目保持排序 (使用 heapq 模块) 且首先检索最低值条目。
heapq
在内部,这 3 种队列类型使用锁,以临时阻塞竞争线程;不管怎样,它们并不是为处理线程内的重入而设计的。
此外,模块实现简单 FIFO 队列类型, SimpleQueue ,具体实现提供额外保证,以换取更小功能。
SimpleQueue
The queue 模块定义下列类和异常:
构造函数为 FIFO 队列。 maxsize is an integer that sets the upperbound limit on the number of items that can be placed in the queue. Insertion will block once this size has been reached, until queue items are consumed. If maxsize <= 0,队列大小无限。
构造函数为 LIFO 队列。 maxsize is an integer that sets the upperbound limit on the number of items that can be placed in the queue. Insertion will block once this size has been reached, until queue items are consumed. If maxsize <= 0,队列大小无限。
优先级队列构造函数。 maxsize is an integer that sets the upperbound limit on the number of items that can be placed in the queue. Insertion will block once this size has been reached, until queue items are consumed. If maxsize <= 0,队列大小无限。
The lowest valued entries are retrieved first (the lowest valued entry is the one that would be returned by min(entries) ). A typical pattern for entries is a tuple in the form: (priority_number, data) .
min(entries)
(priority_number, data)
若 data elements are not comparable, the data can be wrapped in a class that ignores the data item and only compares the priority number:
from dataclasses import dataclass, field from typing import Any @dataclass(order=True) class PrioritizedItem: priority: int item: Any=field(compare=False)
构造函数为无界 FIFO 队列。简单队列缺乏高级功能 (譬如:任务追踪)。
Added in version 3.7.
异常被引发当非阻塞 get() (或 get_nowait() ) 被调用当 Queue 对象为空。
get()
get_nowait()
异常被引发当非阻塞 put() (或 put_nowait() ) 被调用当 Queue 对象已满。
put()
put_nowait()
Queue 对象 ( Queue , LifoQueue ,或 PriorityQueue ) 提供的公共方法的描述见下文。
LifoQueue
PriorityQueue
返回队列的近似大小。注意,qsize() > 0 不保证后续 get() 不会阻塞,qsize() < Maxsize 保证 put() 不会阻塞。
返回 True 若队列为空, False 否则。若 empty() 返回 True 它不保证后续调用 put() 不会阻塞。同样,若 empty() 返回 False 它不保证后续调用 get() 不会阻塞。
True
False
返回 True 若队列是满的, False 否则。若 full() 返回 True 它不保证后续调用 get() 不会阻塞。同样,若 full() 返回 False 它不保证后续调用 put() 不会阻塞。
Put item 进队列。若可选自变量 block 为 True 和 timeout is None (默认),阻塞若有必要直到空闲槽可用。若 timeout 是正数,它阻塞最多 timeout 秒并引发 Full 异常若在该时间内无可用空闲槽。否则 ( block 为 False),将项放入队列若空闲槽立即可用,否则引发 Full 异常 ( timeout 被忽略在这种情况下)。
None
Full
相当于 put(item, block=False) .
put(item, block=False)
移除并返回项从队列。若可选自变量 block 为 True 和 timeout is None (默认),阻塞若有必要直到项可用。若 timeout 是正数,它阻塞最多 timeout 秒并引发 Empty 异常若在该时间内无可用项。否则 ( block 为 False),返回项若立即可用,否则引发 Empty 异常 ( timeout 被忽略在这种情况下)。
Empty
3.0 之前在 POSIX 系统,和对于 Windows 所有版本,若 block 为 True 和 timeout is None ,此操作进入底层锁的不间断等待。这意味着不会发生异常,尤其 SIGINT 不会触发 KeyboardInterrupt .
KeyboardInterrupt
相当于 get(False) .
get(False)
提供 2 方法以支持追踪排队任务是否已被守护消费者线程完全处理。
指示先前排队任务已完成。用于队列消费者线程。对于每个 get() 用于抓取任务,后续调用 task_done() tells the queue that the processing on the task is complete.
task_done()
若 join() is currently blocking, it will resume when all items have been processed (meaning that a task_done() call was received for every item that had been put() into the queue).
join()
引发 ValueError if called more times than there were items placed in the queue.
ValueError
阻塞,直到获取并处理队列中的所有项为止。
The count of unfinished tasks goes up whenever an item is added to the queue. The count goes down whenever a consumer thread calls task_done() to indicate that the item was retrieved and all work on it is complete. When the count of unfinished tasks drops to zero, join() unblocks.
如何等待排队任务完成的范例:
import threading import queue q = queue.Queue() def worker(): while True: item = q.get() print(f'Working on {item}') print(f'Finished {item}') q.task_done() # Turn-on the worker thread. threading.Thread(target=worker, daemon=True).start() # Send thirty task requests to the worker. for item in range(30): q.put(item) # Block until all tasks are done. q.join() print('All work completed')
SimpleQueue 对象提供的公共方法的描述见下文。
Return the approximate size of the queue. Note, qsize() > 0 doesn’t guarantee that a subsequent get() will not block.
返回 True 若队列为空, False 否则。若 empty() 返回 False 它不保证后续调用 get() 不会阻塞。
Put item into the queue. The method never blocks and always succeeds (except for potential low-level errors such as failure to allocate memory). The optional args block and timeout are ignored and only provided for compatibility with Queue.put() .
Queue.put()
CPython 实现细节: This method has a C implementation which is reentrant. That is, a put() or get() call can be interrupted by another put() call in the same thread without deadlocking or corrupting internal state inside the queue. This makes it appropriate for use in destructors such as __del__ 方法或 weakref 回调。
__del__
weakref
相当于 put(item, block=False) ,提供是为兼容 Queue.put_nowait() .
Queue.put_nowait()
另请参阅
multiprocessing.Queue
用于多进程 (而不是多线程) 上下文的队列类。
collections.deque is an alternative implementation of unbounded queues with fast atomic append() and popleft() operations that do not require locking and also support indexing.
collections.deque
append()
popleft()
contextvars — 上下文变量
contextvars
键入搜索术语或模块、类、函数名称。