天天看點

了解Python的協程機制-Yield

根據pep-0342 coroutines via enhanced generators,原來僅僅用于生成器的yield關鍵字被擴充,成為python協程實作的一部分。而之是以使用協程,主要是出于性能的考慮:一個活躍的python線程大約占據8mb記憶體,而一個活躍線程隻使用1kb不到記憶體。對于io密集型的應用,顯然輕量化的協程更适用。

原來,yield是一個statement,即和return一樣的語句,但是在pep-0342後,yield statement被改造為了yield expression。其文法如下:

<code>yield_atom ::= "(" yield_expression ")"</code>

<code>yield_expression ::= "yield" [expression_list]</code>

yield表達式隻能在函數體内使用,會導緻該函數變為一個 生成器函數

當生成器函數被調用,它會傳回一個 生成器 ,用以控制生成器函數的執行.

生成器的第一次執行,必須使用<code>gen_func.next()</code>方法,函數會執行到第一條yield的位置并傳回該處yield語句之後的表達式值。

接下來,可以交替使用<code>gen_func.next</code>與<code>gen_func.send</code>方法對協程的執行。<code>send</code>方法會傳入一個參數,該參數即yield表達式的值,可以在生成器函數裡面被接收。而<code>next</code>則不傳入參數。當<code>send</code> or <code>next</code>被執行時,生成器函數會從yield表達式處繼續執行。直到下一次出現yield語句,再次傳回yield之後表達式的值。

最簡單的協程使用,通過<code>next</code>與<code>send</code>控制協程的運作。

<code>def coroutine_print():</code>

<code>while true:</code>

<code>message = yield</code>

<code>print message</code>

<code></code>

<code>it = coroutine_print()</code>

<code>next(it)</code>

<code>it.send('hello')</code>

<code>it.send('world')</code>

<code># result:</code>

<code># hello</code>

<code># world</code>

稍微複雜的一個例子,使用yield的傳回值。

假設我們希望執行一個流式計算,不斷有資料到達,我們希望計算這些資料的平均數。

<code>def avg_coroutine():</code>

<code>cnt = 0</code>

<code>sum = 0</code>

<code>new_value = yield</code>

<code>sum += new_value</code>

<code>cnt += 1</code>

<code>new_value = yield float(sum) / cnt</code>

<code>in [6]: it = avg_coroutine()</code>

<code>in [7]: it.next()</code>

<code>in [8]: it.send(10)</code>

<code>out[8]: 10.0</code>

<code>in [9]: it.send(20)</code>

<code>out[9]: 15.0</code>

<code>in [10]: it.send(30)</code>

<code>out[10]: 20.0</code>

<code>in [11]: it.send(110)</code>

<code>out[11]: 42.5</code>

如本例所示,在使用協程時,首先調用<code>next</code>方法停止在第一條<code>yield</code>處,并使用<code>send</code>方法向協程内發送資料,并擷取更新後的平均值。

一旦協程開啟,僅僅通過send與yield隻能對協程進行簡單的控制。throw方法提供了在協程内引發異常的接口。通過主叫者調用throw在協程内引發異常,協程捕獲異常的方式,可以實作主叫者與協程之間的通信。

需要注意的是,使用<code>throw</code>方法在協程内引發的異常,如果沒有被捕獲,或者内部又重新raise了不同的異常,那麼這個異常會傳播到主叫者。

同時<code>throw</code>方法同<code>send</code>與<code>next</code>一樣,都會使協程繼續運作,并傳回下一個yield表達式中的表達式值。且同樣的,如果不存在下一個yield表達式協程就結束了,主叫方會收到<code>stopiteration exception</code>。

<code>close</code>方法與<code>throw</code>方法類似,都是在協程暫停的位置引發一個異常,進而向協程發出控制資訊。不同于<code>throw</code>方法可以抛出自定義異常,<code>close</code>方法會固定抛出<code>generatorexit</code>異常。當這個異常沒有被捕獲或者引發<code>stopiteration exception</code>時,<code>close</code>方法會正常傳回。這個比較好了解,協程設計者捕獲了<code>generatorexit</code>異常并完成清理,保證清理幹淨了,是以最後不會再有yield傳回值引發<code>stopitertation</code>。如果因為設計錯誤導緻仍然有下一個yield,那麼就會抛出<code>runtimeerror</code>.

python文檔中給出的應用四個協程api的樣例。

<code>&gt;&gt;&gt; def echo(value=none):</code>

<code>... print "execution starts when 'next()' is called for the first time."</code>

<code>... try:</code>

<code>... while true:</code>

<code>... try:</code>

<code>... value = (yield value)</code>

<code>... except exception, e:</code>

<code>... value = e</code>

<code>... finally:</code>

<code>... print "don't forget to clean up when 'close()' is called."</code>

<code>...</code>

<code>&gt;&gt;&gt; generator = echo(1)</code>

<code>&gt;&gt;&gt; print generator.next()</code>

<code>execution starts when 'next()' is called for the first time.</code>

<code>1</code>

<code>none</code>

<code>&gt;&gt;&gt; print generator.send(2)</code>

<code>2</code>

<code>&gt;&gt;&gt; generator.throw(typeerror, "spam")</code>

<code>typeerror('spam',)</code>

<code>&gt;&gt;&gt; generator.close()</code>

<code>don't forget to clean up when 'close()' is called.</code>

這和原來的生成器。這時候使用<code>gen_func.send()</code>會報錯,因為沒有接受

當某個生成器函數被執行時,生成器函數會開始執行,直到第一條yield表達式為止。

這條表達式會有一個傳回值,它取決于使用了生成器的哪個方法去恢複生成器函數的控制流。

next方法會恢複gen func的執行,這裡yield表達式會傳回值none。

而gen func會繼續運作到下一條yield語句,并将yield後面的值反回。

send和next類似,不同的是她可以傳入一個值作為yield表達式的計算值。如果用send去開啟gen func的執行,那麼參數必須為none,因為這時候沒有yield語句在等待傳回值。

<a href="https://www.python.org/dev/peps/pep-0342/">pep 342 — coroutines via enhanced generators</a>

new in version 2.5.

the yield expression is only used when defining a generator function, and can only be used in the body of a function definition. using a yield expression in a function definition is sufficient to cause that definition to create a generator function instead of a normal function.

when a generator function is called, it returns an iterator known as a generator. that generator then controls the execution of a generator function. the execution starts when one of the generator’s methods is called. at that time, the execution proceeds to the first yield expression, where it is suspended again, returning the value of expression_list to generator’s caller. by suspended we mean that all local state is retained, including the current bindings of local variables, the instruction pointer, and the internal evaluation stack. when the execution is resumed by calling one of the generator’s methods, the function can proceed exactly as if the yield expression was just another external call. the value of the yield expression after resuming depends on the method which resumed the execution.

all of this makes generator functions quite similar to coroutines; they yield multiple times, they have more than one entry point and their execution can be suspended. the only difference is that a generator function cannot control where should the execution continue after it yields; the control is always transferred to the generator’s caller.

5.2.10.1. generator-iterator methods

this subsection describes the methods of a generator iterator. they can be used to control the execution of a generator function.

note that calling any of the generator methods below when the generator is already executing raises a valueerror exception.

generator.next()

starts the execution of a generator function or resumes it at the last executed yield expression. when a generator function is resumed with a next() method, the current yield expression always evaluates to none. the execution then continues to the next yield expression, where the generator is suspended again, and the value of the expression_list is returned to next()‘s caller. if the generator exits without yielding another value, a stopiteration exception is raised.

<code>generator.send(value)</code>

resumes the execution and “sends” a value into the generator function. the value argument becomes the result of the current yield expression. the send() method returns the next value yielded by the generator, or raises stopiteration if the generator exits without yielding another value. when send() is called to start the generator, it must be called with noneas the argument, because there is no yield expression that could receive the value.

<code>generator.throw(type[, value[, traceback]])</code>

raises an exception of type type at the point where generator was paused, and returns the next value yielded by the generator function. if the generator exits without yielding another value, a stopiteration exception is raised. if the generator function does not catch the passed-in exception, or raises a different exception, then that exception propagates to the caller.

<code>generator.close()</code>

raises a generatorexit at the point where the generator function was paused. if the generator function then raises stopiteration (by exiting normally, or due to already being closed) or generatorexit (by not catching the exception), close returns to its caller. if the generator yields a value, a runtimeerror is raised. if the generator raises any other exception, it is propagated to the caller. close() does nothing if the generator has already exited due to an exception or normal exit.

here is a simple example that demonstrates the behavior of generators and generator functions:

<code># execution starts when 'next()' is called for the first time.</code>

<code># don't forget to clean up when 'close()' is called.</code>