【整理】RabbitMQ publish方法中的immediate和mandatory属性

      鉴于在 rabbitmq 的使用过程中，很多同学搞不清楚 basic.publish 接口中 mandatory 和 immediate 的背后含义，特搜集整理网上相关文章，以便彻底理解该问题。

===== 我是三体分隔线 ===== 

<a href="http://my.oschina.net/moooofly/blog/384969#">?</a>

1

2

3

4

<code>...</code>

<code>feature removal</code>

<code>23896 remove support</code><code>for</code> <code>amqp's</code><code>"immediate"</code> <code>publish mode</code>

5

6

7

8

9

10

11

12

<code>removal of</code><code>"immediate"</code> <code>flag</code>

<code>what changed? we removed support</code><code>for</code> <code>the rarely-used</code><code>"immediate"</code> <code>flag on amqp's basic.publish.</code>

<code>why on earth did you</code><code>do</code> <code>that? support</code><code>for</code> <code>"immediate"</code> <code>made many parts of the codebase</code><code>more</code> <code>complex,</code>

<code>particularly around mirrored queues. it also stood</code><code>in</code> <code>the way of our being able to deliver</code>

<code>substantial performance improvements</code><code>in</code> <code>mirrored queues.</code>

<code>what</code><code>do</code> <code>i need to</code><code>do</code><code>? if you just want to be able to publish messages that will be dropped</code><code>if</code> <code>they</code>

<code>are not consumed immediately, you can publish to a queue with a ttl of 0.</code>

<code>if you also need your publisher to be able to determine that this has happened, you can also use the</code>

<code>dlx feature to route such messages to another queue, from</code><code>which</code> <code>the publisher can consume them.</code>

13

14

<code>...publish(short reserved-1, exchange-name exchange, shortstr routing-key, bit mandatory, bit immediate)</code>

<code>bit mandatory</code>

<code>this flag tells the server how to react</code><code>if</code> <code>the message cannot be routed to a queue. if this flag is</code><code>set</code><code>,</code>

<code>the server will</code><code>return</code> <code>an unroutable message with a return method. if this flag is zero, the server</code>

<code>silently drops the message.</code>

<code>the server should implement the mandatory flag.</code>

<code>bit immediate</code>

<code>this flag tells the server how to react</code><code>if</code> <code>the message cannot be routed to a queue consumer immediately.</code>

<code>if this flag is</code><code>set</code><code>, the server will</code><code>return</code> <code>an undeliverable message with a return method. if this flag</code>

<code>is zero, the server will queue the message, but with no guarantee that it will ever be consumed.</code>

<code>the server should implement the immediate flag.</code>

amqp's basic.publish method has two flags - mandatory and immediate - that trigger checks at the server and result in messages getting returned to clients with basic.return when the necessary conditions aren't met. see http://www.rabbitmq.com/amqp-0-9-1-reference.html#basic.publish for the details. i can just about see why 'mandatory' may be useful, but 'immediate' has always struck me as a bizarre and i fear that anybody using it may well do so w/o fully understanding the semantics - can't blame them, it *is* bizarre. because of that, and since it complicates the code in many places, we are considering dropping support for 'immediate'. so is anybody actually using the 'immediate' flag?

i'd support this move. have never used "immediate".

just curious, are there any benefits for removing it -- is it preventing / complicating support for other features?

yes. it accounts for a fair bit logic scattered across the code base and we've just found another case where in order to significantly improve performance of mirrored queues we'd have to make yet more code aware of the 'immediate' flag.

well, the specific thing that we've bumped into in this case is that support for immediate is greatly slowing down ha queues. we could be smarter about this, at the cost of tangling the code further. so if we want to make ha queues faster (and we do!) we can either keep immediate and add a bunch of code at the same time, or remove immediate and delete a bunch of code. there are quite a number of places where we have notably greater complexity just to support this weird feature that (we suspect) no-one uses. it just feels very disproportionate.

matthias / simon, thanks for the background. i'm not using the immediate flag, but i've considered using it.  admittedly, i'm not fully aware of exactly how it works. it would be helpful to have a better understanding of the semantics. can you explain what the publisher is notified of (if anything?) when there are 10 queues bound to an exchange named highpriority, a message with immediate flag true is published to highpriority, and 7 out of the 10 queues have the message consumed immediately.

no notification is sent to the publisher in that case. a basic.return is only sent if *no* queues were able to send the message to a consumer immediately.

here are a few more warts about 'immediate':

- while a message might get sent out to a consumer straight away, there is of course no guarantee that such a message will reach the consumer, get processed and ack'ed. if any of these steps go wrong the message will end up in the queue (unless the consuming happened with no-ack, in which case it is simply lost), even though supposedly it was delivered immediately.

- a queue can have a consumer and yet 'immediate' delivery may fail (and a basic.return issued) because: a) the consumer isn't quick enough and a backlog of messages has built up, b) the consumer has configured a 'basic.qos' prefetch count and that the limit has been reached

- the interaction with transactions is not at all obvious. this does apply to 'mandatory' too though. see http://www.rabbitmq.com/semantics.html#tx

matthias, thanks for the explanation. considering your points below, i now have a better understanding on your thought that the actual use-cases and practicality to publish messages with the immediate flag are minimal. in light of the details, i won't be using it -- faster mirrored queues sounds better to me. :o)

couldn't it be possible emulate immediate by delivering to a ttl'ed queue, then having a dead-letter queue that the publisher consumes from to be notified that a message has not been consumed?

yes. it's not an exact emulation, but arguably more flexible, e.g. it gives greater control over how the 'returned' messages are dealt with.

plus 1 for faster ha queues and less complexity in the code base.

15

16

17

18

19

20

21

22

<code>the immediate and mandatory fields are part of the amqp specification, and are also covered</code><code>in</code> <code>the</code>

<code>rabbitmq faq to clarify how its implementers interpreted their meaning:</code>

<code>mandatory</code>

<code>this flag tells the server how to react</code><code>if</code> <code>a message cannot be routed to a queue. specifically,</code><code>if</code>

<code>mandatory is</code><code>set</code> <code>and after running the bindings the message was placed on zero queues</code><code>then</code> <code>the message</code>

<code>is returned to the sender (with a basic.</code><code>return</code><code>). if mandatory had not been</code><code>set</code> <code>under the same</code>

<code>circumstances the server would silently drop the message.</code>

<code>or</code><code>in</code> <code>my words,</code><code>"put this message on at least one queue. if you can't, send it back to me."</code>

<code>immediate</code>

<code>for a message published with immediate</code><code>set</code><code>,</code><code>if</code> <code>a matching queue has ready consumers</code><code>then</code> <code>one of them</code>

<code>will have the message routed to it. if the lucky consumer crashes before ack'ing receipt the message</code>

<code>will be requeued and</code><code>/or</code> <code>delivered to other consumers on that queue (</code><code>if</code> <code>there's no crash the messaged</code>

<code>is ack</code><code>'ed and it'</code><code>s all</code><code>done</code> <code>as per normal). if, however, a matching queue has zero ready consumers the</code>

<code>message will not be enqueued</code><code>for</code> <code>subsequent redelivery on from that queue. only</code><code>if</code> <code>all of the matching</code>

<code>queues have no ready consumers that the message is returned to the sender (via basic.</code><code>return</code><code>).</code>

<code>or</code><code>in</code> <code>my words, "if there is at least one consumer connected to my queue that can take delivery of a</code>

<code>message right this moment, deliver this message to them immediately. if there are no consumers connected</code>

<code>then</code> <code>there</code><code>'s no point in having my message consumed later and they'</code><code>ll never see it. they snooze, they lose."</code>

最后给出两篇中文说明，可以帮助理解：