self-proclaimed internet enthusiast!

Message Queues ¶

ZeroMQ ¶

• Documentation
• ZeroMQ and Ruby

RabbitMQ ¶

• Documentation
• Reasoning about memory use
• Reliability
• Clustering
• Classic cluster formation
• Node count
• Long vs short node names
• Hostname resolution
• Ports - amqp 5672 - amqp-tls 5671 - management/api 15672 - management/api-tls 15671 - prometheus 15692
• Proxy protocol - for AWS ELB, HAProxy etc.
• Cookie

Membership

• https://www.rabbitmq.com/clustering.html#cluster-membership
• All data/state required for the operation of a RabbitMQ broker is replicated across all nodes.
• An exception to this are message queues, which by default reside on one node, though they are visible and reachable from all nodes.
• To replicate queues across nodes in a cluster, use a queue type that supports replication.

Clients

• https://www.rabbitmq.com/clustering.html#clustering-and-clients
• A client can connect to any node and perform any operation.
• Nodes will route operations to the quorum queue leader or queue master replica transparently to clients.
  • https://www.rabbitmq.com/quorum-queues.html
  • https://www.rabbitmq.com/ha.html#master-migration-data-locality
• All supported messaging protocols a client is only connected to one node at a time.
• In case of a node failure, clients should be able to reconnect to a different node, recover their topology and continue operation. For this reason, most client libraries accept a list of endpoints (hostnames or IP addresses) as a connection option. The list of hosts will be used during initial connection as well as connection recovery, if the client supports it.

Queues

• With quorum queues, clients will only be able to perform operations on queues that have a quorum of replicas online.
• This topic is covered in the Quorum Queues and Classic Mirrored Queues guides.
  • https://www.rabbitmq.com/quorum-queues.html
  • https://www.rabbitmq.com/ha.html
• RabbitMQ gets an HA upgrade: https://www.rabbitmq.com/blog/2020/04/20/rabbitmq-gets-an-ha-upgrade/
  • You cannot turn a classic queue into a quorum queue via policies though. The x-queue-type property set to quorum must be included in the queue declaration. It is a quorum queue from birth.
• With classic mirrored queues, there are scenarios where it may not be possible for a client to transparently continue operations after connecting to a different node. They usually involve non-mirrored queues hosted on a failed node.
  • https://www.rabbitmq.com/ha.html#non-mirrored-queue-behavior-on-node-failure

Failure

• https://www.rabbitmq.com/clustering.html#clustering-dealing-with-failure
• Forcing node boot: https://www.rabbitmq.com/clustering.html#forced-boot

Stats and Metrics

• https://www.rabbitmq.com/clustering.html#clustering-and-stats
• https://www.rabbitmq.com/prometheus.html#overview-prometheus
  • https://grafana.com/rabbitmq

Cluster Setup

• Starting: https://www.rabbitmq.com/clustering.html#starting
• Creating: https://www.rabbitmq.com/clustering.html#creating
  • Join cluster: stop rabbitmq, reset node, join_cluster, start rabbitmq
• Restarting cluster nodes: https://www.rabbitmq.com/clustering.html#restarting
• Rolling restarts and readiness probes
  • https://www.rabbitmq.com/clustering.html#restarting-readiness-probes
  • https://www.rabbitmq.com/clustering.html#restarting-transcript