Sharded exchanges can be enabled by using the a plugin with rabbitmq. It is useful for distributing load across multiple queue for the same routing keys.
Using sharded exchanges, you can shard messages across the cluster so that a single node doesn’t become overwhelmed.
Sometimes we run into weird bugs or buggish situations that are frustrating but are actually fun to solve. I ran into a similar problem recently. A good old
NumberFormatException in the open-source analytics dashboard service:
Metabase to create product metric dashboards. Recently, they allowed caching long running queries and it allows you to set the max entry size as a
MAX CACHE ENTRY SIZE setting in
kilobytes. This is where the fun starts!
Let’s say you have a datasource with multiple tables with a few Million rows each. A typical architecture will probably include some kind of an Analytics warehouse. For instance, AWS Redshift is a pretty good implementation and understands the PSQL dialect.
To keep the analytics data fresh, we need some way of shoveling data into that DB. In this scenario, we used
Mysql as the OLTP-style, main application Datasource and
Redshift as the analytics cluster.