26 Jun 2021

HAProxy - a tale of two queues

A tale of two queues In 2020 many services saw a drastic spike in their traffic. We saw almost 5-8x sustained higher traffic than our previous max traffic peaks. This lead to our HAProxy instances struggling to keep up.
10 Aug 2020

Scaling RabbitMQ with shared exchanges

Sharded exchanges can be enabled by using the a plugin with rabbitmq. It is useful for distributing load across multiple queue for the same routing keys.

Using sharded exchanges, you can shard messages across the cluster so that a single node doesn’t become overwhelmed.

16 Jan 2018

Good old bash for startup dependency management

A simple and clean way to wait for dependencies to be available in Unix*y environments. I often use this script to orchestrate my docker containers.
10 Dec 2017

A fun integer overflow bug investigation

Sometimes we run into weird bugs or buggish situations that are frustrating but are actually fun to solve. I ran into a similar problem recently. A good old NumberFormatException in the open-source analytics dashboard service: Metabase.

We use Metabase to create product metric dashboards. Recently, they allowed caching long running queries and it allows you to set the max entry size as a MAX CACHE ENTRY SIZE setting in kilobytes. This is where the fun starts!

30 Jul 2017

Shoveling data to Redshift with pipes and filters

Let’s say you have a datasource with multiple tables with a few Million rows each. A typical architecture will probably include some kind of an Analytics warehouse. For instance, AWS Redshift is a pretty good implementation and understands the PSQL dialect.

To keep the analytics data fresh, we need some way of shoveling data into that DB. In this scenario, we used Mysql as the OLTP-style, main application Datasource and Redshift as the analytics cluster.