Databases and Storage With Free Software
-
SQL Maxis: Why We Ditched RabbitMQ And Replaced It With A Postgres Queue
We recently ripped out RabbitMQ from our system and replaced it with a queue built on top of our Postgres db and written in SQL. The change took half a day to implement + test, and had a net delta of -580 LOC. Much more importantly, it drastically improved the reliability and resiliency of our system. This is a writeup of that change, the rationale behind it, and how we did it.
Note: this is not an indictment against RabbitMQ or other queuing systems. It’s quite possible (likely, really) that we were holding it wrong. If anything, it’s a push to keep things simple. And, of course, a very biased reminder that SQL is unbelievably powerful. In fact, it often really is all you need. Do we dream of a world where everything is SQL? Maybe.
-
How does database sharding work?
If you’ve used Google or YouTube, you’ve probably accessed sharded data. Sharding helps you scale out your database by storing partitions of your data across multiple servers instead of putting everything on a single giant one. This post will walk through how database sharding works, how to think about implementing your own sharded database, and some useful tools out there that can help, with a particular focus on MySQL and Postgres.
-
A distributed database load-balancing architecture with ShardingSphere
Apache ShardingSphere is a distributed database ecosystem that transforms any database into a distributed database and enhances it with data sharding, elastic scaling, encryption, and other capabilities. In this article, I demonstrate how to build a distributed database load-balancing architecture based on ShardingSphere and the impact of introducing load balancing.
-
How collation of punctuation and whitespace works
First, let’s think about what we might like to happen. Maybe we want to treat punctuation and whitespace characters just like other characters. They sort in some relative location, and that’s it. Or maybe we want to mostly ignore these characters for sorting. Both of these are valid choices, and where people get upset is mainly when they expected one and get the other. In fact, Unicode allows both of these choices (and more!), and the implementation has to pick one or make users select one. (To keep it simple, I’ll just write “Unicode” in this article to mean one of the various standards that affect collation of Unicode characters. See the previous article for details.)
-
Ubuntu Blog: Cloud storage at the edge with MicroCeph
Over the years, our enterprise data centre users have told us how much they love the end to end experience of an application-centric solution like Juju to manage their entire infrastructure. Juju is a software operator framework that abstracts the specifics of operating complex software and makes it simple and straightforward to deploy, operate and relate complimentary pieces of software, reducing cost and providing flexibility. Charmed Ceph uses Juju to manage the entire lifecycle of deployment, configuration, and operations of a Ceph cluster.
But, what if the use case is different? Maybe all that is required is simple and repeatable Ceph storage deployments.
-
Free Dolly: Introducing the World's First Truly Open Instruction-Tuned LLM
databricks-dolly-15k contains 15,000 high-quality human-generated prompt / response pairs specifically designed for instruction tuning large language models. Under the licensing terms for databricks-dolly-15k (Creative Commons Attribution-ShareAlike 3.0 Unported License), anyone can use, modify, or extend this dataset for any purpose, including commercial applications.