Programming Leftovers
-
Are you sure your Python ABI is actually stable?
The age of Python’s packaging ecosystem also sets it apart: among general-purpose languages, it is predated only by Perl’s CPAN. This, combined with the mostly independent development of packaging tooling and standards, has made Python’s ecosystem among the more complex of the major programming language ecosystems. Those complexities include: [...]
-
Who controls parallelism? A disagreement that leads to slower code
If you’re using NumPy, Polars, Zarr, or many other libraries, setting a single environment variable or calling a single API function might make your code run 20%-80% faster. Or, more accurately, it may be that your code is running that much more slowly than it ought to.
The problem? A conflict over who controls parallelism: your application, or the libraries it uses.
Let’s see an example, and how you can solve it.
-
[Old] "How do I test X" is almost always answered with "by controlling X"
First, there is the question: "can I control X?". Because if you cannot, testing it becomes impossible. When X is some external service, or tool, for example a payment provider or email-server, and there is no way to control it, you cannot (, and therefore should not) test it.
-
On Launching
Consumer products lend themselves more to feature creep, which is the enemy of PMF. It's never been easier to create software, which means faster MVPs, but also the possibility to build more out before you even reach your users.
-
Why middleware may not be the right abstraction for your data policies.
Every frontend developer knows, or at least suspects, that backends are hard. But why would that be?
There is nothing intrinsically difficult with the business logic of backends: after all, it’s just code and algorithms. A fizzbuzz is a fizzbuzz: equally useless on the backend as it is on the frontend.
-
More Evidence for Problems in VM Warmup
VMs start programs running in an interpreter where they observe which portions run frequently. Those portions are then compiled into machine code which can then be used in place of the interpreter. The period between starting the program and all of the JIT compilation completing is called the warmup time. Once warmup is complete, the program reaches a steady state of peak performance.
At least, that's what's supposed to happen: our work showed that, on widely studied small benchmarks, it often doesn't. Sometimes programs don't hit a steady state; sometimes, if they do reach a steady state, it's slower than what came before; and some programs are inconsistent in whether they hit a steady state or not. A couple of examples hopefully help give you an idea. Here's an example of a "good" benchmark from our dataset which starts slow and hits a steady state of peak performance: [...]
-
Making a Go program 42% faster with a one character change
If you read the title and thought “well, you were probably just doing something silly beforehand”, you’re right! But what is programming if not an exercise in making silly mistakes? Tracking down silly mistakes is where all the fun is to be had!
I’ll also state the usual benchmarking caveat up front: the 42% speedup was measured while running the program on my data on my computer, so take that number with a big old pinch of salt.
-
Promoting the Use of R in Mali
The R Consortium recently caught up with Fousseynou Bah of the Bamako Data Science Group (also on Facebook) and talked about the budding R community in Mali. Online events allowed the group to broaden its horizons and invite international speakers to present at their events. They hope to host hybrid events in the future to make the most out of both online and physical event formats.