Programming Leftovers

posted by Roy Schestowitz on Sep 27, 2023

• Game of Trees 0.93 released

  Version 0.93 of Game of Trees has been released (and the port updated).

• Comments On Comments

  The state of comments in many codebases is not ideal. Often, we can and should
  replace should comments with code that checks;
  replace what comments with meaningful identifiers; and
  explain why more often and more thoroughly.

• Exercise in Coherence

  There is an exercise in Saunders Mac Lane’s “Categories for the Working Mathematician” that was a lesson in humility for me. Despite several hints provided by Mac Lane, all my attempts to solve it failed. Finally my Internet search led me to a diagram that looked promissing and it allowed me to crack the problem.

  Why do I think this is interesting? Because it shows the kind of pattern matching and shape shifting that is characteristic of categorical proofs. The key is the the use of visual representations and the ability to progressively hide the details under the terseness of notation until the big picture emerges.

• Scrum Destroys Consulting Firms

  Agile and Scrum are quickly destroying the human species, I’ve even written a book about it. Here is how it is single-handedly destroying IT consulting firms. I’ve been at IT consulting for over 30 years now and watched as Agile rotted firms from the inside out. What destroyed them (and most IT projects) was the adoption of scrum.

• "Help, iterators made my Rust program slower!"

  Recently in a programming community I belong to, someone presented a problem. They had a Rust program which was using threads and for loops. When they updated the code to use iterators, it got dramatically slower. Why did this happen?

• Mastering Histogram Breaks in R: Unveiling the Power of Data Visualization

  Histograms are a fundamental tool in data analysis and visualization, allowing us to explore the distribution of data quickly and effectively. While creating a histogram in R is straightforward, specifying breaks appropriately can make a world of difference in the insights you can draw from your data. In this blog post, we will delve into the art of specifying breaks in a histogram, providing you with multiple examples and encouraging you to experiment on your own.

  Before we get started, it’s worth mentioning that this topic has been explored in depth by Steve Sanderson in his previous blog post. If you’re interested in diving even deeper, make sure to check out his article here: Steve’s Blog Post on Optimal Binning. Now, let’s embark on our journey into the fascinating world of histogram breaks in R.

• Making a function that defines functions in GNU Emacs ELisp

  If you try this in an Emacs *scratch* buffer, it may well work. If you put this into a .el file (one that has no special adornment) and use it to create a bunch of functions in that file, then try to use one of them, Emacs will tell you 'mh-visit-: Symbol’s value as variable is void: folder-name'. This is because folder-name is dynamically scoped, and so is not captured by the lambda we've created here; the 'folder-name' in the lambda is just a free-floating variable. As far as I know, there is no way to create a lexically bound variable and a closure without making all elisp code in the file use lexical binding instead of dynamic scoping.

• Syntax Error #7

  Syntax Error is a newsletter about debugging for developers, students, hobbyists, curious and duck fans. In this September issue of Syntax Error we look do a bit of roleplaying to take a look at bug that sneaked into a system and how to debug it. Read full article at syntaxerror.tech/syntax-error-7/and either subscribe to the email or RSS feed to catch all of them.

• Notes from “Weathering Software Winter” by Devine Lu Linvega

  Really paints a picture of how fragile modern digital hardware and software is to any kind of transplantation outside a very specific context (e.g. access to ubiquitous internet connectivity and power).

  What follows are a few of the excerpts that stuck out to me and why I liked them.

• How Hard is it to Adapt a Memory Allocator to CHERI?

  CHERI is a set of things (adapted CPUs, adapted operating systems, adapted libraries) that, collectively, aim to stop software bugs becoming security flaws. If you'd like to know more, I gave some of my thinking on CHERI in Two Stories for "What is CHERI?".

  In this post I'm going to flesh out an observation I made in that post, which is that some software needs thoughtful adaption to CHERI if we want to get the security advantages we hope for. Exactly what that thoughtful adaption might look like will vary, probably substantially, between different pieces of software. What, for instance, might it look like for critical, widely used, components? In this post I'm going to look at how memory allocators (henceforth "allocators"), one of software's most fundamental building blocks, can be adapted to CHERI. If you find this interesting, but want greater depth than I'll go into here, you might be interested in the paper Picking a CHERI Allocator: Security and Performance Considerations that this post is based upon.

• Tetris Clone Uses 1000 Lines Of Code, And Nothing Else

  If you’re programming on a modern computer, you typically make use of lots of work done by other people. There’s operating systems to abstract away the complexities of modern hardware, standard libraries to implement common tasks, and tons of third-party libraries that prevent you from having to reinvent the wheel all the time: you’re definitely not the first one trying to draw graphics onto a screen or store data in a file.

• Processes, Threads, And… Fibers?

  You’ve probably heard of multithreaded programs where a single process can have multiple threads of execution. But here is yet another layer of creating multitasking programs known as a fiber. [A Graphics Guy] lays it out in a lengthy but well-done post. There are examples for both x64 and arm64, although the post mainly focuses on x64 for Windows. However, the ideas will apply anywhere.

• PostScript’s sudden death in Sonoma

  PostScript is an old stack-based interpreted language designed at a time when code security had barely been conceived, and malicious software hardly existed. Among its attractive features is the fact that any PostScript object can be treated as data, or executed as part of a program, and can itself generate new objects that can in turn be executed. More recently, security researchers have drawn attention to the fact that it’s a gift for anyone wishing to write and distribute malicious code. As it’s effectively an image format, embedding malware inside a PostScript file could enable that to be run without user interaction, as with some other graphics formats.

• Mastering Data Visualization with Pairs Plots in Base R

  Data visualization is a crucial tool in data analysis, allowing us to gain insights from our data quickly. One of the fundamental techniques for exploring relationships between variables is the pairs plot. In this blog post, we’ll dive into the world of pairs plots in base R. We’ll explore what they are, why they are useful, and how to create and interpret them.

• Why Not Rust Crypto?

  One of the most frequent questions I receive about *ring* is "Why not Rust Crypto?," referring to the Rust Crypto project led by Tony Arcieri and Artyom Pavlov. People ask this because *ring* and Rust Crypto implement similar functionality, and Rust Crypto is 100% Rust, while *ring* is a mix of Rust, C, and assembly language. Why do these things exist in parallel instead of working together? Why don't I help the Rust Crypto project?

• Exploratory Data Analysis for Humanities Data

  One of the goals of the course was to try to teach enough computing to a mostly non-technical and definitely not computer-experienced population that they could use computers to do an interesting and new (to them) exploration of some dataset that they found intriguing.

  After much discussion, Meredith and I decided that for the programming aspects, we would devote half of each class meeting to a "studio" where I would lead the students through hands-on computing exercises on some small dataset, followed by having the students do similar exercises on larger datasets outside the class.

  We planned to spend one week on Unix file system and command-line basics, a week on grep and similar tools (including a real text editor), a week on Awk, and a couple of weeks on Python. Not surprisingly, everything took longer than expected, and a fair number of students struggled mightily in spite of help from their peers, Meredith and myself, and two exceptional colleagues from the CDH, Sierra Eckert and Ryan Heuser.

• A deep dive into Clang's source file compilation

  This post describes how different libraries work together to create the final relocatable object file.

• Build Great Software By Repeatedly Encountering It

  You can get pretty damn far by incessantly encountering a piece of software over and over, fixing everything you find along the way that doesn’t feel right.

• Irreducible complexity

  I tell myself I should break tasks up into smaller steps. Doing so makes the overall endeavour easier, because the objectives and requirements for smaller things are easier to conceptualise and track. It reduces the barrier to entry, by making a massive task seem less intimidating. You also get the satisfaction of ticking off a few things off a list, rather than having one glaring item judging you for not finishing it off after weeks of work.

• GTK Applications Are Largely Irrelevant.

  Since I switched to KDE and stopped hitting myself over the head with all of that GNOME stupidity which made using my PC very unpleasant, I’ve identified maybe a few applications that make any substantial usage of GNOME that I actually still have.

  You should probably just remove as many as you can, honestly.

• Old-School CGI Scripts!

  All in all this took only two hours or so. Old-school CGI is pretty awesome like that - Hidden values meant the whole thing could be stateless: [...]

• Accidentally Turing-Complete

  Some things were not supposed to be Turing-complete. This is a collection of such accidents.

  Stuff which is somehow limited (stack overflows, arbitrary configuration, etc) is still considered Turing complete, since all "physical" Turing machines are resource limited.

• Finding a circle in a chart by @ellis2013nz

  Here’s how I do this brute force method in R: [...]

• Notes to self on Woodpecker-CI

  Gitea’s Actions didn’t convince me when I toyed with them, so I looked further and found Woodpecker-CI which I set up using this guide after removing the actual Gitea container because I already have that running as a service. Woodpecker-CI’s documentation is plentiful and quite understandable, but I missed an introduction on why to use docker-compose and not the binaries. After toying around I decided to actually do the Docker dance as I’d be needing that for the Woodpecker agents anyway.

• Parsing integers quickly with AVX-512

  It is very fast: you can parse a sequence of random 32-bit integers at about 40 cycles per integer, using about 128 instructions.

• 2023.39 Releaseses

  Justin DeVuyst has released the 2023.09 release of the Rakudo Compiler, Haytham Elganiny has released a new version of the Pakku package manager, and two authors published their first module on the Raku Ecosystem this week. Not to mention 15 other authors providing updates to existing modules and two new modules! A releasy week for sure!

• Upsert in SQL

  Upsert is an operation that ➊ inserts new records into the database and ➋ updates existing ones. Let's see how it works in different DBMS. The examples are interactive, so you can read and practice.

• Powerlevel10k

  Powerlevel10k and Starship are both powerful. Starship works with many shell, while Powerlevel only works with ZSH, so that is the first reasons to exclude it. If you want a common prompt across Bash ZSH, Fish shell and ZSH, Starship is the one you want. Also Starship has invested more time into documenting the available config options.

• How Unix shells used to be used as an access control mechanism

  Once upon a time, one of the ways that system administrators controlled who could log in to what server was by assigning special administrative shells to logins, either on a particular system or across your entire server fleet. Today, special shells (mostly) aren't an effective mechanism for this any more, so modern Unix people may not have much exposure to this idea. However, vestiges of this live on in typical Unix configurations, in the form of /sbin/nologin (sometimes in /usr/sbin) and how many system accounts have this set as their shell in /etc/passwd.