Language Selection

English French German Italian Portuguese Spanish

Programming Leftovers

Filed under
Development

  • A simple update queue for HN&&LO - The occasional scrivener - Gustaf Erikson's blog

    One thing that’s bothered me for a while is that I don’t keep the score and comments for Hackernews entries up to date in a timely manner. So far, I’ve been going back and re-reading last month’s entries but this means that the daily view generally isn’t up-to-date.

    The HN API is hosted on Firebase and thus has the ability to get real-time data, but there’s no Perl interface to it, and I really don’t have time to tinker with an entire async setup for a page that’s updated every hour at most.

  • Beyond the Four-Document Model • Buttondown

    We pejoratively talk about “copy and paste programming” as something done by incompetent programmers who don’t know how to write code from scratch.

  • Tail I lose - All this

    I’ve been using the tail command for about 25 years, so you might think I’d know something about it. But last week, as I was putting together a short shell script (not this one), I opened the man page for tail and learned something new. Two things, actually, which surprised me, as tail doesn’t really do that much.

  • How Kubernetes Broke Git

    How did Kubernetes push git to its limit? Some stories from my time (2016-2019) working on Kubernetes.

    No atomicity across subprojects – In 2016, Kubernetes was still a monorepo. Everything was developed in a single repository. This meant that developers could reuse CI infrastructure easily and ensure that changes to the different components (e.g., kube-proxy, kube-apiserver, kubelet) would work together.

    However, downstream projects needed to build on the API. That meant vendoring parts of Kubernetes or separating API specifications from the code.

    Transitioning to different subproject repositories wasn't easy. It happened gradually and painfully. The plan was to continue developing the libraries in the monorepo (under a special staging subfolder) and sync the changes to new project repositories. But, of course, this led to all sorts of problems – unreliable syncs, missing commit history, different commit SHAs, and more.

    The solution might seem simple, but even simple problems become difficult at scale, especially when many different people and organizations are involved.

    A system that could record atomic commits across projects or a better submodule experience would have allowed for more flexible developer organization, especially as the project grew to a new scale.

    No benevolent dictator to merge patches – While the Linux kernel successfully scaled on git, Kubernetes had a different governing model. There was no Linus Torvalds to collect patches and manually apply them.

  • Things I wish everyone knew about Git (Part II)

    A Git repository is an append-only filesystem. You can add snapshots of files and directories, but you can't modify or delete anything. Git commands sometimes purport to modify data. For example git commit --amend suggests that it amends a commit. It doesn't. There is no such thing as amending a commit; commits are immutable.

    Rather, it writes a completely new commit, and then kinda turns its back on the old one. But the old commit is still in there, pristine, forever.

    In a Git repository you can lose things, in the sense of forgetting where they are. But they can almost always be found again, one way or another, and when you find them they will be exactly the same as they were before. If you git commit --amend and change your mind later, it's not hard to get the old ⸢unamended⸣ commit back if you want it for some reason.

  • Introducing Cupcake

    If your goal is to prevent insecure software from being developed, the worst thing you can do is to dictate a bunch of increasingly arcane-sounding requirements to developers working against a tight deadline then shame them when they fuck it up.

    A much better strategy is to give developers a tool, with minimal fuss and dependencies, that’s tuned for security out-of-the-box and always does The Right Thing for them.

    Even better is if you introduce it as a write-less, do-more tool that saves developers time and frustration. This incentivizes them to use it over a quick-and-dirty, error-prone, artisanal approach to solving the same problems.

  • FOSS4Spectroscopy: R vs Python | R-bloggers

    If you aren’t familiar with it, the FOSS for Spectroscopy web site lists Free and Open Source Software for spectroscopic applications. The collection is of course never really complete, and your package suggestions are most welcome (how to contribute). My methods for finding packages are improving and at this point the major repositories have been searched reasonably well.

    A few days ago I pushed a major update, and at this point Python packages outnumber R packages more than two to one. The update was made possible because I recently had time to figure out how to search the PyPi.org site automatically.

  • Commoditization of Large Language Models

    The company behind GPT-3, OpenAI, hasn't released the non-public datasets or model, but they are all trivially recreated without much issue.

    You only need about $12 million to train the model from scratch. And you don't even need to do that anymore.

    There are plenty of open-sourced models to pick from. There's GPT-J from a set of independent researchers. Meta is open-sourcing OPT-175B. Stanford researchers are open-sourcing their model, Diffusion-LM.

  • Fixed vs. random effects for browsing data – a simulation | R-bloggers

    When you work with trace data — data that emerge when people interact with technology — you will notice that such data often have properties that open up questions about statistical modelling. I currently work with browsing records, obtained at several times from the same users (i.e., a panel data set). A first typical characteristic of such data: Browsing behaviors are skewed. If you’re interested in people reading politically extreme web sites, you will find a few people doing it a lot, and most people doing little to none.

    A second characteristic relates to the panel nature of the data. If you’re looking at people’s visits to online shops, they do not change their habits much over time — at least these within-person differences are not as great as the differences across people. Panel data lend itself to hierarchical modelling, for example with (1) fixed effects (FE) or (2) random-effects multilevel modelling (RE). For commonalities and differences between these two modelling approaches, see for example Gelman and Hill (2006).[1] The amount of within-person variation relative to between-person variation has important implications for these two approaches.

    Below, I simulate the performance of FE vs. RE models, with these data characteristics in mind. I am mainly interested in the statistical power of each model, although my code can be easily adapted to examine, for example, bias and the RSME. The code builds on two sources: Conceptually, on the paper “Should I Use Fixed or Random Effects?” by Clark and Linzer (2015)[2]. In terms of code architecture, on the R package simhelpers, and its documentation.[3]

  • How many languages do we need to learn about responsible machine learning? useR! 2022 Conference | R-bloggers

    It might seem that, we don’t have much choice, because the most popular languages in data science are R and Python or if you prefer Python and R. But today we are not talking about these languages!

    During the useR!2022 conference, we can meet with English, Spanish and French. Because of the fact that only English is close to us we created our workshops and papers in this language. But! Is it possible to organize a workshop in another language?

    Yes, it can be done and we did it! We submitted workshop in 5 languages in parallel (English, Spanish, Polish, Turkish and Vietnamese)!

  • How to Calculate Relative Frequencies in R? | R-bloggers

    How to Calculate Relative Frequencies in R?, The relative frequencies/proportions of values in one or more columns of a data frame can frequently be calculated in R.

More in Tux Machines

today's howtos

  • How to Change Comment Color in Vim – Fix Unreadable Blue Color

    Are you annoyed about the comment color in vim? The dark blue color of the comment is often hard to read. In this tutorial, we learn how to change the comment color in Vim. There are few methods we can use to look vim comment very readable.

  • How to Add Repository to Debian

    APT checks the health of all the packages, and dependencies of the package before installing it. APT fetches packages from one or more repositories. A repository (package source) is basically a network server. The term "package" refers to an individual file with a .deb extension that contains either all or part of an application. The normal installation comes with default repositories configured, but these contain only a few packages out of an ocean of free software available. In this tutorial, we learn how to add the package repository to Debian.

  • Making a Video of a Single Window

    I recently wanted to send someone a video of a program doing some interesting things in a single X11 window. Recording the whole desktop is easy (some readers may remember my post on Aeschylus which does just that) but it will include irrelevant (and possibly unwanted) parts of the screen, leading to unnecessarily large files. I couldn't immediately find a tool which did what I wanted on OpenBSD [1] but through a combination of xwininfo, FFmpeg, and hk I was able to put together exactly what I needed in short order. Even better, I was able to easily post-process the video to shrink its file size, speed it up, and contort it to the dimension requirements of various platforms. Here's a video straight out of the little script I put together: [...]

  • Things You Can And Can’t Do

    And it got me thinking about what you can and can’t do — what you do and don’t have control over.

  • allow-new-zones in BIND 9.16 on CentOS 8 Stream under SELinux

    We run these training systems with SELinux enabled (I wouldn’t, but my colleague likes it :-), and that’s the reason I aborted the lab: I couldn’t tell students how to solve the cause other than by disabling SELinux entirely, but there wasn’t enough time for that.

  • Will the IndieWeb Ever Become Mainstream?

    This is an interesting question, thanks for asking it, Jeremy. I do have some history with the IndieWeb, and some opinions, so let’s dive in.

    The short answer to the question is a resounding no, and it all boils down to the fact that the IndieWeb is really complicated to implement, so it will only ever appeal to developers.

  • How to Install CUPS Print Server on Ubuntu 22.04

    If your business has multiple personal computers in the network which need to print, then we need a device called a print server. Print server act intermediate between PC and printers which accept print jobs from PC and send them to respective printers. CUPS is the primary mechanism in the Unix-like operating system for printing and print services. It can allow a computer to act as a Print server. In this tutorial, we learn how to set up CUPS print server on Ubuntu 22.04.

Open Hardware: XON/XOFF and Raspberry Pi Pico

  • From XON/XOFF to Forward Incremental Search

    In the olden days of computing, software flow control with control codes XON and XOFF was a necessary feature that dumb terminals needed to support. When a terminal received more data than it could display, there needed to be a way for the terminal to tell the remote host to pause sending more data. The control code 19 was chosen for this. The control code 17 was chosen to tell the remote host to resume transmission of data.

  • Raspberry Pi Pico Used in Plug and Play System Monitor | Tom's Hardware

    Dmytro Panin is at it again, creating a teeny system monitor for his MacBook from scratch with help from our favorite microcontroller, the Raspberry Pi Pico. This plug-and-play system monitor (opens in new tab) lets him keep a close eye on resource usage without having to close any windows or launch any third-party programs. The device is Pico-powered and plugs right into the MacBook to function. It has a display screen that showcases a custom GUI featuring four bar graphs that update in real-time to show the performance of different components, including the CPU, GPU, memory, and SSD usage. It makes it possible to see how hard your PC is running at a glance.

Security Leftovers

How to Apply Accent Colour in Ubuntu Desktop

A step-by-step tutorial on how to apply accent colour in Ubuntu desktop (GNOME) with tips for Kubuntu and others. Read more