Programming Leftovers

-
A simple update queue for HN&&LO - The occasional scrivener - Gustaf Erikson's blog
One thing that’s bothered me for a while is that I don’t keep the score and comments for Hackernews entries up to date in a timely manner. So far, I’ve been going back and re-reading last month’s entries but this means that the daily view generally isn’t up-to-date.
The HN API is hosted on Firebase and thus has the ability to get real-time data, but there’s no Perl interface to it, and I really don’t have time to tinker with an entire async setup for a page that’s updated every hour at most.
-
Beyond the Four-Document Model • Buttondown
We pejoratively talk about “copy and paste programming” as something done by incompetent programmers who don’t know how to write code from scratch.
-
Tail I lose - All this
I’ve been using the tail command for about 25 years, so you might think I’d know something about it. But last week, as I was putting together a short shell script (not this one), I opened the man page for tail and learned something new. Two things, actually, which surprised me, as tail doesn’t really do that much.
-
How Kubernetes Broke Git
How did Kubernetes push git to its limit? Some stories from my time (2016-2019) working on Kubernetes.
No atomicity across subprojects – In 2016, Kubernetes was still a monorepo. Everything was developed in a single repository. This meant that developers could reuse CI infrastructure easily and ensure that changes to the different components (e.g., kube-proxy, kube-apiserver, kubelet) would work together.
However, downstream projects needed to build on the API. That meant vendoring parts of Kubernetes or separating API specifications from the code.
Transitioning to different subproject repositories wasn't easy. It happened gradually and painfully. The plan was to continue developing the libraries in the monorepo (under a special staging subfolder) and sync the changes to new project repositories. But, of course, this led to all sorts of problems – unreliable syncs, missing commit history, different commit SHAs, and more.
The solution might seem simple, but even simple problems become difficult at scale, especially when many different people and organizations are involved.
A system that could record atomic commits across projects or a better submodule experience would have allowed for more flexible developer organization, especially as the project grew to a new scale.
No benevolent dictator to merge patches – While the Linux kernel successfully scaled on git, Kubernetes had a different governing model. There was no Linus Torvalds to collect patches and manually apply them.
-
Things I wish everyone knew about Git (Part II)
A Git repository is an append-only filesystem. You can add snapshots of files and directories, but you can't modify or delete anything. Git commands sometimes purport to modify data. For example git commit --amend suggests that it amends a commit. It doesn't. There is no such thing as amending a commit; commits are immutable.
Rather, it writes a completely new commit, and then kinda turns its back on the old one. But the old commit is still in there, pristine, forever.
In a Git repository you can lose things, in the sense of forgetting where they are. But they can almost always be found again, one way or another, and when you find them they will be exactly the same as they were before. If you git commit --amend and change your mind later, it's not hard to get the old ⸢unamended⸣ commit back if you want it for some reason.
-
Introducing Cupcake
If your goal is to prevent insecure software from being developed, the worst thing you can do is to dictate a bunch of increasingly arcane-sounding requirements to developers working against a tight deadline then shame them when they fuck it up.
A much better strategy is to give developers a tool, with minimal fuss and dependencies, that’s tuned for security out-of-the-box and always does The Right Thing for them.
Even better is if you introduce it as a write-less, do-more tool that saves developers time and frustration. This incentivizes them to use it over a quick-and-dirty, error-prone, artisanal approach to solving the same problems.
-
FOSS4Spectroscopy: R vs Python | R-bloggers
If you aren’t familiar with it, the FOSS for Spectroscopy web site lists Free and Open Source Software for spectroscopic applications. The collection is of course never really complete, and your package suggestions are most welcome (how to contribute). My methods for finding packages are improving and at this point the major repositories have been searched reasonably well.
A few days ago I pushed a major update, and at this point Python packages outnumber R packages more than two to one. The update was made possible because I recently had time to figure out how to search the PyPi.org site automatically.
-
Commoditization of Large Language Models
The company behind GPT-3, OpenAI, hasn't released the non-public datasets or model, but they are all trivially recreated without much issue.
You only need about $12 million to train the model from scratch. And you don't even need to do that anymore.
There are plenty of open-sourced models to pick from. There's GPT-J from a set of independent researchers. Meta is open-sourcing OPT-175B. Stanford researchers are open-sourcing their model, Diffusion-LM.
-
Fixed vs. random effects for browsing data – a simulation | R-bloggers
When you work with trace data — data that emerge when people interact with technology — you will notice that such data often have properties that open up questions about statistical modelling. I currently work with browsing records, obtained at several times from the same users (i.e., a panel data set). A first typical characteristic of such data: Browsing behaviors are skewed. If you’re interested in people reading politically extreme web sites, you will find a few people doing it a lot, and most people doing little to none.
A second characteristic relates to the panel nature of the data. If you’re looking at people’s visits to online shops, they do not change their habits much over time — at least these within-person differences are not as great as the differences across people. Panel data lend itself to hierarchical modelling, for example with (1) fixed effects (FE) or (2) random-effects multilevel modelling (RE). For commonalities and differences between these two modelling approaches, see for example Gelman and Hill (2006).[1] The amount of within-person variation relative to between-person variation has important implications for these two approaches.
Below, I simulate the performance of FE vs. RE models, with these data characteristics in mind. I am mainly interested in the statistical power of each model, although my code can be easily adapted to examine, for example, bias and the RSME. The code builds on two sources: Conceptually, on the paper “Should I Use Fixed or Random Effects?” by Clark and Linzer (2015)[2]. In terms of code architecture, on the R package simhelpers, and its documentation.[3]
-
How many languages do we need to learn about responsible machine learning? useR! 2022 Conference | R-bloggers
It might seem that, we don’t have much choice, because the most popular languages in data science are R and Python or if you prefer Python and R. But today we are not talking about these languages!
During the useR!2022 conference, we can meet with English, Spanish and French. Because of the fact that only English is close to us we created our workshops and papers in this language. But! Is it possible to organize a workshop in another language?
Yes, it can be done and we did it! We submitted workshop in 5 languages in parallel (English, Spanish, Polish, Turkish and Vietnamese)!
-
How to Calculate Relative Frequencies in R? | R-bloggers
How to Calculate Relative Frequencies in R?, The relative frequencies/proportions of values in one or more columns of a data frame can frequently be calculated in R.
-
- Login or register to post comments
Printer-friendly version- 882 reads
PDF version
More in Tux Machines
- Highlights
- Front Page
- Latest Headlines
- Archive
- Recent comments
- All-Time Popular Stories
- Hot Topics
- New Members
today's howtos
| Open Hardware: XON/XOFF and Raspberry Pi Pico
|
Security Leftovers
| How to Apply Accent Colour in Ubuntu Desktop
A step-by-step tutorial on how to apply accent colour in Ubuntu desktop (GNOME) with tips for Kubuntu and others.
|



This section of TuxMachines will no longer have new stories in it (with some caveats, including this post). To see the latest stories go to

.svg_.png)
Content (where original) is available under CC-BY-SA, copyrighted by original author/s.


Recent comments
2 days 2 hours ago
2 days 6 hours ago
2 days 6 hours ago
3 days 13 hours ago
3 days 14 hours ago
3 days 15 hours ago
3 days 15 hours ago
3 days 16 hours ago
3 days 18 hours ago
3 days 20 hours ago