Programming Leftovers
-
Idiomdrottning ☛ NIH with a ribbon bookmark
Sometimes you get a contribution to your git project and you love their idea but you want to implement things your own way because you’ve got NIH.
Here’s how to do it politely so they still get credit in the git log.
-
Lawrence Tratt ☛ Four Kinds of Optimisation
You will not be surprised to learn that I think both forms of optimism misplaced. Partly this is because, as hardware and software have become more sophisticated, it has become harder to understand their effects on performance. But, perhaps more fundamentally, we tend to overestimate how much we know about the software we're working on. We overemphasise the parts of the system we've personally worked on, particularly those we've most recently worked on. We downplay other parts of the system, including the impact of dependencies (e.g. libraries).
The solution to the first of these observations is fairly widely known — one should rigorously profile a program before assuming one knows where it is spending the majority of its time. I deliberately say "rigorously profile" because people often confuse "I have profiled a program once" with "I have built up a good model of a program's performance in a variety of situations". Sometimes, a quick profiling job is adequate, but it can also mislead. Often it is necessary to profile a program with different inputs, sometimes on different machines or network configurations, and to use a variety of sampling and non-sampling approaches.
-
Rlang ☛ How to Predict a Single Value Using a Regression Model in R
Regression models are a powerful tool for predicting future values based on historical data. They are used in a wide range of industries, including finance, healthcare, and marketing.
-
Rlang ☛ Renewable energy in Europe
Using data from Eurostat we will try to show the spatio-temporal properties of this dataset by placing plots of the renewable energy share change on the map, for each country.
-
Python
-
Trail Of Bits ☛ Our audit of PyPI
This audit was sponsored by the Open Tech Fund as part of their larger mission to secure critical pieces of internet infrastructure. You can read the full report in our Publications repository.
Late this summer, we performed an audit of Warehouse and cabotage, the codebases that power and deploy PyPI, respectively. Our review uncovered a number of findings that, while not critical, could compromise the integrity and availability of both. These findings reflect a broader trend in large systems: security issues largely correspond to places where services interact, particularly where those services have insufficiently specified or weak contracts.
-
Python Package Index ☛ PyPI has completed its first security audit
We are proud to announce that PyPI has completed its first ever external security audit. This work was funded in partnership with the Open Technology Fund (OTF), a previous supporter of security-related improvements to PyPI.
The Open Technology Fund selected Trail of Bits, an industry-leading cybersecurity firm with significant open-source and Python experience, to perform the audit. Trail of Bits spent a total of 10 engineer-weeks of effort identifying issues, presenting those findings to the PyPI team, and assisting us as we remediated the findings.
-
Python Package Index ☛ Security Audit Remediation: Warehouse
This post is a deeper dive into the remediation of the security audit findings for the Warehouse - the main codebase for PyPI.org.
The audit report can be found here. I highly recommend reading that for the fullest context first.
-
Python Package Index ☛ Security Audit Remediation: cabotage
cabotage was initially developed in 2018 as part of the Mozilla Open Source Support Award that enabled the Python Software Foundation (PSF) to fund a team of contracted developers and a project manager to complete the development and deployment of warehouse and sunset the original PyPI codebase.
A primary goal of cabotage is to reduce the PSF Infrastructure's dependence on a specific provider for running PyPI, while providing self-service of configuration for project administrators and fully automated deployments. It is in-effect a "Platform as a Service" that deploys applications into bog-standard Kubernetes clusters, no YAML required.
-
Seth Michael Larson ☛ Querying every file in every release on the Python Package Index
What is interesting about this graphic is it needs access to files within Python distributions like wheels and source distributions on PyPI. This is something that's difficult to access without actually downloading the artifact. So how can I query this information for every package since 2005?
-
-
Shell/Bash/Zsh/Ksh
-
Vidarholen ☛ What exactly was the point of [ “x$var” = “xval” ]?
For any POSIX compliant shell, the value of the x-hack is exactly zero: this comparison works without the x 100% of the time. But why was it a thing?
-