Programming Leftovers

posted by Roy Schestowitz on Jan 09, 2023

• How to implement Celery with Django – NextGenTips

  Perhaps you might be wondering like me what the heck is Celery? Let’s begin by first understanding what Celery is. Celery is an open-source asynchronous task queue that is based on distributed message passing. And what is a task queue? A task queue’s input is a unit of work called a task. They are used as a mechanism to distribute work across threads or machines.

• How to Write to a CSV File in Python

  Knowing how to write to a CSV file in Python is essential for handling excel-related files and automating repetitive tasks. This tutorial shows you how to write to CSV file with (A) local data and (B) data scraped from a third-party website.

  Let’s get started.

• Performance comparison of converting list to data.frame with R language

  When you are working with large datasets performance comes to everyone’s mind. Especially when converting datasets from one data type to another. And choosing the right method can make a huge difference.

  So in this case, I will be creating a dummy list, and I will convert the values in the list into data.frame.

  Simple function to create a large list (approx. 46MB with 250.000 elements and each element consists of 10 measurements).

• All about sanitizer interceptors

  Many sanitizers want to know every function in the program. User functions are instrumented and therefore known by the sanitizer runtime. For library functions, some (e.g. mmap, munmap, memory allocation/deallocation functions, longjmp, vfork) need special treatment. Sanitizers leverage symbol interposition to redirect such function calls to its own implementation: interceptors. Other library functions can be treated as normal user code. Either instrumenting the function or providing an interceptor is fine. In some cases instrumenting is infeasible (e.g. assembly source files, When can glibc be built with Clang?) or is inefficient, and interceptors may be the practical choice.

  This article talks about how interceptors work and the requirements of sanitizer interceptors.

• Open source codebases are the bedrock of today’s enterprise systems

  My friend and colleague, Vijira published a post summarising his technical keynote on “Open Hybrid Cloud: The Foundation for a Robust Digital Transformation” as part of Red Hat Summit Connect last year.

  There are key takeaways from the post and you should read that first before continuing here.

• Use this open source API gateway to scale your API | Opensource.com

  An API gateway is a single point of entry for incoming calls to an application programming interface (API). The gateway aggregates the services being requested and then returns the appropriate response. To make your API gateway effective, it's vital for you to design a reliable, efficient, and simple API. This is an architectural puzzle, but it's one you can solve as long as you understand the most important components.

• Optimizing our longest nightly job — a story of sessions, complexity, and toilets

  One day, we noticed that one of these commands was in fact still running after more than 20h, and that the next run would indeed be run before the previous one has ended. This was a big problem — not because the data could have been corrupted (it could not thanks to our idempotency requirement), but because it indicated clearly that we had a performance bottleneck, and probably some scalability issues down the line.

  We took on a mission to find out how to improve this run time; we learned a lot along the way. Come with us!