Technical Articles

Now, why do this? What’s wrong with Medium and Substack and Blogger and Wordpress and all the tens of websites out there? Nothing. You could totally just create a blog there and publish. That’s why Content Management Systems exist - so you could focus on the content and not on the management . But if you’re like me and want some more control over how you publish your blogs, customize them etc. , you will find this and the other upcoming articles in this category interesting. ...

I had the opportunity to work with some of the functions in the itertools package in Python over the month and I thought before the month ends in a few minutes, I’ll write about a few of them - a) The Count function - This is useful when you want to iterate from a starting number , say ‘n’ up to an indeterminate number that you do not know at the point of starting the iterator. The example below assumes the end limit to be 10 numbers from the point of starting the counter. The second argument to the ‘count’ function is a “step” parameter. ...

This is a rather simplistic topic but one of my good friends has been trying to setup Streamlit and has been unsuccessful at it. So he asked if I could help him out. If this helps him atleast, I will have done my job. Hence this topic for today’s mini essay. Going forward, pretty much every local package I install will use uv. (How to install uv? https://everythingpython.substack.com/p/uv-python-packager-written-in-rust) Operating system under use - Windows (but this will work on Ubuntu as well) So let’s create a folder for our Streamlit apps - ...

Chapter 2 - Data Models and Query Languages Thoughts [02-10-2024]

Chapter 1 - Reliable, Scalable and Maintainable Applications In [[Reading DDIA - Part 2]] , I read about the second way in which design decisions are typically driven - Scalability. This section is about what the book says is the third factor - Maintainability. What is a maintainable system? My thoughts prior to reading the section - Something they said in the San Diego Machine Learning struck me as my reaction as well - Maintainability is not one of the “concerns” I would have listed for a data intensive system. But now that it is, let’s think about it - What does it mean to maintain a system? My interpretation is - how would the system be set up for functioning smoothly over time? To that end, ...

Chapter 1 - Reliable, Scalable and Maintainable Applications In [[Reading DDIA - Part 1]] , I read about Reliability. The next section is Scalability . First let me talk about the assumptions I made about the term Scalability, what I understand from it prior to reading Martin Kleppman’s thoughts on the topic - What is a scalable system? My thoughts prior to reading the section - Scale refers to how does a system grow. Growth can be needed for multiple reasons - ...

(Here’s the preface in case you missed it - [[Reading DDIA - Preface]]) Chapter 1 - Reliable, Scalable and Maintainable Applications This chapter’s goal is to effectively give us a high level overview of what this book is fundamentally about - Thinking about data systems along the axes of : Reliability Scalability Maintainability What is a reliable system? My thoughts before reading the section : A system that is available to return results/outputs that are expected of it when it is employed. If I’m accessing an API, if the API is available 99 out of 100 times, then its reliability is 99% If a server is available for 23 hrs 45 minutes per day then its availability is 23.75/24 = 98.9% But is availability == reliability? Let’s see. A reliable human is someone who does something that he says he will do or is expected to do every time. Similarly a reliable system is one that returns the expected results every time that they are used. An ATM is reliable if I can withdraw money from it in the denominations that I want, when I want it. For example, the ATM in Puttenahalli , JP Nagar is perpetually unavailable. Everytime I go there, the system is under maintenance. It is 0% reliable. Let’s read the chapter. ...

Over the next few weeks, I want to read a book that has been long-pending - “Designing Data Intensive Applications” by Martin Kleppman. I’ve been building data applications for a while now but most, if not all the learnings, have been through trial and error after error after error. I’ve learnt a decent bit from observing more senior engineers, architects, design documents and building architectures little by little on my own, but not yet I think, in a structured, deliberate manner. I’m hoping to learn that in this book along with the other books I will likely have to refer along the way. ...

What is a package in Python? Hold up, isn’t this one of those terms we use synonymously with other terms like library, module etc? Well no. Let’s learn what the differences are - Module A module can be any file with a .py extension. For example - hero.py def hero(): return f"Anyone" print(hero()) This module when executed using python hero.py returns the string “Anyone”. Because a hero can be Anyone. Package A package is typically a collection or a group of related modules. The indication of a package is via an empty init.py file alongside the rest of the package’s modules. ...

Building a Classifier Fetching stories from Hacker News Extracting key information from articles Using AI to determine article topics Code - {% gist 5181b0e3c9957617598d4b446609ba01 hn_scraper_llm_topics.py %} {% gist d6a97bbd3534c532d148ea8a1efc851b strings.py %} {% gist d2fc27b41a17497466621b7e25c924fa data_models.py %} {% gist 3bb526888e592de66c82eba391dd08ea keys.py %}