Programming

Atomic Operations Explained: From Hardware to High-Level Code (using Go)

Modern applications frequently rely on atomic operations, with atomic counters being one of the most common use cases. For example, when multiple threads execute in parallel, increment and decrement operations must be atomic to prevent race conditions. For this discussion, I’ll be using Go as the reference language, but the core concepts apply to other programming languages as well.This section provides a refresher on managing race conditions in Go. If you’re already familiar with this topic, feel free to skip...

Read

Locking in Databases and Isolation Mechanisms

should become a familiar term if you are dealing with locks. In a large application accessed by thousands of users, concurrency is inevitable. Your application should be able to handle multiple requests simultaneously. When you execute operations concurrently, the results can be conflicting. For e.g. if you are reading a row while someone else is writing to it simultaneously, then you are bound to get inconsistent data. If we execute these transactions sequentially, then we don’t need concurrency control. But...

Read

12 Factor Spark Applications

Spark is a distributed data processing engine that is widely used in batch processing and stream processing platforms. Building such platforms comes with a fair share of challenges beyond those required for continuous delivery in the microservices world such as data drift, bad data, and data security. Similar to the Twelve-Factor app that is an outstanding methodology/set of principles for building web apps, we at Sahaj have realized over time that a set of concepts/patterns can be applied to building data processing...

Read

Kafka Basics and Core concepts

Source: https://www.confluent.io/ Let’s start by answering the question “What is Kafka?”. Kafka is a Distributed Streaming Platform or a Distributed Commit Log Let’s try to understand those jargons. Kafka works as a cluster of one or more nodes that can live in different Datacenters, we can distribute data/ load across different nodes in the Kafka Cluster, and it is inherently scalable, available, and fault-tolerant. Kafka stores data as a stream of continuous records which can be processed in different methods. This...

Read