Data Engineering
Data is omnipresent. Constantly evolving and churning, it is the life force of any business. The ability to harness it and unlock its full potential determines the trajectory of a business. We partner with our clients to solve complex problems of big data infrastructure and data engineering; with scalable analytics and data platforms that power digital experiences and provide actionable insights, we bolster data-centric business models.
Client Stories
Blogs
Choosing the right tool while building your Data Platform: DBT vs. Spark(By example)
Written by: Niranjan Ingale Table of contents What is a Data Platform, Spark & DBT: Let’s start with some definitions Data Platform: A data platform is a comprehensive infrastructure that facilitates the ingestion, storage, processing, analysis, and management of data...
Citizens of The Great Barrier Reef — A story of scale, complexity and collaboration for conservation
The Great Barrier Reef, Australia’s iconic ecological site, is not only important to the continent but to the planet as a whole. With over 3000 individual reefs stretching over 2000 kms and home...
Passwordless Auth between Cloud provider and Github
I discuss how I deployed my app to AWS Amplify through Github actions. However, you can deploy to any cloud provider and service of your choice by following this method.
Building secure systems with PII Data Protection Techniques — Part II
A practical guide to implementing sensitive data encryption using Postgres Pgcrypto In the previous blog, we discussed the importance of securing PII and sensitive data points captured by enterprises as part of their business...
Building secure systems with PII Data Protection Techniques — Part I
Today, storing a person’s KYC information, credit card/debit card information, or other similar sensitive details to charge for services availed on a platform is commonplace. This entails storing & dealing with Personally Identifiable...
DIVOC — A groundbreaking solution to orchestrate programs of scale worldwide
India is a huge country. Creating solutions for the world’s second largest population comes with its own set of challenges that need to be dealt with through capacity building, scale operations and interoperable...
Locking in Databases and Isolation Mechanisms
Concurrency control should become a familiar term if you are dealing with locks. In a large application accessed by thousands of users, concurrency is inevitable. Your application should be able to handle multiple requests...
Helm incubator kafka setup with SSL auth
Helm chart for incubator kafka is deprecated now. But still I feel this helm chart is very handy for doing PoC setup.ref : https://github.com/helm/charts/tree/master/incubator/kafka Incubator kafka helm chart supports SSL auth setup for brokers...
MLOps: Building a healthy data platform
Spoiler: MLOps is to ML Platforms what DevOps is to most tech products. If you think this means MLOps is automating your deployments, this article is for you.
Data storage patterns, versioning and partitions
When you have large volumes of data, storing it logically helps users discover information and makes understanding the information easier. In this post, we talk about some of the techniques we use to...
Our Clients
