PinnedBalachandar PaulrajEssential Considerations for Data Engineers When Selecting a NoSQL DatabaseIn the realm of modern data engineering, the choices abound, and the stakes are high. Data engineers are the architects of the digital age…3 min read·Sep 4, 2023--18--18
PinnedBalachandar Paulraj2022 : Modern Data StackYou might have seen multiple posts around this subject as time keeps evolving and bringing changes into tech stack, however this includes…5 min read·May 3, 2022--15--15
PinnedBalachandar PaulrajDuckDB: Primer on the subject and fascinating highlightsThroughout our data engineering journey, we’ve come across a myriad of database management systems (DBMS). But what sets DuckDB apart from…4 min read·Jul 2, 2023--6--6
Balachandar PaulrajHarnessing the Potential of Databricks Liquid Clustering: A Dynamic Data Layout Scaling with Growth…Databricks made waves at the previous Data + AI Summit by introducing Liquid Clustering alongside Delta Universal Format (UniForm) and…3 min read·Jan 3, 2024--2--2
Balachandar PaulrajAirbyte Spotlight: The Open-Source Solution for Data Integration — Features, Benefits, and…Data integration is foundational to the success of modern businesses, fostering better decision-making, operational efficiency, and…3 min read·Dec 26, 2023--2--2
Balachandar PaulrajFast-Track PySpark UDF execution with Apache ArrowDevelopers often create custom UDFs (user-defined-functions) in their Spark code to handle specific transformations. This allows users to…4 min read·Nov 19, 2023----
Balachandar PaulrajRAY: Distributed computing framework for ML & AIThe evolving domain of artificial intelligence and machine learning is witnessing an unprecedented demand for tools that are efficient…4 min read·Nov 6, 2023----
Balachandar PaulrajKey Database Compaction Strategies Used In Distributed SystemIn the realm of distributed database systems, the adoption of compaction strategies plays a pivotal role in the effective management of…3 min read·Sep 4, 2023----
Balachandar PaulrajApache Paimon: A fresh face joins the frayRecently, few people might have heard about Apache Paimon. Undergoing incubation at the Apache Software Foundation (ASF), Apache Paimon is…3 min read·Apr 3, 2023--1--1
Balachandar PaulrajCan Snowpark supersede Databricks and AWS EMR?An Overview on Snowpark compared with Spark3 min read·Jun 28, 2022--6--6