Sivaprasad MandapatiinDev GeniusMulti Cloud Analytics-Azure and GCPData silos constitute separate or segmented storage locations inside an organization where information exists and managed independently…Jun 25, 2023Jun 25, 2023
Sivaprasad MandapatiinDev GeniusGoogle Dataform vs DBT, Introduction to Dataform SQL workflowI always wonder DBT tool is ruling in the ELT world and why would not be a product with similar functionalities and use cases of DBT and…Jun 10, 20232Jun 10, 20232
Sivaprasad MandapatiinDev GeniusMigrating Spark Jobs to Google Cloud & File event sensor to Dynamically Create Spark Cluster…Google Cloud Analytics primary aspect is to separate Storage and Computation and pay for what you use . On-Premises Hadoop and Spark…Jul 1, 2022Jul 1, 2022
Sivaprasad MandapatiinThe StartupSpark Streaming & Real Time Analytics on AWSThis tutorial describes a real time analytics frame work using spark streaming and window functions on AWS real time streaming application…Feb 26, 2021Feb 26, 2021
Sivaprasad MandapatiSpark Joins Tuning Part-2(Shuffle Partitions,AQE)Continuation to my tuning spark join series. In this article ,I would like to demonstrate every spark data engineer’s nightmare…Feb 12, 20211Feb 12, 20211
Sivaprasad MandapatiinThe StartupSpark Joins Tuning Part-1(Sort-Merge vs Broadcast)Parallelization is Spark’s bread and butter. The back bone of Spark architecture is Data should be split into pieces(Partitions) and and…Feb 7, 20211Feb 7, 20211
Sivaprasad MandapatiSpark Parallelization Key FactorsSpark is an unified analytics engine for Bigdata Processing, with built-in modules for ETL,Streaming,SQL,Machine Learning and Graph…Jan 30, 2021Jan 30, 2021
Sivaprasad MandapatiReal Time Framework on AWS using Kinesis,Lambda and DynaoDBAs we are in bigdata era, Organizations continuously produce data, We do batch processing ETL’s and place it in decision making system (ex…Jan 21, 2021Jan 21, 2021
Sivaprasad MandapatiinAnalytics VidhyaGoogle BigQuery Machine Learning (BQML) on Covid-19 Data setWriting ML algorithms is tedious job which requires you to know lot many things including strong programming in Python,R,Ruby etc. But…Jan 18, 2021Jan 18, 2021
