Sivaprasad Mandapati – Medium

Sivaprasad Mandapati

Sivaprasad Mandapati
in
Dev Genius

Multi Cloud Analytics-Azure and GCP

Data silos constitute separate or segmented storage locations inside an organization where information exists and managed independently…

5 min readJun 25, 2023

--

Multi Cloud Analytics-Azure and GCP

--

Sivaprasad Mandapati
in
Dev Genius

Google Dataform vs DBT, Introduction to Dataform SQL workflow

I always wonder DBT tool is ruling in the ELT world and why would not be a product with similar functionalities and use cases of DBT and…

6 min readJun 10, 2023

--

2

Google Dataform vs DBT, Introduction to Dataform SQL workflow

--

2

Sivaprasad Mandapati
in
Dev Genius

Migrating Spark Jobs to Google Cloud & File event sensor to Dynamically Create Spark Cluster…

Google Cloud Analytics primary aspect is to separate Storage and Computation and pay for what you use . On-Premises Hadoop and Spark…

5 min readJul 1, 2022

--

Migrating Spark Jobs to Google Cloud & File event sensor to Dynamically Create Spark Cluster…

--

Sivaprasad Mandapati
in
The Startup

Spark Streaming & Real Time Analytics on AWS

This tutorial describes a real time analytics frame work using spark streaming and window functions on AWS real time streaming application…

4 min readFeb 26, 2021

--

Spark Streaming & Real Time Analytics on AWS

--

Sivaprasad Mandapati

Spark Joins Tuning Part-2(Shuffle Partitions,AQE)

Continuation to my tuning spark join series. In this article ,I would like to demonstrate every spark data engineer’s nightmare…

4 min readFeb 12, 2021

--

Spark Joins Tuning Part-2(Shuffle Partitions,AQE)

--

Sivaprasad Mandapati
in
The Startup

Spark Joins Tuning Part-1(Sort-Merge vs Broadcast)

Parallelization is Spark’s bread and butter. The back bone of Spark architecture is Data should be split into pieces(Partitions) and and…

4 min readFeb 7, 2021

--

1

Spark Joins Tuning Part-1(Sort-Merge vs Broadcast)

--

1

Sivaprasad Mandapati

Spark Parallelization Key Factors

Spark is an unified analytics engine for Bigdata Processing, with built-in modules for ETL,Streaming,SQL,Machine Learning and Graph…

3 min readJan 30, 2021

--

Spark Parallelization Key Factors

--

Sivaprasad Mandapati

Real Time Framework on AWS using Kinesis,Lambda and DynaoDB

As we are in bigdata era, Organizations continuously produce data, We do batch processing ETL’s and place it in decision making system (ex…

4 min readJan 21, 2021

--

Real Time Framework on AWS using Kinesis,Lambda and DynaoDB

--

Sivaprasad Mandapati
in
Analytics Vidhya

Google BigQuery Machine Learning (BQML) on Covid-19 Data set

Writing ML algorithms is tedious job which requires you to know lot many things including strong programming in Python,R,Ruby etc. But…

4 min readJan 18, 2021

--

Google BigQuery Machine Learning (BQML) on Covid-19 Data set

--

Sivaprasad Mandapati

Sivaprasad Mandapati

Azure ||Google Cloud Certified||AWS|| Big Data,Spark,ETL Frameworks,Informatica|| Database migration Specialist||Data Architect||Google Cloud Authorized Trainer

Help
Status
About
Careers
Blog
Privacy
Terms
Text to speech
Teams