As data volumes grow, switch from batch processing to stream analytics

Jun 21, 2021

--

This tip is short enough to be a Twitter thread. I’m preserving it here:

Batch Processing

Stream Analytics

Lak Lakshmanan

Written by Lak Lakshmanan

articles are personal observations and not investment advice.

Recommended from Medium

Dataflow Vs Dataproc

Akash Waitage
in
Google Cloud - Community

Dataflow Vs Dataproc

Which GCP service should I choose?

8 min readApr 16

--

4

How I Built a Data Lakehouse With Delta Lake Architecture

Nicholas Leong

How I Built a Data Lakehouse With Delta Lake Architecture

Data Engineer Explains the Data Lakehouse Architecture

10 min readSep 18

--

4

Lists

New_Reading_List

174 stories124 saves

Natural Language Processing

652 stories257 saves

Staff Picks

457 stories305 saves

Unit Testing in Data Engineering: A Practical Guide

Sam Zamany

Unit Testing in Data Engineering: A Practical Guide

In the realm of data engineering, one often overlooked practice is unit testing. Many might think that unit tests are merely a software…

10 min readJul 12

--

4

It’s Time to Say GoodBye to pd.read_csv() and pd.to_csv()

Avi Chawla
in
Towards Data Science

It’s Time to Say GoodBye to pd.read_csv() and pd.to_csv()

Discussing another major caveat of Pandas

4 min readMay 26, 2022

--

52

The future of the data engineer — Part I

Analytics at Meta

The future of the data engineer — Part I

Introduction

10 min readApr 3

--

15

The ChatGPT Hype Is Over — Now Watch How Google Will Kill ChatGPT.

AL Anany

The ChatGPT Hype Is Over — Now Watch How Google Will Kill ChatGPT.

It never happens instantly. The business game is longer than you know.

6 min readSep 1

--

315

See more recommendations

Help
Status
Writers
Blog
Careers
Privacy
Terms
About
Text to speech
Teams