Hi Memphis,

Thanks for your contribution and I completely agree that Apache Kafka, Apache Spark and Hadoop are the best solution to stream stock price data in almost real time (micro-batches).

Having said that, this article is meant to focus less on batch vs stream processing and more on creating an introductory tutorial on Airflow as a scheduling tool (particularly how to set DAGs properly) :D

By the way, have you ever used any streaming tool for a project? If yes, how difficult was to implement a working pipeline the very first time?

Thank you :)

Antonello