Smart City End to End Realtime Data Engineering Project | Get Hired as an AWS Data Engineer

CodeWithYu147:072024Ciência e Tecnologia

In this video, you will be building a Smart City End to End Realtime data streaming pipeline covering each phase from data ingestion to processing and finally storage. We'll utilize tools like IOT devices, Apache Zookeeper, Apache Kafka, Apache Spark, Docker, Python, AWS Cloud, AWS Glue, AWS Athena, AWS IAM, AWS Redshift and finally PowerBI to visualize data on Redshift. MORE DATA ENGINEERING VIDEOS AVAILABLE on datamasterylab.com Like this video? - Buy me a coffee: https://www.buymeacoffee.com/yusuf.ganiyu - Become a member: https://www.youtube.com/@CodeWithYu/join Timestamps: 0:00 Introduction 1:29 System Architecture 7:22 Project Setup 9:00 Docker containers setup and coding 26:17 IOT services producer 38:19 Vehicle information Generator 48:10 GPS Information Generator 50:13 Traffic information Generator 53:13 Weather information Generator 58:35 Emergency Incident Generator 1:03:39 Producing IOT Data to Kafka 1:14:43 AWS S3 setup with policies 1:16:38 AWS IAM Roles and Credential

Chegaste aqui por um link? Lança tu também.

🎲 Lançar os Dados

Reproduzido pelo leitor oficial incorporado. Ver no YouTube.