Projects

Speech data collection using kafka cluster

• 
• April 2022
• The proposed data pipeline was built on Apache Kafka, an open-source distributed event streaming platform. By combining messaging, storage, and stream processing, the data pipeline allow collection, storage and analysis of real-time audio datasets. The data pipeline consists of the following key components: Data producers, Data consumers, Apache Kafka cluster, Amazon S3 bucket Connectors and Apache Spark Stream preprocessors.
• Used techstacks: AWS S3 Bucket, AWS athena, spark, kafka, ELT pipeline, Airflow, docker

GitHub Link Article

---

Data warehouse for pNEUMA | open-traffic datasets

• 
• June 2022
• Built a scalable data warehouse tech-stack with ELT pipeline composed of MYSQL Database, dbt transformation, Airflow orchestration and Redash visualization.
• Used techstacks: dbt, ELT pipeline, Airflow, Redash.

GitHub Link Article

---

USGS LIDAR Data warehouse

• 
• Sep 2022
• We are very interested in how water flows through a maize farm field. This knowledge will help us improve our research on new agricultural products being tested on farms. How much maize a field produces is very spatially variable. Even if the same farming practices, seeds and fertilizer are applied exactly the same by machinery over a field, there can be a very large harvest at one corner and a low harvest at another corner. We would like to be able to better understand which parts of the farm are likely to produce more or less maize, so that if we try a new fertilizer on part of this farm, we have more confidence that any differences in the maize harvest are due mostly to the new fertilizer changes, and not just random effects due to other environmental factors. We are tasked to produce an easy to use, reliable and well designed python module that domain experts and data scientists can use to fetch, visualise, and transform publicly available satellite and LIDAR data. In particular, our code should interface with USGS 3DEP and fetch data using their API.
• Used techstacks: LASzip, PDAL, CloudCompare, USGS 3DEP API and GIS software packages such as QGIS or ArcGIS.

GitHub Link

---

Speech Recognition

• 
• March 2022
• Built end-to-end speech recognition model that transcribes Amharic speech to text.
• Used ML frameworks like TensorFlow, PyTorch and NLP
• Used techniques like Fourier transforms, short-time Fourier transforms and Mel-frequency cepstral coefficients (MFCCs).

GitHub Link Article

---

• More Projects More Articles