I will design and implement scalable data engineering pipelines
Data Scientist, Big Data and AI Engineer, end to end solutions
About this Gig
Why work with me?
Do you need reliable, low-latency data pipelines and a clean, queryable data platform? I help people and small teams turn messy streams and files into production-ready data that powers dashboards, ML models, and business reports. I hold an MSc in Data Science & Intelligent Systems and a background in engineering I design pipelines that reduce manual work, save cloud costs, and deliver fresh, trustworthy data. Even if you need to design architectures in batch or in real time, I am here to help you realize your ideas.
Note : If you want to see my detailed portfolio, message me to send you the link.
Services I offer ?
- End-to-end ETL/ELT pipelines (batch & streaming)
- Real-time streaming architecture (Kafka, Spark Structured Streaming)
- Data lake / Lakehouse design (bronze/silver/gold medallion layers)
- Data integration: APIs, databases, S3/GCS, message brokers
- Automated data quality checks, monitoring, and alerting
- Data partitioning, compaction, and cost/latency optimization
Tools & Technologies ?
Python, Apache Spark, Kafka, Delta Lake, Databricks, Airflow, AWS (S3), GCP, PostgreSQL, MongoDB, Parquet/Avro, Docker, CI/CD basics
My Portfolio
FAQ
What do I need to provide?
Please share your raw data samples (CSV, JSON, database access, etc.), a description of your desired outcomes, and any tech preferences. The more details you give about your data and objectives, the better the solution.
Which technologies will you use?
I typically use Apache Spark (PySpark), Kafka for streaming, Delta Lake/S3 or HDFS for storage, and SQL/Python for transformations. Let me know if you have specific preferences (e.g., AWS, GCP, or Azure tools)
What is turnaround time?
Delivery depends on project scope. The packages above give estimated timelines, but we’ll agree on exact deadlines once I review your requirements.
What if I’m not satisfied?
Client satisfaction is my priority. Each package includes revisions (as listed). If something isn’t right, I will work with you to make it right.
