Danial Shabbir

Danial Shabbir Data Engineer / Backend Developer (- https://www.linkedin.com/in/danial-shabbir/ Islamabad, Pakistan https://stackoverflow.com/users/- https://github.com/danielox https://www.researchgate.net/profile/Danial-Sh - https://www.hackerrank.com/danialmaluk7 PROFILE I’m an enthusiastic and diligent software technology professional. Having worked on a variety of projects around various domains equips me with the ability to adapt, enhance and optimise towards quality solution delivery. I’m an equally exceptional performer under pressure with core focus at adhering timelines. PROFESSIONAL EXPERIENCE Solutions Engineer, Motive (Formerly KeepTruckin) Connecting with customers to understand their reporting needs Designing and developing custom analytical reports using Snowflake and SQL Integrating Motive's data warehouse with customer warehouses Using Python, SQL, Snowflake, Airflow, and Bash scripts to query, analyze, and visualize data Working with stakeholders to communicate insights and recommendations Developed a custom report that helped a customer to reduce their fuel costs by 10% (450k USD). RESEARCH WORK Twitter Trends: A Ranking Algorithm Analysis on Real Time Data, Expert Systems with Applications Empirical Analysis of Machine Learning Algorithms for Multiclass Prediction, Hindawi Journal SKILLS • • • Big Data - AI/ML Apache Spark, Apache Kafka, Apache NiFi, ELK Stack • • • Data Engineer / Backend Developer, Nowasys Limited Oct 2022 – present | United Kingdom Managed, Processed & Delivered terabytes of big data, intensive workloads via AWS EMRs (PySpark) Efficient use & processing of S3 Data Lakes via Distributed Computing Big Data technologies Refactored, Optimised previous big data processing pipelines to reduce 35% processing time and boosting marketing team's efficiency. Managed AWS EMR workflows with Apache Airflow by scheduling, monitoring & scaling of DAGs Reduced cost of AWS EMRs by following best practices of big data to process large data efficiently on commodity servers • Amazon Web Services IAM, EMR, RDS, S3, Redshift, Lambda, Glue, EC2, Cloud9 Programming Languages Python, SQL, PHP, JavaScript ES6 Reporting & Analytics Power BI, Kibana, Pandas, Seaborn, Bokeh, Plotly Data Science H2O AutoML, NLTK, Spacy, Hugging Face, BiTerm, Scikit Learn • • • Backend Frameworks / Server Tools Flask, Gunicorn, Django, Laravel Soft Skills Adaptability, Collaboration, Communication Skills, Creativity, Critical Thinking, Presentation Skills, Problem Solving Skils • Machine Learning Engineer - Big Data, Rapidev DMCC Oct 2020 – Apr 2022 | Islamabad, Pakistan Developed an end-to-end real-time ETL Pipeline to move 20 million rows of data daily from HDFS, and S3 to data warehouses Created, updated, and maintained SQL and Python-based custom reports for internal end-users and external clients. Created advanced ml models, visualizations, and measures to build Machine Learning dashboards. Implemented near real time log analytics pipeline using elastic search , Logstash and kibana. • • • • Backend Developer, Premium Meat Jan 2019 – Oct 2019 | Auckland, New Zealand To process online payments with the specified suburbs Designed analytics dashboard presenting the health and significant aggregations to business Coordinated with the stakeholders on project progress Deployed and maintained digital ocean VPS for website optimization • • • • PROJECTS OSINT, Open Source Intelligence Tool Performed ETL with Spark and Kafka. Implemented machine learning models on the pipeline. Implemented microservices for as per requirements ASR, Spech to Text Opinion Mining A end-to-end ETL pipeline with machine learning for video to audio conversion and transcription PlayRevz, Analyze customer reviews and feedback A web-based application that analyzes, monitors, google play store app reviews,ratings and classifies each review for different categories. Premium Meat, Fully Personalized e-commerce store A complete meat-seller e-commerce with a fully customizable admin panel, dashboard, e-payments and reporting with data warehousing capabilities