Eman Ali Mughal

Eman, Mughal Senior Software / Big Data Architect Having a strong (4 years) educational background of computer sciences with my professional experience (20 years overall, 5 years Big Data + 15 years Software Development) in different reputable organization. Passionate, Enthusiastic, Ambitious, Committed and consistent in every field I have chosen so far related to software. Achieving the excellence is my key identity and quality. Summary of Work Experience House # 1027 / 1, Street # 3-B Cantt View Colony, Range Road, Rawalpindi, Pakistan M: - Email:-https://www.linkedin.com/in/eman-mughal-6bbb0722/ Data Engineering Skills Senior Software / Big Data Architect Apache Airflow NorthBay Solutions, Lahore, Pakistan Duration (04/2022 –Now) Spark Development Tasks/Achievements Hive Development ● Contributed in design of monitoring and trigger system for many data pipelines, developed in multiple technologies. Like, Hadoop, Spark, Airflow, Snowflake, AWS, EMR. ● Architected ELT pipeline in snowflake using javascript stored procedures. Data size was 1.4 Billion records and 1 TB of data. Ingestion took place in 1 hour 20 minutes on XL virtual warehouse. ● Worked on projects where i have used, API integrations and called AWS Lambda from snowflake. Setup architecture and security framework for Snowflake and on boarded various datasets for quick sharing with customers. ● Worked on snowpipe with snowflake serverless compute. Deployed data profiling framework using spark and Amazon Deequ. ● worked extensively on AWS, SNS, SQS, S3, Lambda using boto3 python library. ● Mentoring couple of new resources, prepare them for client interaction. Tools Used: Airflow, S3, AWS, Hive, Nifi, Kafka, Spark, Scala, Databricks, Snowflake, Bash Scripting, Elasticsearch, Kibana, Logstash Spark SQL Databricks ETL Elastic Search Business Intelligence Amazon S3 Apache NiFi Scala Spark Streaming Spark Structured Streaming Snowflake AWS GCP Linux Shell Scripting Python Mongo DB Lead Big Data Engineer S&P Global, Islamabad, Pakistan Duration (04/2019 –03/2022) Tasks/Achievements ● Design an architecture of Big Data Lake to ingest data from 200 Different sources having a volume of 500 + TB per annum using Spark / DeltaLake / Nifi / Kafka / AWS. ● Designed and delivered ingestion pipeline and analytical workloads for various use cases using Snowflake / Databrikcs / Delta Lake / Airflow / NiFi / AWS / S3. ● Setup architecture and security framework for Snowflake and onboarded various datasets for quick sharing with customers. IntelliJ Amazon Lambda Hadoop / HDFS / MapR OLTP OLAP Software Developer / Architect Skills ● Established monitoring framework for ingestion and analytics using Elasticsearch / Logstash and Kibana. Software Architecture ● Deployed data profiling framework using spark and Amazon Deequ. Architectural Styles Windows Services Tools Used: Airflow, S3, AWS, Hive, Nifi, Kafka, Spark, Scala, Databricks, SOLID Snowflake, Bash Scripting, Elasticsearch, Kibana, Logstash Senior Web Architect Asterisk Solutions Pvt Ltd, Islamabad, PK Duration (12/2017 – 03/2019) Tasks/Achievements ● Integrated Amazon Rest APIs with Shopping Cart Application. ● Integrated EBay APIs with shopping cart application ASP.NET / MVC / Web API WCF / WPF / Prism Angular / React Knockout / JQuery / JS Node Js C# / PHP / VB.NET ● Created ETL Pipeline for transferring Products / Invoices / Customers data into Amazon S3 in parquet format for reporting purpose. OOP / OOP Design ● Conceived, Developed & Deployed Big Data Analytics Engine for Shopping Cart Application. DI Containers / IoC EF / NHibernate / ORM Azure Web Services CTO - Entrepreneur Azure Service Bus Vibrant Illusions Pvt Ltd, Islamabad, PK Duration (10/2009 – 11/2017) Twitter / Facebook API ● Worked in the capacity of Founder / CTO for this company. Ecommerce Development ● Lead / Instructed / Involved in creating Architecture for many largescale enterprise level Web / Desktop / Mobile Applications. Azure Queue / Table /Blobs ● Migrated huge RDBMS system from SQL Server into Data Warehouse for one of our clients. Agile Development ● Restructured Data Model or and transferred into Dimension modeling in snowflake. TDD / Mock Testing ● Created Elasticsearch / Airflow / Apache Spark Cluster in GCP Cloud. Html / CSS / CSS3 SCRUM Jira VSTS Sr – Software Developer / Project Lead DevOps While Wings Consulting LLC Duration (01/2007 – 09/2009) SQL Server ● A unique experience to work on an architecture that was simply outstanding. Custom ORM, Custom templates, AJAX. A software that sales and purchases stock images and photos. ● Worked in money exchange software. Sr – Software Developer Xitos Pvt Ltd Duration (06/2006 – 12/2006) ● Worked on a project that deals in shopping cart applications. Software Developer Ovex Technologies, Islamabad, PK Duration (06/2005 – 05/2006) ● Created Middleware software for Amazon Catalog Management. Our customer was Amazon’s Seller, We have developed an automated Software consist of SQL Server Jobs scheduled to generate Excel templates to update catalog and inventory information on Amazon Marketplace. ● Worked in Custom record management software. Software Developer Aspire Technologies, Islamabad, PK Duration (01/2005 – 05/2005) ● Developed Custom Framework in php and integrated with Smarty Stored Procedures Database Design CICD Project Management Leadership Skills Communication Teamwork Leadership Mentoring Flexibility Creativity Negotiation Presentation Determination Perseverance templates to automatically create UI forms with CRUD operations code in the back end. ● Worked on some custom modules in client-oriented software. Software Developer SoonSoft, Islamabad, PK Duration (02/2003 – 08/2004) ● Created Industry Automation desktop application for Football and Pharmaceutical industries. This application was developed in .NET technologies 1.1. This software covers many modules, like Inventory system, Manufacturing system, Billing, Invoice generation etc. Software Developer Elixir Technologies, Islamabad, PK Duration (04/2002 – 07/2003) ● Worked on Telemedicine Software for Elixir Technologies. ● Developed Kids portal in php. Project Engagement Details Achievements ETL Development Duration (04/2019 – 02/2020) S&P Global Big Data Pipeline Spark | Hive | Nifi | Airflow | Scala |Bash Scripting ● Created Big Data ETL Pipeline for different Data sets using Apache NiFi for file acquisition, S3 for Raw Storage, Spark for processing, transforming, storage. ● Created Airflow DAGs for multiple datasets. ● Enhanced the code base to accommodate multiple transformation logics. Enhanced ETL Pipeline using Databricks Duration (03/2020 – 12/2020) Creator of Udemy Courses  ShoppingCart in ASP.NET MVC, Web API, Knockout, Architecture | Udemy  https://www.udemy.com/managed-extensibility-framework-meffrom-novice-to-guru/ CEO Recognition https://www.credential.net/5f50db25-e4c0-451e-96230d8d8c333829 S&P Global Big Data Pipeline Islamabad, PK Databricks | Spark | NiFi | Snowflake ● Changed the architecture of previous ETL pipeline to Databricks ● Implemented Notebook driven development ● Introduced Databricks standards of Raw / Silver / Golden zones ● Finally delivered data into Snowflake / Feeds / APIs Essential Tech Award Implemented Alerting / Monitoring Framework Duration (01/2021 – Now) https://badges.spglobal.com/e024d8e2-ba9e-4536-b24f7b2a93be2549 S&P Global Islamabad, PK ELK | Elasticsearch | Kibana ● Setup Monitoring framework on top of our ETL data pipeline ● Designed a Dashboard in Kibana to check record counts in different zones ● Implemented Alerting modules on top of Gray Trace – Gray Traffic Detection System Duration (06/2016 – 05/2017) Customer: Jazz & SCO, Pakistan Telecommunication Islamabad, PK Hortonworks HDP Framework | Hive | Spark | Qliksense / Tableau | Hadoop ● Developed Gray Traffic Detection System using advanced analytics ● Successfully detected 90% of the gray calls; Identified location of SIM boxes and cell sites involved in gray traffic; Identified MSISDNs and IMEI’s involved in the fraud. ● Qlik Sense based reporting using Business Intelligence Dashboards Education BS(CS) BIIT – University of Arid Agriculture, Islamabad Duration (1998 – 2002) FSC Anti-Money Laundering Search Engine Govt College Satellite Town, Rawalpindi Duration (1994 - 1995) 06/2017 Customer: Mobilink Microfinance Bank Ltd Micro-Finance Banking Islamabad, PK Elastic Search | Kibana | Hive | Spark | Hadoop ● Used Elastic Search to index blacklists issued by UN Security Council, State bank and other regulatory bsodies ● Developed interface for MMBL business managers to verify customer against potential AML involvement of customers ● Developed an intelligent watch-dog to trigger alarms against customers potentially involved in money laundering. Advanced Analytics 03/2017 Ministry of Defence & NRTC Defence Islamabad, PK Hortonworks | Hadoop | Hive | Spark | Qliksense ● Advanced analytics solution for Pakistan Army.