Jose Antonio Da Silva Dias

I am a Data Engineer specialized in designing and maintaining scalable data solutions within the AWS ecosystem. My work focuses on building reliable pipelines that enable data-driven decision-making and support analytics, governance, and business intelligence initiatives.

I have solid experience with AWS Glue, Lambda, Step Functions, and S3, developing and orchestrating ETL processes that automate data ingestion, transformation, and storage across different layers of the data lake. I’m also skilled in Athena and DynamoDB, which I use to query, integrate, and manage structured and semi-structured data efficiently.

My background in Python allows me to create robust and modular data pipelines, integrating APIs, implementing validation routines, and optimizing performance using PySpark. I pay close attention to code quality, reusability, and maintainability, ensuring that solutions remain efficient and scalable as data volumes grow.

I also work with Terraform to provision and automate AWS resources, promoting infrastructure as code practices that improve reliability, consistency, and deployment speed. This approach helps ensure seamless data operations and better control over the environments where data pipelines run.

Data quality and governance are central to my approach. I implement mechanisms to validate, monitor, and track data lineage, increasing trust and compliance across data assets. I believe that clean, well-structured, and traceable data is the foundation for all successful analytics and machine learning initiatives.

Driven by curiosity and continuous improvement, I’m always looking for ways to optimize performance, reduce costs, and simplify processes within AWS. My goal is to deliver high-impact data solutions that empower teams and organizations to make smarter, faster, and more reliable decisions.