Data Engineer with 2+ years of experience in building and optimizing ETL pipelines, data processing, and workflow orchestration. Proficient in Python, SQL, Apache Spark, and Apache Airflow, with hands-on experience in developing scalable data engineering solutions. Strong problem-solving and analytical skills, with a continuous-learning mindset and the ability to adapt quickly to new technologies and business requirements.
Data Engineer
DataEconomy
- Developed Python/FastAPI services to dynamically generate and manage Apache Airflow DAGs, enabling pipeline orchestration, scheduling, execution, and self-service job lifecycle management.
- Built centralized Airflow job monitoring and management capabilities, including manual and scheduled runs, start/stop/pause/resume controls, log access, and custom scheduling calendars.
- Implemented automated notification services integrating Jira, Microsoft Teams, and email for job start, warning, success, failure, and retry events.
- Developed and optimized ETL pipelines using PySpark and Polars, improving data processing performance and resource efficiency.
- Built automated source-to-target reconciliation pipelines to identify matched, mismatched, missing, and duplicate records, strengthening data quality validation.
Trainee Engineer
DataEconomy
- Developed production-grade Python/FastAPI REST APIs using SQLAlchemy and Pydantic for automated schema discovery, metadata extraction, validation, and data cataloging across enterprise data sources.
- Built XCrawler, a scalable metadata extraction and cataloging service to automate dataset discovery and centralize metadata management across heterogeneous data systems.
- Implemented reusable, plug-and-play data-source onboarding components, enabling seamless integration of new data platforms with minimal configuration.
- Improved metadata quality and consistency through standardized API development, robust data validation, and maintainable architecture.
- Collaborated with engineering teams to troubleshoot issues, optimize services, and reduce manual effort in metadata discovery and data-source onboarding.
B. Tech
R.V.R. & J.C. College of Engineering · 8.11/10
Intermediate (MPC)
HIE Junior College · 980/1000
10th
Geetha Gurukulam High School · 10/10
Databricks Certified Data Engineer Associate
Databricks
Oct 2026Databricks Accredited Generative AI Fundamentals
Databricks
Aug 2026Astronomer Certification: DAG Authoring for Apache Airflow 3
Astronomer
Nov 2025Astronomer Certification: Apache Airflow 3 Fundamentals
Astronomer
Sept 2025
Attendee · Oct 24, 2026