Tarun Modadugu
Data Engineer
Results-driven GCP Data Engineer with 3 years of experience designing, building, and optimizing scalable data pipelines and cloud-based data solutions. Strong expertise in Python, SQL, PySpark, Apache Spark, Airflow, dbt, BigQuery, Dataproc, Cloud Storage, and GCP, with experience in ETL/ELT, data modeling, legacy pipeline migration, and workflow automation. Proven track record of improving pipeline performance, automating manual processes, optimizing cloud workloads, and delivering reliable data solutions.
Software Engineer
Accenture
- Client : Virgin Media O2
- Successfully assisted the migration of legacy data pipelines from Cloudera (NiFi, Hive) to Google Cloud Platform, developing internal frameworks (CIF, DVL) to significantly improve scalability, reduce pipeline failures by 40%, and accelerate data processing by 30%.
- Designed and orchestrated robust Apache Airflow pipelines to automate decryption, decompression, and movement of files from Cloud Storage to processing buckets, enabling seamless ingestion of 100+ GB of data daily.
- Built a scalable DVL framework using DBT and Dataproc (PySpark) to validate and transform data from BigQuery external tables, ensuring complete data consistency and achieving 100% data quality before loading into staging layers.
- Automated multiple job-triggering workflows using Python, reducing manual operational effort and improving overall pipeline reliability and efficiency.
- Optimized BigQuery workloads through advanced query tuning, table partitioning, and cost-efficient storage strategies, resulting in a 30% reduction in query latency and a 25% decrease in monthly query costs.
Data Engineer
Lloyds Technology Centre
- Automated FSCS regulatory monitoring using Apache Airflow, dbt, and Python, reducing processing time by over 95%.
- Developed SCD Type 2 models in dbt to enhance historical customer record maintenance and data traceability.
- Migrated legacy Hive MapReduce pipelines to Java Spark, reducing processing time by approximately 1-2 hours and improving overall pipeline efficiency.
- Built a multithreaded Python API client with throttling and rate-limiting mechanisms to stream data in batches to downstream teams, improving data accessibility and reducing data delivery delays.
- Designed business-specific dimension tables using dbt and Airflow to deliver tailored datasets and reduce downstream dependencies on centralized data.
Text-to-Learn: AI-Powered Course Generator
Built an AI-powered learning platform that turns any topic into a structured course with AI-generated lessons and curated YouTube videos using React and Node.js/Express. Implemented secure authentication with email/password and Google OAuth 2.0 using Passport, JWT, and session-based OAuth state. Developed a modular REST API with Express + Mongoose to persist users, courses, topics, and AI outputs; enforced validation and consistent error handling. Integrated RabbitMQ (CloudAMQP) to offload AI generation into an async job queue with a dedicated worker, jobId-based polling, status tracking (pending/processing/completed/failed), and idempotent job submission to prevent duplicate processing under burst traffic. Integrated the OpenAI API to generate course outlines and detailed subtopic explanations and stored results for fast retrieval. Connected YouTube Data API v3 to fetch relevant videos per subtopic and displayed them alongside AI content for multimodal learning. Deployed the backend on a mobile device using Termux, exposed securely via a Cloudflare Tunnel reverse proxy, enabling public access without traditional server infrastructure.
Computer Science and Engineering
Andhra Loyola Institute of Engineering and Technology
Google Cloud Certified Professional Data Engineer
Google Cloud Certified Professional Cloud Architect
Google cloud certified Associate Cloud Engineer
Attendee · Oct 24, 2026