B.Tech Computer Science

VNR VJIET - (2013 - 2017)

Data Engineer

Fifth Third Bank - (2023 - 2024)

Designed and optimized Python-based ETL pipelines leveraging BigQuery, AWS services (S3, EC2, Lambda), and data formats like JSON and CSV to process and transform large datasets efficiently. 
Developed and automated workflows using Apache Airflow, Terraform, and AWS CloudFormation to streamline ETL processes, manage AWS infrastructure, and enhance pipeline reliability. 
Worked extensively with Spark (RDDs, PySpark, and Scala) to implement data transformations, migrate MapReduce jobs, and analyze semi/unstructured data in XML and JSON formats. 
Utilized Hive and Pig scripting to perform ETL operations, optimized queries with partitioning and bucketing, and enabled efficient data aggregation on AWS Cloud. 
Integrated multiple cloud databases (Snowflake, DynamoDB, Cloud SQL) with tools like Oracle PL/SQL, Sqoop, and PostgreSQL, ensuring seamless data access and scalable processing. 
Enhanced CI/CD pipelines using Bitbucket, SonarQube, and GitHub, managing version control, code quality, and repository migration while promoting best practices in Agile workflows. 
Designed data visualization solutions using Tableau, connecting to dynamic and static datasets, and performing SQL-based transformations for actionable insights. 

Data Engineer

Metlife - (2022 - 2023)

esigned and developed Spark applications using RDDs, DataFrames, and Datasets to perform large-scale data transformations, loading data into HDFS, and optimizing Spark SQL queries for efficient data processing.
Built scalable data pipelines on Azure Data Platform services, including Azure Data Lake, Data Factory, Databricks, and Azure SQL Data Warehouse, for data ingestion, transformation, and storage.
Automated workflows and ETL processes using Apache Airflow and shell scripting, ensuring seamless data pipeline execution and monitoring in production environments.
Developed customized UDFs in Spark and implemented data preprocessing and feature engineering using Python for handling large-scale datasets and addressing missing values.
Created and optimized data warehouse structures using dimensional modeling, including Star and Snowflake schemas, and implemented ETL processes for loading structured and unstructured data into HDFS.
Utilized Tableau and Power BI for creating interactive dashboards, enabling teams to derive insights from big data platforms by connecting to sources like SQL Server, Azure SQL, and Oracle.
Worked on MongoDB for CRUD operations, indexing, and replication, and integrated Docker containers with workflows to enhance scalability and deployment efficiency.

Data Engineer

Solugenix - (2019 - 2021)

Created scalable Spark applications using Scala and PySpark for large-scale data transformations, denormalization, and quality checks. Leveraged Spark SQL for efficient data processing and analysis.
Utilized GCP services like BigQuery, Dataflow, Cloud Run, Cloud SQL, and Vertex AI to build robust, cloud-native solutions that optimized workflows and enhanced system performance.
Streamlined deployment pipelines using Bamboo, Docker, and BitBucket for continuous integration and delivery, ensuring efficient build, testing, and deployment processes across environments.
Integrated OpenAI and Vertex AI for advanced natural language processing and data retrieval, improving chatbot capabilities and user interaction.
Extracted, transformed, and loaded data from JSON, relational databases, and other sources using Spark DataFrames and Hive. Conducted data profiling and developed data quality metrics with SQL and Python.
Prepared interactive Tableau dashboards to summarize key business metrics such as configurations, quotes, orders, and e-commerce data.
Actively participated in Agile Scrum processes using Jira to manage timelines, tasks, and resources, ensuring timely project deliveries and efficient teamwork.

Data Analyst

Mastercard - (2017 - 2019)

Performed extensive data profiling, requirement analysis, and pattern identification using SQL, PL/SQL, and complex queries across dimensional and relational data warehouses.
Developed and optimized MapReduce jobs for handling large datasets in Hadoop, leveraging HDFS compression and custom Hive UDFs for efficient data processing.
Automated the flattening of JSON data from Cassandra into structured formats using Hive and Python, enabling seamless integration into downstream processes.
Designed schemas in HBase, processed unstructured data formats like XML, JSON, and Avro, and implemented robust ETL pipelines in a Hadoop ecosystem.
Developed Python scripts for SQL injection detection, permission analysis, and vulnerability checks to enhance database security and optimize performance.