Data Engineer (Production Support – AWS EMR)
Data Engineer (Production Support – AWS EMR)
22
Hyderabad
Job Views:
Created Date: 2026-06-25
End Date: 2026-08-23
Experience: 10 - 12 years
Salary: 1500000
Industry: IT
Openings: 1
Primary Responsibilities :
Production Support & Incident Management
- Monitor and support AWS EMR clusters and Big Data applications in production environments.
- Troubleshoot and resolve production issues, failures, and performance bottlenecks.
- Perform root cause analysis (RCA) and implement permanent fixes.
- Participate in on-call support and critical incident management.
- Analyze application logs and system alerts to proactively identify issues.
AWS EMR & Cluster Management
- Manage AWS EMR cluster lifecycle including provisioning, scaling, optimization, and decommissioning.
- Monitor cluster health, resource utilization, and cost efficiency.
- Apply upgrades, patches, and security updates to AWS EMR environments.
- Optimize cluster performance and resource allocation.
Data Pipeline Support
- Support ETL/ELT pipelines built using Spark, Scala, Talend, Hive, and Presto.
- Ensure smooth execution of data ingestion, transformation, and loading processes.
- Maintain data quality, consistency, and availability across platforms.
- Support integrations with S3, Redshift, Snowflake, MySQL, Aurora, and PostgreSQL.
Performance Optimization
- Tune Spark jobs and Hive queries for maximum efficiency.
- Optimize SQL queries and data processing workflows.
- Improve storage strategies and data access performance.
- Identify and resolve long-running jobs and resource-intensive processes.
Monitoring & Reporting
- Configure and maintain monitoring tools such as:
- AWS CloudWatch
- Datadog
- Prometheus
- Create dashboards, alerts, and health monitoring reports.
- Generate daily, weekly, and monthly operational reports.
- Monitor SLA compliance and system performance metrics.
Automation & Workflow Management
- Develop automation scripts using Python, Shell Scripting, or Java.
- Manage workflows using:
- Apache Airflow
- Oozie
- AWS Step Functions
- Automate repetitive operational tasks to improve efficiency.
Collaboration & Documentation
- Work closely with Data Engineers, Developers, DevOps, QA, and Business teams.
- Maintain SOPs, runbooks, troubleshooting guides, and architecture documents.
- Create source-to-target mappings, flow diagrams, and system documentation.
- Support deployment activities and release management processes.
Experience Requirements:
Education
- Bachelor's Degree in:
- Computer Science
- Information Technology
- Engineering
- Related Technical Discipline
Experience
- Minimum 10+ years of experience in Data Engineering, Big Data, or Production Support.
- Minimum 3–5 years of hands-on AWS Cloud experience.
- Experience supporting enterprise-scale distributed systems.
Technical Skills
AWS Technologies
- AWS EMR
- Amazon S3
- AWS Lambda
- AWS Step Functions
- AWS CloudWatch
- AWS Redshift
- Aurora MySQL
- PostgreSQL
Big Data Technologies
- Apache Spark
- Scala
- Hive
- Presto
- Kafka
- Apache NiFi
ETL & Data Integration
- Talend
- Sqoop
- Any enterprise ETL tool
Workflow & Scheduling
- Apache Airflow
- Oozie
Programming & Scripting
- Scala
- Python
- Shell Scripting
- Java
Database Skills
- Advanced SQL
- Query Optimization
- Data Warehousing Concepts
- Data Modeling
Build & Deployment Tools
- Maven
- Bamboo
- Stash
Preferred Requirements
- Experience with CI/CD tools:
- Jenkins
- GitLab CI/CD
- Experience with container technologies:
- Docker
- Kubernetes
- AWS Certifications:
- AWS Certified Solutions Architect
- AWS Certified Data Analytics / Big Data Specialty
- Knowledge of Data Governance and Cloud Security.
- Experience with enterprise monitoring solutions.
- Exposure to DevOps practices and Infrastructure as Code.
Soft Skills
- Strong analytical and problem-solving skills.
- Excellent troubleshooting capabilities.
- Ability to work in high-pressure production environments.
- Strong communication and stakeholder management skills.
- Customer-focused approach to production support.
- Ability to work independently and within cross-functional teams.
- Strong documentation and process management skills.
Key Skills
AWS EMR | Apache Spark | Scala | Talend | ETL | AWS Cloud | S3 | Lambda | Step Functions | CloudWatch | Redshift | Snowflake | MySQL | PostgreSQL | Kafka | NiFi | Hive | Presto | Airflow | Oozie | SQL | Python | Shell Scripting | Java | Data Warehousing | Data Modeling | Production Support | Big Data | Performance Tuning | Troubleshooting | CI/CD | Jenkins | Docker | Kubernetes | Monitoring | Cloud Computing
Location
: Alliance Recruitment Agency UAE
Share Job :
Loading share buttons...