Hadoop Database Administrator (DBA) AWS, PySpark, Splunk

Job Description: We are seeking a skilled Hadoop Database Administrator (DBA) to manage and maintain large-scale Hadoop environments, ensuring optimal performance, scalability, and reliability. The ideal candidate will have strong expertise in AWS cloud services , PySpark , and Splunk integration for performance monitoring and analytics.

Responsibilities:

• Manage and administer Hadoop clusters , including installation, configuration, and troubleshooting.

• Design, develop, and optimize data pipelines to support business and analytical requirements.

• Integrate AWS services (such as S3, EMR, Lambda, and EC2) with the Hadoop ecosystem.

• Implement and maintain Splunk dashboards for system monitoring, performance tuning, and alerting.

• Ensure high availability, security, and fault tolerance of Hadoop infrastructure.

• Conduct regular cluster maintenance, capacity planning, and performance tuning.

• Collaborate with data engineering and DevOps teams to support data workflows and automation.

• Maintain detailed documentation of configurations, procedures, and troubleshooting steps.

Required Skills:

• Strong hands-on experience with Hadoop ecosystem tools (HDFS, Hive, HBase, YARN, Oozie, etc.).

• Proficiency in AWS services and cloud-based data solutions.

• Expertise in PySpark for data processing and transformation.

• Experience with Splunk for system monitoring and performance analysis.

• Strong scripting skills (Python, Shell, or similar).

• Excellent problem-solving and communication skills.

Preferred Qualifications:

• Experience with big data performance tuning and cluster optimization .

• Knowledge of security frameworks (Kerberos, Ranger, or similar).

• Bachelor s degree in Computer Science, Information Technology, or related field.