Hadoop Administrator Job Description
Hadoop administrator
provides support and maintenance and its eco-systems include HDFS, Yarn, Hive, Impala, Spark, Kafka, HBase, Cloudera Work Bench, Splunk and Power BI.
Hadoop Administrator Duties & Responsibilities
To write an effective hadoop administrator job description, begin by listing detailed duties, responsibilities and expectations. We have included hadoop administrator job description templates that you can modify and use.
Sample responsibilities for this position include:
Leverage NoSQL, Hive, Sqoop, Flume, Yarn, and other Hadoop ecosystem utilities for data movement
Set-up high availability and disaster recovery for Hadoop cluster
Monitor and audit SNAP ingest process (Filemon, Kafka, X15)
Monitor and audit SNAP ingest and streaming processes (Filemon, Kafka, X15, Data Torrent, Striim, etc)
Perform data extraction from SQL Server, HAWQ, Hive, X15 to support audit/expedited requests
Perform network administration (connections, data flows, firewalls, performance, access)
Understanding of Hadoop security, specifically Kerberos integration with ability to implement
Understanding of job and task scheduling and administration of jobs
Understanding of data ingestion using Sqoop and/or Flume
Review Hadoop environments and determine compliance with industry best practices
Hadoop Administrator Qualifications
Qualifications for a job description may include education, certification, and experience.
Licensing or Certifications for Hadoop Administrator
List any licenses or certifications required by the position: SSL, CE, AWS, SAP, HDP, DOD, TLS/SSL, GCP, II, IAT
Education for Hadoop Administrator
Typically a job would require a certain level of education.
Employers hiring for the hadoop administrator job most commonly would prefer for their future employee to have a relevant degree such as Bachelor's and Master's Degree in Computer Science, Technical, Engineering, Information Technology, Education, Information Systems, Business, Computer Engineering, Administration, Software Engineering
Skills for Hadoop Administrator
Desired skills for hadoop administrator include:
Hive
YARN
HBase
Spark
Troubleshooting Core Java Applications
Disaster recovery
Performance
Kafka
MapReduce
Best practices related to security
Desired experience for hadoop administrator includes:
Ability to deploy Hadoop cluster, add and remove nodes, keep track of jobs, monitor critical parts of the cluster, configure name-node high availability, schedule and configure it and take backups
Well versed in installing & managing distributions of Hadoop (Hortonworks, Cloudera )
Experience using and troubleshooting Open Source technologies
A record of working effectively with application and infrastructure teams
Review existing DB infrastructure and determine areas of improvement
Implement software lifecycle methodology to ensure supported release and roadmap adherence
Hadoop Administrator Examples
1
Hadoop Administrator Job Description
Job Description Example
Our company is hiring for a hadoop administrator. Thank you in advance for taking a look at the list of responsibilities and qualifications. We look forward to reviewing your resume.
Responsibilities for hadoop administrator
- Accountable for storage, performance tuning and volume management of Hadoop clusters and MapReduce routines
- Involves designing, capacity arrangement, cluster set up, performance fine-tuning, monitoring, structure planning, scaling and administration
- Monitor Hadoop cluster connectivity and performance
- SQL and performance tuning at all levels
- Document new environments and develop standards for organization to support a robust
- Directs and participates with team members in the analysis, development and delivery of all support and project work for assigned function(s)
- Capacity planning and implementation of new/upgraded hardware and software releases for storage infrastructure
- Application support (Leavely, Platfora, Securonix, Splunk/Hunk, Katana)
- Change management (svcnow, hardware upgrades/replacements, vendor engagement)
- Support and reengineer Python scripts (Konflux)
Qualifications for hadoop administrator
- Ability to communicate technical ideas and concepts to less technical audiences
- Experience with industry-standard operational practices such as change management, incident management
- Expert scripting with shell scripting (Bash, PHP, PERL, Python)
- Experience with RDBMS technologies and SQL language
- Experience with Hadoop Ecosystem Security (Kerberos, AD integration, and/or other ecosystem components such as Ranger, Knox, Rhino, Falcon)
- Experience with Amazon AWS and AWS technologies such as EMR, EC2, IAM, S3, Lambda, Data Pipeline, are a strong plus
2
Hadoop Administrator Job Description
Job Description Example
Our growing company is searching for experienced candidates for the position of hadoop administrator. Please review the list of responsibilities and qualifications. While this is our ideal list, we will consider candidates that do not necessarily have all of the qualifications, but have sufficient experience and talent.
Responsibilities for hadoop administrator
- Frameflow alert creation and active monitoring
- Assets Inventory (Vuln remediation, IAM, AIM, CIITAM, SvcNow, OpsMetrics, internal tracking)
- Security inventory (IIQ groups, service accounts, passwords in EPV, CyberArk cred provider)
- Manage and coordinate platform-wide backups and restores
- Isilon & Cisco Administration
- ESX and VMware administration (configure/deploy VM resources using AXP approved tools)
- Puppet and kick start programming
- Ability to effectively communicate with enterprise storage, network, and backup teams
- Configure and deploy vSphere 5.5, vSphere 6.X, vCenter 5.5, vCenter 6.X, virtual machines, vNetworking (preferred with distributed vSwitches)
- Support systems that conform to architectural guidelines and QA policies and procedures
Qualifications for hadoop administrator
- This position will be responsible for implementation and support of the Enterprise Hadoop environment, including
- The administrator consultant will have experience in installing, administering, and supporting Windows and Linux operating systems /hardware in an enterprise environment
- Expertise in system administration and programming skills
- Ability to work independently in a team environment on moderate to highly complex issues
- AWS and Google Compute experience a plus
- Five years data mining experience
3
Hadoop Administrator Job Description
Job Description Example
Our growing company is searching for experienced candidates for the position of hadoop administrator. To join our growing team, please review the list of responsibilities and qualifications.
Responsibilities for hadoop administrator
- Working with data application teams and customers to setup new Hadoop users, Linux users, Kerberos principals and LDAP integration and verification of HDFS, Hive, Pig and MapReduce access for the new users
- Performance tuning and troubleshooting of Hadoop clusters and Hadoop MapReduce routines
- Identify, remediate, and track technical issues in a highly complex and dynamic environment
- Plan, coordinate and implement security measures to safeguard information in data warehouse against accidental or unauthorized damage, modification or disclosure
- Learn and execute change/incident management processes and procedures
- Recognize and work appropriately with confidential and sensitive information
- Integrate innovative solutions involving different technologies/products and support the development of testing and acceptance of prototypes and product recommendations
- Create and follow documented processes and procedures while implementing improvements
- Leverage system admin skills for automation, optimization, and operational efficiencies
- Ensure issues are identified, tracked, documented, and reported on and resolved in a timely manner
Qualifications for hadoop administrator
- 4+ years of administrator experience on Big Data Hadoop Ecosystem and related technologies (Sqoop, Hive, Pig, Flume)
- Prior experience with performance tuning, capacity planning, and workload mapping a plus
- Coding and developing in Hadoop
- 3+ years of strong Linux and Hadoop administration background in a production support setting is required
- Expert in SQL and/or Shell scripting, stored procedures, and construction of highly complex queries is required
- Hands on experience working with unstructured data is required
4
Hadoop Administrator Job Description
Job Description Example
Our innovative and growing company is looking to fill the role of hadoop administrator. If you are looking for an exciting place to work, please take a look at the list of qualifications below.
Responsibilities for hadoop administrator
- Have technical knowledge about the administration and control of the Hadoop System, including the associated operating system, related tools, network and hardware
- Creating and maintaining engineering documents and system designs
- Maintain appropriate written documentation for operational procedures and system design
- Experience with monitoring systems and diagnostic tools
- Development, adherence, and enforcement of security standards and policies
- On call production support (24x7)
- Basic understanding of Computer processing operations and Operating Systems
- Review computer systems logs and highlight potential issues for further research
- Write basic computer batch and shell scripts to solve reporting or automation issues
- Add, remove and modify user access to systems with basic uidance and assistance
Qualifications for hadoop administrator
- Understanding of architecture and support around Big Data and BI solutions is required
- Experience with database and query performance tuning, sizing, and optimization is required
- Deep knowledge of server hardware and configuration is required
- Strong understanding of high availability, fault tolerance, load balancing, and clustering is preferred
- Expert in scripting, stored procedures, and construction of highly complex queries is preferred
- Production support experience with SQL Server, Greenplum, PostgreSQL, or Oracle is a plus
5
Hadoop Administrator Job Description
Job Description Example
Our innovative and growing company is searching for experienced candidates for the position of hadoop administrator. If you are looking for an exciting place to work, please take a look at the list of qualifications below.
Responsibilities for hadoop administrator
- Install basic software licenses for renewals
- Reset file permissions and make modifications to directory structures as needed
- Assist in monitoring and tuning the application
- Remidate system vulnerabilities based on documentation and research
- Seeking college degree in Computer Science or other related field
- December 2017 Graduates
- Exposure to Hadoop//Linux
- Basic understanding of server computing
- Supports the Database Administration functions of the system
- Stewardship the overall Cloudera architecture and infrastructure
Qualifications for hadoop administrator
- Working knowledge of clustering, replication, and log shipping techniques is a plus
- Support experience and/or knowledge of DASD, Isilon, X15, Kafka, Filemon, Konflux, or Splunk is a plus
- Experience administering Pivotal Hadoop or Hortonworks distribution is a plus
- Experience administering TCP/IP based networks is a plus
- Experience administering & supporting Cisco UCS is a plus
- Experience with performance engineering for storage is preferred