Hadoop Developer Job Description
Hadoop developer
provides support and maintenance and its eco-systems include HDFS, Yarn, Hive, Impala, Spark, Kafka.
Hadoop Developer Duties & Responsibilities
To write an effective hadoop developer job description, begin by listing detailed duties, responsibilities and expectations. We have included hadoop developer job description templates that you can modify and use.
Sample responsibilities for this position include:
Implement SQL/Hive queries, jrxml templates and Jasper reports within a team using best practices to meet product or solution deliverables
Proficient user of SQL, preferably in Oracle environment
Drive high quality of all deliverables across complete SDLC – using Agile best practices for speed to market
Work with architecture team to provide input on standards and best practices
Champion good practice and techniques
Opportunity to be cross trained in advanced analytical technique
Refactor code/queries to increase operating efficiency and extensibility and/or adapt programs to new requirements
Mine the available HDFS data to produce reusable aggregate information that can enhance the overall business as requested by Product Management, Finance and other departments within QuickPlay
Experienced in performance, optimisation and troubleshooting highly technical issues on your own and within a collaborative team setting
Experienced with industry leading large scale database platforms
Hadoop Developer Qualifications
Qualifications for a job description may include education, certification, and experience.
Licensing or Certifications for Hadoop Developer
List any licenses or certifications required by the position: AWS, GCP
Education for Hadoop Developer
Typically a job would require a certain level of education.
Employers hiring for the hadoop developer job most commonly would prefer for their future employee to have a relevant degree such as Bachelor's and Master's Degree in Computer Science, Engineering, Education, Information Technology, Technical, Information Systems, Mathematics, Business, Science, Software Engineering
Skills for Hadoop Developer
Desired skills for hadoop developer include:
SQL
Theories
Principles
Database structures
Impala
Practices
Spark
Hadoop architecture components
Hive
Information Technology fields and computer systems
Desired experience for hadoop developer includes:
Experience optimizing and automating code fail safe design
Minimum 5 years’ experience performing data mining, analysis and/or implementing end to end enterprise grade software with a strong focus on Big Data
Proper understanding of machine learning algorithms
Familiarity with Apache Mahout and other machine learning open source projects
Excellent understanding of data mining concepts such as cluster analysis, pattern recognition, outlier evaluation and classification, correlation and prediction
Experience using NoSQL Databases such as Couch DB, MongoDB or Cassandra would be a plus
Hadoop Developer Examples
1
Hadoop Developer Job Description
Job Description Example
Our growing company is hiring for a hadoop developer. Thank you in advance for taking a look at the list of responsibilities and qualifications. We look forward to reviewing your resume.
Responsibilities for hadoop developer
- Build SQL/HIVE databases and tables
- Developing distributed computing Big Data applications using Spark, Elastic Search on HDP Hadoop
- Identify, capture and communicate the issues and risks promptly
- Coding various Services, Components, and participating in the development of Java and Big Data/Hadoop processes
- Conduct/Lead technical discussions with internal and external stakeholders
- Coordinate with cross-functional teams , testing teams and drive resolution of open items and issues
- End-to-end technology delivery for key strategic and regulatory projects
- Understand business requirements through user stories and user centred design, and identification of appropriate solutions
- Production of solution designs that meet the business requirements and are deliverable and supportable
- Experimentation with minimum viable products to verify best technology choices and solution design
Qualifications for hadoop developer
- Experience using J2EE (EJB, JMS, JDBC, JSP, JSF, Struts, Tag Libraries, AJAX), XML, WebServices, Oracle (database modeling) would be a major plus
- Able to build complex queries based on large data sets aggregated across various client applications
- Excellent understanding of relational query languages
- Experience using Talend Studio 5.2 and above or another ETL tool eg
- Experience using NoSQL Databases such as MongoDB or Cassandra would be a plus
- Communicate with direct an Offshore team
2
Hadoop Developer Job Description
Job Description Example
Our company is hiring for a hadoop developer. If you are looking for an exciting place to work, please take a look at the list of qualifications below.
Responsibilities for hadoop developer
- Test possible solutions with customers as early as possible
- Manage big data infrastructure including data processes
- Execute end-to-end unit testing (Extraction, transformation, load)
- Execute data load and support data reconciliation in scope with data migration
- Participate in defect analysis and resolution
- Provide Enterprise Hadoop Development expertise that will be supporting the business functions
- Work with the team to create and maintain design documents and participate in cross training with other team members
- Support existing solutions and assist in the resolution of hard to solve data problems
- Structure and design framework including applications design
- Administration, maintenance, control, and optimization of Hadoop capacity, security, configuration, process scheduling, and errors
Qualifications for hadoop developer
- Must have excellent communication skills and be able to present a coherent picture to people with technical and finance backgrounds
- Implement complex algorithms efficiently using PIG Latin language and user defined function in Java/Python
- Create and optimize workflow using Oozie scripts
- Research into Hadoop ecosystem for efficient and optimized solutions
- Create solutions involving disparate data sources viz
- Ability to architect and design optimized solutions for handling large data in an efficient and time bound manner
3
Hadoop Developer Job Description
Job Description Example
Our innovative and growing company is looking for a hadoop developer. To join our growing team, please review the list of responsibilities and qualifications.
Responsibilities for hadoop developer
- Coding various Services, Components, and participating in the development of Java and Big Data/Hadoop processes and Platform build outs
- Develop applications using open source or in house frameworks
- Evaluating existing and proposed data models and how to best access and query them existing and proposed data interfaces and how to clearly document them, including specification of data flow models, data flow timing, data mapping, and data transformation rules including data validations and controls
- Write PIG scripts, develop data integration and ingestion processes
- Architect data pipelines and integrations with internal and external sources
- Build job automation and scheduling using Control-M, unit and system testing
- Designing, developing, testing, tuning and building a large-scale data processing system, for Data Ingestion and Data products that allow the Client to improve quality, velocity and monetization of our data assets for both Operational Applications and Analytical needs
- Install, configuration, management and monitoring of various Hadoop and database systems
- Perform upgrades, scripting, task automation, backups/recovery
- Documentation of the installation and upgrade processes
Qualifications for hadoop developer
- Total experience of 4-6 years with at least 3+ years on Hadoop technologies
- Skilled in algorithm design is desirable
- Understanding of RDBMS technologies and SQL language
- Experience with Spark, HBase, Java (MapReduce), Python (linux shell like scripts) development
- 3 years proven experience in ETL (Syncsort DMX-h, Ab Initio, IBM - InfoSphere Data Replication, ), mainframe skills, JCL
- Proficiency with Java, Python, Scala, HBase, Hive, Map Reduce, ETL, Kafka, Mongo, PostgreSQL, Visualization technologies
4
Hadoop Developer Job Description
Job Description Example
Our company is growing rapidly and is looking for a hadoop developer. Thank you in advance for taking a look at the list of responsibilities and qualifications. We look forward to reviewing your resume.
Responsibilities for hadoop developer
- Loading from disparate data sets using Hadoop stack of ingestion and workflow tools
- Configure and implementation of Data Marts in Hadoop platform
- Design and develop efficient systems that involve Big Data
- Integrate 3rd party API's (REST, JSONP, OAuth)
- Join a tightly knit team solving hard problems the right way
- Own meaningful parts of our service, have an impact, grow with the company
- Engineer will be responsible for documenting detailed system specifications, design, development, testing using development tools & continuous integration processes
- Engineer has the capability to handle ad-hoc requests
- Engineers lead efforts, oversee work results, provide training and serve as a technical resource for other Developers
- Responsibility for the Development and Implementation of data-stream components for Industrial Iot Applications
Qualifications for hadoop developer
- Proficiency with Java, Python, Scala, HBase, Hive, Map Reduce, ETL, Kafka, Mongo, Postgres, Visualization technologies
- BS in Computer Science or similar technical degree or specialized training/equivalent work experience
- Must be hands-on, strong team player and be extremely flexible and able to work in deadline driven, fast paced environment adopting to constantly changing business priorities
- Be self-starter, ability to work independently or with others and achieve goals in a matrixed org setting
- New and emerging technologies excite you, but you are mindful enough to employ the right solutions for addressing the business problems
- Above 4 years of working experience
5
Hadoop Developer Job Description
Job Description Example
Our company is searching for experienced candidates for the position of hadoop developer. We appreciate you taking the time to review the list of qualifications and to apply for the position. If you don’t fill all of the qualifications, you may still be considered depending on your level of experience.
Responsibilities for hadoop developer
- Identification of new technological toolchain/implementation trends in Big Data
- Align with the data science team regarding machine learning efforts in the cloud
- Participate in development of data store, risk margin and client reporting applications as a developer
- Work with large systems that comprise of several applications
- Participate or lead in design of large scale analytical systems
- Participate or develop large scale systems
- Spend up to 25% of their time doing administration and maintenance activities during critical data gathering seasons
- Spends as much time as possible learning the data types of our science domain
Qualifications for hadoop developer
- Candidates need to have and advanced knowledge of technologies such as technologies such as J2EE, Spring, Spark, Hadoop ,Vertica
- Expertise in Bigdata technologies is a plus
- Bachelor of Science in Computer Science, Information Technology or equivalent experience
- 2+ years of hands on development experience with Hadoop ecosystem
- ETL workflow development and scheduling experience
- Proficiency in Java and/or Scala development, in a Linux based environment