Big Data Developer Job Description
Big data developer
provides care and feeding of our Big Data environments and its interfaces built upon technologies in the Hadoop Ecosystem including Hive, Hbase, Spark, and Kafka.
Big Data Developer Duties & Responsibilities
To write an effective big data developer job description, begin by listing detailed duties, responsibilities and expectations. We have included big data developer job description templates that you can modify and use.
Sample responsibilities for this position include:
Development (Coding) for Hadoop, Spark and Java and Angular Js
Collaborate with like-minded team members to establish best practices, identify optimal technical solutions (20%)
Reviewing code and providing feedback relative to best practices, improving performance
Design, develop and test a large-scale, custom distributed software system using latest Java, Scala and Big data technologies
Adhering to the appropriate SDLC and Agile practices
Actively contribute to the technological strategy definition (design, architecture and interfaces) in order to effectively respond to our client’s business needs
Participate in technological watch and the definition of standards to ensure that our systems and data warehouses are efficient, resilient and durable
Able to provide guidance and coaching to associate software developers
Experienced in using Informatica or similar products, with an understanding of heterogeneous data replication techniques
Comfortable with performance tuning, improvement, balancing, usability and automation
Big Data Developer Qualifications
Qualifications for a job description may include education, certification, and experience.
Licensing or Certifications for Big Data Developer
List any licenses or certifications required by the position: AWS, II, IAT, DB, SDLC, AI, AGILE, IC, SSL
Education for Big Data Developer
Typically a job would require a certain level of education.
Employers hiring for the big data developer job most commonly would prefer for their future employee to have a relevant degree such as Bachelor's and Master's Degree in Computer Science, Engineering, Technical, Information Systems, Science, Information Technology, Mathematics, Graduate, Software Engineering, Education
Skills for Big Data Developer
Desired skills for big data developer include:
Hive
SQL
Impala
Python
Spark
Kafka
Application
Data and infrastructure architecture disciplines
Practices
Java
Desired experience for big data developer includes:
Champion the creation of BKMs, guidance & framework of all Big Data development technologies such as Nutch, Sqoop, and flume, Hive, Pig, Mahout, and HBASE Also coach and ramp the other Big Data team members
Good understanding of the TDFG’s retail businesses, their operations
Familiarity with Hadoop, Agile development methodologies
Experience with scripting languages (Python, Perl)
Understanding of Analytics software, R, SAS, Matlab
1+ years of Java programming and Scrum Agile development
Big Data Developer Examples
1
Big Data Developer Job Description
Job Description Example
Our company is searching for experienced candidates for the position of big data developer. To join our growing team, please review the list of responsibilities and qualifications.
Responsibilities for big data developer
- Defines, develops or modifies application modules or enterprise-wide software systems using disciplined software development processes and makes modules production ready by moving them to libraries, completing forms, following procedures, completing version control documents
- Develop Hadoop and Web-Service based applications and components (50%)
- Interface with Users, BAs, and Developers to walk through data design (30%)
- Define comprehensive unit test coverage of data transformations
- Demonstrating analytical technical skills
- Should have basic knowledge in BSS data model specially CRM, OMS, Billing
- Be part of a team of Hadoop developers working on Big Data project with ample opportunity to grow professionally, functionally and technically
- Work with other groups to ensure smooth delivery with high standards and quality
- Understand requirements, propose and deliver superior solutions
- Write well-structured beautiful clean code
Qualifications for big data developer
- Understanding of Big Data Hadoop Ecosystem components (Sqoop, Hive, Pig, Flume)
- Experience with Java or Big Data Technologies a plus
- Any big data experience with Hadoop, HDFS, cluster management
- 6 years' experience in designing & developing enterprise application solution for distributed systems
- Experience with Hive, Pig & Map Reduce and Hadoop ECO system Framework
- Experience with HBase, Talend, NoSQL databases-Experience Apache Spark or other streaming big data processing preferred
2
Big Data Developer Job Description
Job Description Example
Our company is growing rapidly and is looking for a big data developer. Please review the list of responsibilities and qualifications. While this is our ideal list, we will consider candidates that do not necessarily have all of the qualifications, but have sufficient experience and talent.
Responsibilities for big data developer
- Experience with Big Data technologies will be a plus (Hadoop, Spark, Kafka, Hbase, etc)
- Write SQL queries to validate the dashboard output
- Working experience with database environment - understanding relational database structure and hands-on SQL knowledge to extract/manipulate data for variance testing
- Performing code reviews and pair programming
- Supporting and enhancing current applications
- Design, develop, test, and implement the applicationInvestigate and resolve complex issues while supporting existing applications
- You will work under general direction within a clear framework of accountability
- Be part of the growing Hadoop community
- Act as senior developer on big data security analytics technologies
- Rapidly develop prototype solutions, integrating a variety of open-source and proprietary technologies
Qualifications for big data developer
- Solid understanding of data structures & common algorithms Understanding of time-complexity of algorithms
- Expert-level skills and experience in Java or any distributed programming experience
- Expert-level skills and experience in Hadoop ECO System
- 6 years experience in designin & developing enterprise application solution for distributed systems
- Experienve with Hive, Pig & Map Reduce and Hadoop ECO system Framework
- Knowledge on Python & Scala
3
Big Data Developer Job Description
Job Description Example
Our company is hiring for a big data developer. To join our growing team, please review the list of responsibilities and qualifications.
Responsibilities for big data developer
- Help drive an agile, iterative, DevOps-based lifecycle
- Contribute novel ideas for extracting security intelligence from data
- Identify and document Data Quality rules
- Design, translate business requirements, develop, test, monitor, troubleshoot and administrate our big data environment using our ETL/Data Virtualization/BI Reporting/Big Data Management toolsets, and participate in an on-call rotation
- Participate in maintenance and upgrades for our Big Data platform and toolsets
- Participate in data governance initiatives, maintain metadata (data definitions, relationships, sources, and consumers), and contribute to the company’s Big Data strategies and roadmaps
- Contribute to the support and resolution of incidents which involve all data platforms which includes on-call rotation
- Communicate and accurately document all architectural decisions, plans, goals, functional requirements, and open items and issues to key stakeholders, including management, development teams, and business areas
- Participates in short- and long-term planning sessions with customers to improve business processes and works closely with Technical Review Board and architects to assure all systems are in line with IT long-term strategy
- Assists in developing buy vs
Qualifications for big data developer
- Strong knowledge loading from disparate data sets
- Good skills and experience in Linux, Java, XML, JSON, REST
- Experience with Hadoop and other related technologies
- Deep knowledge of Hive, SQL, databases and data warehouses
- Fluency in Unix
- High attention to detail and constructive attitude
4
Big Data Developer Job Description
Job Description Example
Our company is growing rapidly and is hiring for a big data developer. If you are looking for an exciting place to work, please take a look at the list of qualifications below.
Responsibilities for big data developer
- Manages customer's priorities of projects and requests
- Assesses customer needs utilizing a structured requirements process (gathering, analyzing, documenting, and managing changes) to prioritize immediate business needs and advising on options, risks and cost vs
- Design and implementation of software product (BigData related) including data models and visuzlisations
- Demonstrate participation with the teams you work in
- Deliver good solutions against tight timescales
- Be pro-active, suggest new approaches and develop your capabilities
- Share what you are good at while learning from others to improve the team overall
- Show that you have a certain level of understanding for a number of technical skills, attitudes and behaviours
- Delivering great solutions
- Being focused on driving value back into the business
Qualifications for big data developer
- Pre-processing using Hive / Pig / python/ Unix scripting
- Familiarity with map-reduce and Spark
- A deep understanding of algorithms and data structures
- Experience with an open source software contributor, including GitHub, code.google.com, or apache.com
- Preferred 1 year of production Hadoop experience
- Excellent planning/organizational, problem-solving, analytical, and communication skills, strong presentation and decision-making skills
5
Big Data Developer Job Description
Job Description Example
Our company is growing rapidly and is hiring for a big data developer. To join our growing team, please review the list of responsibilities and qualifications.
Responsibilities for big data developer
- Working with the other principals and developers to oversee and drive agile SDLC
- Collaboratively work with development peers through iteration tasks (pair programming, code reviews)
- Create and implement simple, pragmatic designs to solve real problems
- Takes initiative and proactively communicates within and among the team...a real collaborator
- Collaborate with testing team to ensure productive code
- Use MapReduce to process large data sets
- 3+ years of overall software development experience in an object oriented language
- Recruiting, leading and inspiring a team capable of delivering great solutions
- Form part of the leadership team of DACE Development demonstrating the necessary leadership skills and style
- Strive to become a luminary in data development leadership
Qualifications for big data developer
- Experience developing code on distributed databases using Spark, HDFS, Hive
- 5+ years of experience in Application Developer / Data Architect, or equivalent role
- Strong knowledge of data and data models
- Good understanding of data consumption patterns by business users
- Solid understanding of business processes and structures
- Basic knowledge of the securities trading business and risk