Big Data Software Engineer Job Description
Big data software engineer
provides leadership and guidance for developing and maintaining application architecture and development standards which adhere to industry best practices.
Big Data Software Engineer Duties & Responsibilities
To write an effective big data software engineer job description, begin by listing detailed duties, responsibilities and expectations. We have included big data software engineer job description templates that you can modify and use.
Sample responsibilities for this position include:
Extensive experience programming in Java experience in code optimization and high performance computing
Scripting tools such as Python
Architect, develop, and maintain applications and Web Services using Enterprise Java
Develop solutions to real-time and off line event collecting from various systems
MemSQL
Splunk/Hunk
Apache Pig
Skills &
Experience in software development of large-scale distributed systems &ndash
Good current knowledge of Unix/Linux environments
Big Data Software Engineer Qualifications
Qualifications for a job description may include education, certification, and experience.
Licensing or Certifications for Big Data Software Engineer
List any licenses or certifications required by the position: GCP, COMPTIA, ITIL, AWS
Education for Big Data Software Engineer
Typically a job would require a certain level of education.
Employers hiring for the big data software engineer job most commonly would prefer for their future employee to have a relevant degree such as Bachelor's and Master's Degree in Computer Science, Computer Engineering, Engineering, Technical, Programming Languages, Software Engineering, Education, Mathematics, Science, Information Technology
Skills for Big Data Software Engineer
Desired skills for big data software engineer include:
Application
Data and infrastructure architecture disciplines
Industry wide technology strategies and best practices
Architecture
Python
Econometrics
Machine learning
Operations research
Industry wide technology trends and best practices
Java
Desired experience for big data software engineer includes:
Enjoy working with data &ndash
Collection, transformation and enrichment frameworks such as Flume
Messaging middleware or distributed queuing technologies such as Kafka
MapReduce experience in Hadoop utilizing Pig, Hive, or other query/scripting technology
Expertise in data warehousing and business intelligence
Git, Maven, Jenkins, Sonar, Nexus, Puppet
Big Data Software Engineer Examples
1
Big Data Software Engineer Job Description
Job Description Example
Our innovative and growing company is searching for experienced candidates for the position of big data software engineer. Please review the list of responsibilities and qualifications. While this is our ideal list, we will consider candidates that do not necessarily have all of the qualifications, but have sufficient experience and talent.
Responsibilities for big data software engineer
- Contribute to other areas of the ADP as needed
- Analyze requirements, design, write, test and deploy highly functional code
- Actively participate in the team by making recommendations on how to improve product quality group productivity
- Actively participate in team by making recommendations on how to improve product quality group productivity
- Profile, monitor, undertake performance-analysis, optimization, scaling and performance tuning
- Keep up to date on current trends in the field of big data
- Enjoy working with data – data analysis, data quality, reporting, and visualization
- Visualization tools & libraries, reporting tools, Splunk (Hunk), Tableau, d3.js
- Work with cloud and on-premises product teams to understand data flow and locations and security concerns of information providers
- Divide and coordinate tasks with other development engineers
Qualifications for big data software engineer
- Knowledge/working experience on different BigData technologies - Hadoop, Map Reduce, YARN, Spark, Hive, Kafka, Avro, Cascading
- Experience with developing cloud services
- Experience in working on applications with either millions of users or processing terabytes of data per day
- At least bachelor degree in computer science or very similar discipline
- Previous experience Paid and Organic Customer Acquisition , Adwords, Facebook Advertising
- Experience with a data warehouse technology (Oracle, Teradata, Netezza, Redshift, HDFS Hive ) and relevant data modeling practices
2
Big Data Software Engineer Job Description
Job Description Example
Our company is growing rapidly and is looking for a big data software engineer. To join our growing team, please review the list of responsibilities and qualifications.
Responsibilities for big data software engineer
- Collaborate with with a diverse team of engineers and product managers to improve our machine learning infrastructure and data quality
- Participate in the design, planning and development of solutions based on business requirements
- Participate as part of a project team to design, code, unit test and implement moderately complex programs to support the portfolio management process
- Monitor and report to management on project status, identify issues that may impede attainment of goals, recommend and implement corrective measures
- Review and fix problems uncovered by testing or customer feedback
- In this role, you will be responsible for learning our various systems and be able configure them in a QA environment for testing purposes, and exercise them with a combination of automation technologies
- Taking guidance from senior engineers in the areas of design and implementation and build upon your skill sets accordingly, and applying the knowledge gained for future work
- Identify issues and escalate suggestions for database improvements (tables, indexes, DAL changes)
- Troubleshoot operational DB issues (isolate problem queries or connections, capture and analyze DB traces)
- This individual will apply attained experiences and knowledge in solving complex problems
Qualifications for big data software engineer
- Motivated, self-starter, creative thinker, ability to work alone or as part of a team
- Bachelor’s degree from an accredited college or university in Computer Science, Computer Engineering, Engineering, or a related field with five years of experience
- Lead and participate in analyzing requirements, develop system architecture and designs which meet requirements
- Design and create automated unit and functional tests for production code
- Lead, coach, and help develop more junior engineers on the team
- Advise management on development estimates, technical information, and issue resolution, on architecture and technology choice points
3
Big Data Software Engineer Job Description
Job Description Example
Our company is looking to fill the role of big data software engineer. If you are looking for an exciting place to work, please take a look at the list of qualifications below.
Responsibilities for big data software engineer
- Familiarity with machine learning concepts
- Work on high-quality architecture and implementation design, collaborating with the customer, system/business analysts and onshore architects
- Integration with Notification service via Email, SMS
- You will work on data solutions that support and enable product and business teams at EAN to make data driven decisions
- You will be part of cross-functional team alongside Engineers, Data Scientists and Data Users
- You will collaborate with upstream system architects, developers and program managers to resolve data issues
- Build scalable and high-performant code
- Mentor other less experienced big data engineers within the team
- Work in a devops environment
- Work in an agile and dynamic environment
Qualifications for big data software engineer
- Team player with drive to grow others around them as much as themselves
- Experience with big data/advanced analytics concepts and algorithms
- Experience with machine-learning algorithms, libraries that assist in the integration of those algorithms (Spark MLlib, Weka, R, Mahout)
- Familarity with collaborative development environments
- MS or PhD in Computer Science, preferably with a focus on Statistical Learning, Applied Statistics, Marketing Science, Operations Research
- Experience with large systems software design and development, with extensive knowledge of UNIX/Linux
4
Big Data Software Engineer Job Description
Job Description Example
Our innovative and growing company is hiring for a big data software engineer. To join our growing team, please review the list of responsibilities and qualifications.
Responsibilities for big data software engineer
- Be an integral member in the design, analysis, development and deployment efforts for multiple enterprise application projects to solve complex business problems
- Leads the QE efforts for Big Data team and takes complete ownership of releases
- Helps in building automation framework and CI/CD
- Creates test plans, designs, develops, and executes reusable and maintainable automated scripts
- Works with developers to analyse requirements, and reproduce problems to get them resolved
- Familiarity with pub/sub messaging systems such as Google Cloud Pub/Sub or Apache Kafka
- Experience as a team lead or other technical leader
- You will be building core data capabilities and services and be passionate about software engineering and data
- You should care deeply about writing solid code
- You closely follow industry trends and the open source community and an avid learner
Qualifications for big data software engineer
- Experience with Storm, Spark, HBase, Cassandra, Kafka, etc a plus
- Ability to work in a fast-changing and dynamic environment - Excellent interpersonal, verbal and written communication skills
- At least 3 years knowledge of statistics and experience using statistical packages for analysing large datasets (Excel, SPSS, SAS, R)
- Bachelors or Advance Degree in Computer Science or like field with 7+ years of experience is required
- Willingness to work in an agile, team development environment
- A programming language
5
Big Data Software Engineer Job Description
Job Description Example
Our innovative and growing company is searching for experienced candidates for the position of big data software engineer. Thank you in advance for taking a look at the list of responsibilities and qualifications. We look forward to reviewing your resume.
Responsibilities for big data software engineer
- Works with developers to analyze requirements, and reproduce problems to get them resolved
- Improve the enterprise systems by integrating existing solutions and building new technology
- Implement software which efficiently analyzes very large scale data sets (hundreds of millions of transactions)
- Manage large data sets in regards to insertions, updates and deletions
- Implement intermediate data layers to facilitate fast reporting results
- Implement software which has a low failure rate
- Implement software which can adapt to changes in coming data
- Troubleshoot issue resources
- Implement software with inherent low maintenance overhead
- Develop software that can be monitored and tracked for failures
Qualifications for big data software engineer
- Excellent knowledge of designing and implementing NoSQL databases as Cassandra or similar
- Experience on database design on Hadoop/NoSQL
- Security implementation of a Hadoop/NoSQL environment
- Design and build scalable platform involving truly cutting edge technology and massive amounts of data
- Research and analyze application behaviors and improving performance and stability
- Work within our global network to optimize applications for linear scaling