Staff Data Engineer Job Description
Staff data engineer
provides operational and technical expertise to design, build, upgrade and maintain the Big Data analytics platform, data and applications, addressing a broad range of technologies including Big Data/Hadoop, Spark, Kafka, HBase, Cassandra, Elastic Search on Linux hosted on the Azure or AWS cloud.
Staff Data Engineer Duties & Responsibilities
To write an effective staff data engineer job description, begin by listing detailed duties, responsibilities and expectations. We have included staff data engineer job description templates that you can modify and use.
Sample responsibilities for this position include:
Employ the established Data Governance model including people, process and technology to sustain Data Quality for the data objects and implement the necessary operating mechanisms to ensure compliance
Establish metadata management, data catalogs, data standards & best practices for the data objects working closely with enterprise architects
Build and support scalable and durable data solutions that can enable self-service advanced analytics atHomeAway using both traditional (SQL server) and modern DW technologies (Hadoop,Spark, Cloud, NoSQL ) in an agile manner
Design and develop HADOOP Data Pipeline and Vertica ETL processes to validate and transform data, calculate metrics and attributes, populate the dimensional model
Build scalable big-data distributed data processing systems
Build fault tolerant, self-healing, adaptive and highly accurate data computational and analytic processing applications
Opportunity to work closely with our data scientists, business partners
Facilitate the gathering of data requirements (processes & attributes) across different functional teams & driving process improvements & simplification
Work with cross-functional team to perform process & data analysis, identifying systems of record (SoR) and/or Single Point of Truth (SPOT), defining the Power Plant Configuration solutions & application architecture and deployment
Develop and configure Hadoop/ Big Data components
Staff Data Engineer Qualifications
Qualifications for a job description may include education, certification, and experience.
Licensing or Certifications for Staff Data Engineer
List any licenses or certifications required by the position: ITIL, AWS, CISA, CISSP, ASQ, NIST, CCIE
Education for Staff Data Engineer
Typically a job would require a certain level of education.
Employers hiring for the staff data engineer job most commonly would prefer for their future employee to have a relevant degree such as Bachelor's and Master's Degree in Computer Science, Engineering, Science, Information Technology, Mathematics, Technical, Statistics, Technology, Business, Computer Engineering
Skills for Staff Data Engineer
Desired skills for staff data engineer include:
Database concepts and UNIX/LINUX
Software development life cycle
Spark
Cassandra
Data
Enacting service level agreements and the appropriate escalation and communication plans to maintain them
Infrastructure
Java
Jira and Confluence is desired
Technologies and data engineering experience
Desired experience for staff data engineer includes:
Analyze and develop new applications using Hadoop technologies – Hive/Pig/MapReduce
5-10 years of hands on experience in Linux architecture and tools
Excellent Shell scripting skill
Experienced in design/development/implementation using large datasets
Experience with Zookeeper, Scribe or Flume, Mahout
Skills with Visualization tools will be highly preferred
Staff Data Engineer Examples
1
Staff Data Engineer Job Description
Job Description Example
Our company is searching for experienced candidates for the position of staff data engineer. If you are looking for an exciting place to work, please take a look at the list of qualifications below.
Responsibilities for staff data engineer
- Design, develop, and operate highly-scalable, high-performance, low-cost, and accurate data pipelines in distributed data processing platforms
- Work in the Risk Data Analytics data engineering team
- Help with building big data deployment infrastructure
- Expert in building Data Sources that can scale to handle huge data volume
- Design, code, test, integrate, and maintain the Test Data Supply application suite
- Work in the Small Business Group (SBG) Data Engineering team
- Design the next generation distributed networking system from data
- Enthusiastic and principled about finding and applying the best practices related to dimension modelling and Big data analytics
- Work on ETL and high volume real time data pipeline
- Expert in fine tuning data sources/dashboards/reports for performance
Qualifications for staff data engineer
- Significant (at least 2 year) experience in Java and with overall Hadoop eco system (HDFS, Map Reduce, Pig, Hive, etc…)
- Sound knowledge in ETL and Data Warehouse concepts
- Experience in relational database concepts and database architecture and design
- Expertise in Predictive Analytics
- Experience with cloud-based platforms
- Linux systems administration, managing execution of services, shell scripting, networking
2
Staff Data Engineer Job Description
Job Description Example
Our innovative and growing company is hiring for a staff data engineer. If you are looking for an exciting place to work, please take a look at the list of qualifications below.
Responsibilities for staff data engineer
- Actively participate and influence/drive BI (Tableau) infrastructure/architecture/design setup/changes
- Works closely with functional teams and individuals to design and create dashboards/reports to ensure accurate data and acceptable performance
- Configure, Implement and provide support around our Data Protection initiatives
- Create Data Protection Cloud Patterns with AWS
- Create Policies for Data Protection to be used across the company
- Enable and support Company wide Cloud Data Protection
- Design Data Protection solutions Using existing backup technologies for scalable applications
- Monitor Health of Data Protection Environment
- Troubleshoot issues pertaining to data protection success and performance
- Design and build scalable data models for large analytics datasets in AWS
Qualifications for staff data engineer
- We're partnering with top internet retailers to create new products that bring personalized shopping to the next level
- We're collaborating with top industry partners (Facebook, Google, Yahoo, ) to build out the next generation of programmatic advertising infrastructure
- Expertise with full-stack development with a strong focus on designing and implementing simple, elegant solutions
- Ability to mentor others and thrive in an unstructured and dynamic environment
- Deep knowledge of software development process and expierence with tools in version control as git
- Create and support applications running in a distributed processing environment (Cloud Foundry)
3
Staff Data Engineer Job Description
Job Description Example
Our company is looking to fill the role of staff data engineer. We appreciate you taking the time to review the list of qualifications and to apply for the position. If you don’t fill all of the qualifications, you may still be considered depending on your level of experience.
Responsibilities for staff data engineer
- Transform data into actionable information by data analysis and Statistical/Analysis systems from requests
- Collaborate with cross functional team in analytical methods development to enable new critical process, parametric, yield monitoring, triggering capability, events prediction and system development for quick data-info-action transformation
- To improve root cause discovery methods/techniques to turn data into actionable information as quick as possible with acceptable level of accuracy of root cause identification
- Design and implement cloud foundry based solutions for various domains
- Participate in API-based integrations and back-end based loosely coupled architecture and pipeline
- Provide engineering support to other teams as needed
- Experience developing software against 3rd party APIs / RESTful web services and working with vendors to identify and correct issues and drive enhancements
- Participate in an Agile SDLC to deliver new cloud platform services and components
- Create and support applications running in a distributed processing environment
- Machine learning algorithm assessment, selection and enhancement for the problems being addressed
Qualifications for staff data engineer
- 8+ years of large scale distributed software design and development experience
- Proficiency in Java/C++ and SQL/RDBMS
- Solid foundation of data structure and algorithm
- Experience in developing hosted applications and RESTful services
- Good communication abilities and capacity for teamwork
- Knowledge of multi-core CPU architectures is a big plus
4
Staff Data Engineer Job Description
Job Description Example
Our innovative and growing company is hiring for a staff data engineer. Thank you in advance for taking a look at the list of responsibilities and qualifications. We look forward to reviewing your resume.
Responsibilities for staff data engineer
- You will collaborate deeply with integration and information architects to define a variety of data ingestion pipelines and promote a data driven culture by providing access to real time data and improve business outcomes
- Provide vision for future technology for the Data Platform
- Leadership and mentoring for our engineering team as we continue to rapidly evolve the Data Platform
- You will quickly assess existing platforms and define a data platform strategy and build a roadmap for execution
- Key member of the Enterprise Architecture group recommending appropriate data platform choices for business domains
- You will evolve the architecture to support a wide spectrum of data needs ranging from analytical (descriptive and diagnostic), human-centric (predictive and prescriptive) to machine centric (decisions and actions)
- As a senior technical leader, you will mentor and guide a global team of data engineers
- Partner closely with teams to produce an application that flows seamlessly and works intuitively with our clients
- Work with architecture team on research & analyses of newer technologies for the next generation of ENT applications
- Be comfortable leading meetings and discussions with non-technical teams
Qualifications for staff data engineer
- Experience working with Broadcom XGS network switching hardware and SDKs is a big plus
- A minimum of BS in CS/EE, MS degree is preferable
- Minimum 3 years of experience in research and development of RDBMS or NoSQL or OS related areas (file system, memory management, IO, or networking) or cloud related areas (cloud service architecture, virtualization, cloud storage architecture, or resource management) with Master in CS or a relevant field, or 5 years of experience with a Bachelor degree
- Knowledge in database storage subsystem, parallel query processing, query optimization, distributed transaction processing, workload management, or high availability features
- Experience with PostgreSQL kernel a plus
- Candidates should have at least 1 year of tools development experience for a telecom vendor, service provider or a 3rd party analytics tools company
5
Staff Data Engineer Job Description
Job Description Example
Our company is looking to fill the role of staff data engineer. Please review the list of responsibilities and qualifications. While this is our ideal list, we will consider candidates that do not necessarily have all of the qualifications, but have sufficient experience and talent.
Responsibilities for staff data engineer
- Integrate domain data knowledge into development of data and automation requirements
- Actively partner up with existing data science team members to identify key issues and craft your own solutions based off your knowledge and understanding of the tech stack and tool set
- Work in the CTO-Dev Development analytics platform team, provide end-to-end solution for engineering productivity analytics platform, while leveraging big data tools and technologies
- Work with Data Scientists to identify datasets needed for deep customer insights and for building operational propensity models
- Maintain and build ingestion pipelines to bring required source datasets into the data warehouse
- Define SLAs for data availability and correctness, respond to alerts when SLAs are not being met
- Create profile and scorecards on customer & contact data within various source system
- Build efficient duplication strategies for master data using Informatica MDM Hub
- Design, build and launch new data extraction, transformation and automated remediation in production
- Create Backup Governance and Compliance Tooling / Automation and Reporting
Qualifications for staff data engineer
- A plus to be familiar with architectural components involved in the support of 3G/4G Mobile radio access networks/core, OAM, KPI
- 6+ years of professionalJava/C# software development experience
- Developing applications on Big Data / streaming platforms
- Modify & maintain code written in Java, Python, R
- Experience with different tools for massive data set analysis
- Experience with testing analytics/reporting on processed data