Senior Data Engineer Job Description
Senior data engineer
provides hands-on coding with the ability to outline solution design with coding best practices and perform code reviews.
Senior Data Engineer Duties & Responsibilities
To write an effective senior data engineer job description, begin by listing detailed duties, responsibilities and expectations. We have included senior data engineer job description templates that you can modify and use.
Sample responsibilities for this position include:
Collaborate across teams on the design and maintenance of our Operations data mart (ETL, data modeling, metric design, reporting/dashboarding, ) to assure a stable reporting infrastructure
Lead investigations streams of content usage and A/B testing
Take on research projects to improve data processing and any implemented machine learning frameworks used by our team
Be exemplary in promoting Data Engineering best practices within the team
Introduce new skills and knowledge into the team adding to the already dynamic environment our Data Engineering team has created
Share knowledge to compliment team-wide expertise
Develop high quality software at a senior level
Assist with the elicitation and documentation of system requirements
Understand business and technical requirements and constraints to design effective software solutions
Create conceptual architectures and detailed designs for software solutions
Senior Data Engineer Qualifications
Qualifications for a job description may include education, certification, and experience.
Licensing or Certifications for Senior Data Engineer
List any licenses or certifications required by the position: AWS, ETL, BI, RHCE, CDMP, ITIL, GCP, BASE, SAS, CFA
Education for Senior Data Engineer
Typically a job would require a certain level of education.
Employers hiring for the senior data engineer job most commonly would prefer for their future employee to have a relevant degree such as Bachelor's and Master's Degree in Computer Science, Engineering, Technical, Statistics, Math, Business, Design, Mathematics, Management, General Management
Skills for Senior Data Engineer
Desired skills for senior data engineer include:
SQL
Python
Code reviews
Build processes
Operations
Testing
Spark
Message queuing
Stream processing
Coding standards
Desired experience for senior data engineer includes:
3+ years of experience with Stored Procedures, Common Table Expressions, Window Functions, Partitioning and other advanced RDBMS tasks
Solid understanding of ETL concepts and workflows
Willing and interested to work in new areas and across multiple platforms such as MSSQL, Postgresql, Hadoop, Linux
BS or MS in Computer Science, Applied Mathematics, Statistics or area of study related to data sciences and data mining or related field
5 or more years equivalent work in a data engineering, business analytics, business intelligence or comparable data engineering role, including data warehousing and business intelligence tools, techniques and technology
Experience in Big data integration (you have worked with TBs of data before), HiveQL, Hadoop, MapReduce, REST API Denodo or Datameer
Senior Data Engineer Examples
1
Senior Data Engineer Job Description
Job Description Example
Our company is hiring for a senior data engineer. Please review the list of responsibilities and qualifications. While this is our ideal list, we will consider candidates that do not necessarily have all of the qualifications, but have sufficient experience and talent.
Responsibilities for senior data engineer
- Implement new mechanisms for modeling and predicting reader behavior
- Design and implement experiments and models to answer targeted questions
- Lead technical efforts to modernize and replace legacy reporting solutions
- Lead daily activities of globally distributed development team
- Collaborate with Analysts and Business SMEs to perform detailed requirement analysis for enhancing and creating new reporting and analytics solutions
- Imagine and implement creative and innovative reporting solution designs while adhering to architectural requirements of maintainability and scalability
- Individual will be hands-on and contributing to project deliverables
- Drive system problem resolution and root cause analysis
- Ensure and maintain integrity of code base during concurrent development cycles
- Collaborate with experts in a variety of technologies to come up with the best overall solutions
Qualifications for senior data engineer
- A passion for thoroughness, execution and completion
- Ability to communicate in a clear, articulate, customer-friendly manner to all levels of an organization to present findings and recommendations to team members and senior management
- Experience in summarizing analysis results using PowerPoint and explaining analysis results to senior management levels
- Experience in working with virtual groups across the organization and across the world required
- Approachable, personable and team player comfortable working in an Agile environment
- Experience in managing multiple initiatives in parallel
2
Senior Data Engineer Job Description
Job Description Example
Our company is looking for a senior data engineer. If you are looking for an exciting place to work, please take a look at the list of qualifications below.
Responsibilities for senior data engineer
- Performance diagnostics (SQL, Hardware)
- Perform on-call off-hours support for critical systems
- Ability to travel globally when needed
- Help data scientists optimize productionized Hive queries
- Ensure data quality and help in achieving data governance
- Communicate results and impact to business stakeholders
- Collaborate with other teams to tailor data products accordingly
- Responsible for aspects of new and existing data development and data integration using content-based data products
- Builds and maintains data warehouse databases
- Coordinates with other IT departments to ensure implementation and monitoring of databases
Qualifications for senior data engineer
- Broad background with BI environments, tools and development encompassing relational databases, data warehousing and data mining practices
- At least 1 year of professional experience in implementing customer facing Html based Tableau reports
- Experience in implementing the Tableau Reports using Live Access model
- Understanding of Web concepts (html, css, urls, frames ) to the level appropriate to develop Tableau reports in Html according to UX designs
- Experience in configuring Tableau Reporting system in appliance deployment mode for various business needs
- Being able to articulate and communicate to various types of audiences (developers to managers to business owners)
3
Senior Data Engineer Job Description
Job Description Example
Our innovative and growing company is searching for experienced candidates for the position of senior data engineer. We appreciate you taking the time to review the list of qualifications and to apply for the position. If you don’t fill all of the qualifications, you may still be considered depending on your level of experience.
Responsibilities for senior data engineer
- Work with client- and server-side engineers to ingest new data and properly surface for regular consumption
- Develop robust end-to-end data solutions for structured and unstructured data including, but not limited to, ingestion, parsing, integration, auditing, logging, aggregation, normalization, modeling, and error handling
- Collaborate with cross functional team to resolve data quality and operational issues
- Participate in an on-call rotation to support the Information Management cluster
- Interact directly with end users to gather requirements and consult on data integration solutions
- Identify opportunities for team standardization in coding, deployments, documentation and other related areas and create said standards
- Create and maintain key metadata artifacts including, but not limited to, data lineage, data dictionaries, and Entity Relationship Diagrams (ERD)
- Educate the team in emerging related technologies and identify value add opportunities for their implementation
- Develop Interactive Dashboards and supporting Services that will provide customer intelligence insights
- Help developing a high-profile, innovative product that will revolutionize the way how businesses attribute success to their marketing tech stacks
Qualifications for senior data engineer
- Must be effective in working both independently and in a customer setting
- At least 4 years of hands-on experience working with large data sets
- Minimum of a BS in a quantitative discipline such as Statistics, Engineering, Computer Science, Applied Mathematics
- Bachelor’s Degree in Computer Science, Mathematics or Machine Learning
- 6 years experience with C#/VB, HTML/CSS, JavaScript or C++
- Familiarity with reporting or BI tools
4
Senior Data Engineer Job Description
Job Description Example
Our company is looking to fill the role of senior data engineer. To join our growing team, please review the list of responsibilities and qualifications.
Responsibilities for senior data engineer
- Designing scalable implementations of the models
- Hands-on programming based on TDD, usually in a pair programming environment
- Advising clients on the usage of different distributed storage and computing technologies from the plethora of options available in the ecosystem
- Support ongoing data Integration efforts for all development and production environments including Big Data Design Patterns
- Design and develop applications for all data warehousing components, including Real-Time Data Ingestion techniques, Transformations, aggregations, and related data quality strategy
- Design and implementation of multi-source data channels and ETL processes
- Design, enhance and implement ETL/data ingestion platform on the cloud
- Clear documentation of process flow diagrams and best practices
- Designing and implementing solutions to provide stakeholders self-service access to timely, accurate and actionable data that serve their distinct use-cases
- Improve the efficiency, scalability, and stability of our Vertica and Hadoop data warehouse
Qualifications for senior data engineer
- A Bachelor’s degree and a minimum of 5 years’ relevant experience as a data engineer
- Work as hands-on data modeler and ad-hoc data analyst when needed
- Work with DBAs and DevOps to ensure proper scaling and monitoring of various data stores
- At least 2 years of experience in C, Java, Go, Scala, or Python
- 2+ years of experience working with Cloud platforms and services such as AWS
- 2+ years of experience working with automated build and continuous integration systems such as Ansible, Terraform , Chef and Jenkins
5
Senior Data Engineer Job Description
Job Description Example
Our company is growing rapidly and is searching for experienced candidates for the position of senior data engineer. To join our growing team, please review the list of responsibilities and qualifications.
Responsibilities for senior data engineer
- Design and develop ETL (extract-transform-load) processes to validate and transform data, calculate metrics and attributes, populate data models , using HADOOP, Spark, SQL, and other technologies
- Lead by example, demonstrating best practices for code development and optimization, unit testing, CI/CD, performance testing, capacity planning, documentation, monitoring, alerting, and incident response in order to ensure data availability, data quality, usability and required performance
- Use programming languages such as SAS, R, Python, and SQL to create automated processes for data gathering, cleansing, reporting, and visualization
- Implement systems for tracking data quality, usage, and consistency
- Design and develop new data products, using languages, such as Ruby
- Monitor and maintain system health and security
- Oversee administration and improvements to source control and deployment process
- Prepare unit tests for all work to be released to our live environment (including data validation scripts for data sets releases or changes)
- Implement performance tuning on the databases based on monitoring
- Design and implement data products using Hadoop technologies
Qualifications for senior data engineer
- Hands on deployment experience with Hadoop/Spark, Scala, MySQL, Redshift and Amazon AWS or other cloud base systems
- Comfortable writing code in python, ruby, perl, or equivalent scripting language
- Experience with Cosmos/Scope, SQL, or Hadoop
- At least 3 years of professional work experience programming in Python, Java or Scala
- Master's Degree in Computer Science, Computer Engineering, Data Science or related discipline
- 2+ years of Distributed Computing frameworks such as Apache Spark, Hadoop