This job has expired, please see additional jobs below
Arity - Senior Data Flow Administrator
Allstate
Chicago, IL, United States
Job Details - this job has expired, please see similar jobs below
Founded by The Allstate Corporation in 2016, Arity is a data and analytics company focused on improving transportation. We collect and analyze enormous amounts of data, using predictive analytics to build solutions with a single goal in mind: to make transportation smarter, safer and more useful for everyone.
At the heart of that mission are the people that work here—the dreamers, doers and difference-makers that call this place home. As part of that team, your work will showcase both your intelligence and your creativity as you tackle real problems and put your talents towards transforming transportation.
That’s because at Arity, we believe work and life shouldn’t be at odds with one another. After all, we know that your unique qualities give you a unique perspective. We don’t just want you to see yourself here. We want you to be yourself here.
The Role
As a Sr. Data Flow Administrator, you will be responsible for assisting with the design, architecture, implementation, capacity planning, performance tuning and on-going support of Arity’s HDF/NiFi infrastructure on AWS.
Key Responsibilities
• Manage large scale multi-nodes HDF/NiFi cluster environments residing on AWS
• Handle all HDF/NiFi environment builds, including design, capacity planning, cluster setup, performance tuning and ongoing monitoring
• Perform high-level, day-to-day operational maintenance, support, and upgrades for the HDF Clusters
• Creation of key performance metrics, measuring the utilization, performance and overall health of the clusters
• Capacity planning and implementation of new/upgraded hardware and software releases for the HDF and NiFi infrastructure
• Research and recommend innovative, and where possible, automated approaches for system administration tasks
• Ability to closely calibrate with product owners, product managers and lead engineers and testers
• Provide guidance in the creation and modification of standards and procedures, best practices, high availability and and disaster recovery
• Proactively monitor and setup alerting mechanism for HDF/NiFi Clusters and supporting hardware to ensure system health and maximum availability
Technical Experience and Skills
• Strong working knowledge of Hortonworks Data Flow (HDF) Architecture, Administration, Features and Characteristics, HDF Security
• Experience working with Python or other scripting language, and a good understanding of AWS (Cloud Formation Templates, EC2, Cloudwatch, S3)
• 2-4 years of solid experience designing, building and deploying HDF environment with Ambari
• Expereince building HDF clusters, deploying Ambari, Zookeeper, Kafka, Schema Registry to HDF environment using Cloud Formation Templates
• Strong knowledge in writing shell scripts and Ansible playbooks, and to create Jenkins pipelines for HDF cluster deployments
• Experience creating HDF blueprints and Cluster files to setup HDF Clusters, and knowledge of CLI tools and JMX settings
• Experience in integrating LDAP authentication for Ambari, securing Zookeeper and configuring Ranger on HDF clusters
• Knowledge of creating HDF NiFi Blueprints, setting up NiFi and Linux tuning and exposure to NiFi APIs
• Strong knowledge and experience working with Nifi User Interface, monitoring and managing critical 24/7 applications, optimizing the data flows, and troubleshooting HDF/NiFi infrastructure issues
• Experience designing, building, and troubleshooting end to end data flow pipelines (data ingestion, data routing, data transformation and data persistence) using NiFi
• Strong Knowledge with Hadoop Technologies, Apache Kafka and Cassandra
Qualifications
• Bachelor’s Degree in Computer Science or a related discipline, or equivalent experience/combined education, with some Relevant professional experience and/or specialized training that is commensurate with the assignment
• 2-4 years of experience working with HDF and NiFi environments
• Proven ability to lead multiple high priorities initiative with aggressive timelines leveraging an agile framework
• Comfortable performing in a fast paced, dynamic and ambiguous business environment
• Ability to concentrate on a wide range of loosely defined complex situations, which require creativity and originality, where guidance and counsel may be unavailable
• Excellent listening and communication skills
• Strong problem solving skills with the ability to offer why one technical solutions is more preferable than another
• Ability to work in a team environment in a collaborative manner and remote teams
• Proven track record of sound, effective decision making
The candidate(s) offered this position will be required to submit to a background investigation, which includes a drug screen.
That’s the day-to-day, now let’s talk about the rest of it. As we mentioned, Arity was founded by The Allstate Corporation. But you’ll be working for—and at—Arity. It’s the best of both worlds. You’ll get access to the full suite of Allstate benefits and work in a fast-paced startup culture. That’s more than just free breakfasts, brain breaks and ping pong. It’s a culture that encourages you to be you.
Sound like a fit? Apply now! We can’t wait to meet you.
Arity.com Instagram Twitter LinkedIn
Allstate generally does not sponsor individuals for employment-based visas for this
position.
Effective July 1, 2014, under Indiana House Enrolled Act (HEA) 1242, it is against public policy of the State of Indiana and a discriminatory practice for an employer to discriminate against a prospective employee on the basis of status as a veteran by refusing to employ an applicant on the basis that they are a veteran of the armed forces of the United States, a member of the Indiana National Guard or a member of a reserve component.
For jobs in San Francisco, please click "here" for information regarding the San Francisco Fair Chance Ordinance.
For jobs in Los Angeles, please click "here" for information regarding the Los Angeles Fair Chance Initiative for Hiring Ordinance.
It is the policy of Allstate to employ the best qualified individuals available for all jobs without regard to race, color, religion, sex, age, national origin, sexual orientation, gender identity/gender expression, disability, and citizenship status as a veteran with a disability or veteran of the Vietnam Era.