This job has expired, please see additional jobs below
Big Data Platform Engineer - Data Technologies
Entertainment & Media Industry Company
New York, NY, United States
Job Details - this job has expired, please see similar jobs below
Company runs on data. It’s our business and our product. It’s why thousands of companies partner with us. We're nearing one petabyte and growing, with no end in sight. Our data captures who, what, when, where and why our clients use Company products.
The Company Big Data Services engineering team (or BBDS for short) provides a software platform for hosting large datasets. It’s a mature platform complete with search, analytics and real-time pipeline processing capabilities. The system scales out to petabytes while maintaining low latency, availability and immediate discoverability by clients. This puts us in an enviable and unique position to address the unique challenges of financial markets.
Maybe you love solving intricate engineering issues with large systems. Or you like to dig into interesting problems around platform APIs, real-time data pipelines, search and analytics engines or query optimizers. If this sounds like you, keep reading!
We’ll trust you to:
• Build large distributed systems that will be the heart of our data platform. Your work will enable us to ingest and process trillions of data items
• Provide search and analytics across these structured, semi-structured and unstructured datasets
• Work on a number of large distributed computing systems such as HBase, MySQL Clusters, Kafka, Spark, Lucene, Solr/Elastic, HAProxy and in-memory stream processors
• Adapt and change many technologies to provide solutions for distributed data storage
• Care about synchronization, sub-second latencies, search and discoverability
• Maintain fault tolerance and high availability
You’ll need to have:
• 5+ years experience in Java and JVM, C and Linux system including expertise in low-latency kernel level optimizations
• A background in software engineering and the capability to program in compiled and dynamic languages such as Python and JavaScript
• Expertise in data stores (both transactional and non-transactional) as well as the ability to code in a highly concurrent environment
We’d love to see:
• Experience with distributed systems, RESTful architectures and scalable, low-latency systems that provide high availability
• Deep knowledge of HBase, Spark, Cassandra and the Hadoop ecosystem of technologies or MySQL/WebScaleSQL and InnoDB engines
• A Bachelor’s or Master’s degree in Computer Science, Computer Engineering, Science, Math or equivalent experience
• Deep knowledge of search engines like Apache Lucene and Solr/Elasticsearch