This job has expired, please see additional jobs below
Senior Data Engineer
Foursquare
New York, NY, United States
Job Details - this job has expired, please see similar jobs below
Since our inception in 2009, Foursquare has been a leading force in changing how location information enriches our real-world and digital lives. As a location intelligence company, Foursquare is comprised of two well-known consumer apps, Foursquare and Swarm, as well as thriving media and enterprise products. Our B2B offerings include Places (for developers), Pinpoint and Attribution (for marketers), and Place Insights (for analysts, based on the world's largest foot traffic panel). With more than 200 people across our offices in New York, San Francisco, and in sales offices around the globe, we’re dedicated to our trailblazing mission—enriching consumer experiences and informing business decisions with location intelligence.
About Foursquare’s Enterprise Team:
Foursquare’s Enterprise Engineering team takes on the challenges of working with the full spectrum of data produced by our consumer apps. We build tools and products that allow employees and clients to get rich and meaningful insights about the real world. As a Senior Data Engineer on the Enterprise Engineering team, you will be responsible for creating and maintaining the offline pipeline that wrangles our raw data into SaaS products.
We are looking for a candidate who is ready to bring new ideas to the table; who loves to dig into the data to find the source of a problem or validate an assumption; and someone who can quickly understand, discuss, and optimize the performance characteristics of a complex offline data pipeline. Working with technologies such as Hadoop, Scalding, Luigi, Spark, Mongo and more, and experience in these or other related technologies is a must.
Responsibilities
◦ Maintain and improve our data pipelines using Hadoop, Scalding, Luigi, Spark, Mongo and more
◦ Partner with the Data Science team to investigate and implement advanced statistical models and machine learning pipelines
◦ Identify and implement performance improvements across all pipelines
◦ Data investigations to validate assumptions or find the source of a problem
Qualifications
◦ 3+ years of proven experience working with Hadoop MapReduce and/or other big data technologies and pipelines
◦ You consider yourself both a Data Scientist and a Senior Developer, and are just as happy working on challenging data problems as you are tinkering with the clusterYou have a solid foundation in computer science fundamentals with particular expertise in data structures, algorithms, and design
◦ You obsess over data: everything needs to be accounted for and be thoroughly tested
◦ You are constantly thinking of ways to squeeze better performance out of the pipelines
◦ Strong Java or other object-oriented programming experience or, even better, experience and/or interest in functional languages (we use Scala!)
◦ Bonus points for experience with Scala, Scalding, Luigi,Hive, machine learning pipelines and model training