This job has expired, please see additional jobs below
Big Data Engineer
Farfetch
Porto, , Portugal
Job Details - this job has expired, please see similar jobs below
Description
Responsibilities:
• Design and develop highly scalable, end to end process to consume, integrate and analyze large volume, complex data from sources such as Hive, Flume, Kafka or Storm.
• Provide Data Engineering expertise to multiple teams across our organization. Provide guidance and support to software engineers with industry and internal data best practices.
• Build fault tolerant, adaptive and highly accurate data computational pipelines. Tune queries running over billion of rows of data running in a distributed query engine.
• Researching and implemented new data technologies as needed.
• Working with other teams to understand needs and provide solutions.
• Find innovative solutions through a combination of creative thinking and deep understanding of the problem space.
• Work with the Business Intelligence development team on migration and improve existing SQL Server-based ETLs to Map Reduce and Hive (Cloud) technology to achieve scale and performance.
• Help define and implement new processes on the data warehouse platform and work closely with Data Scientists to transform big data into model-‐ ready forms to support analytic projects.
Key skills and experience:
• Experience working with large data sets (both structured and unstructured) using technologies such as MapReduce, Hadoop, HBase, Hive, Spark and NoSQL technologies
• Strong programming background with languages such as Java, C++, or Python
• Good knowledge of distributed systems
• Background working in cloud environments – AWS, Rackspace, Azure, etc.
• Experience with real-time analysis of sensor and other data from Internet of Things (IoTs) or other connected devices is a plus
• Excellent grasp of algorithmic concepts in computer science (e.g., sorting, data structures, etc.)
• Experience in the design, development and release of enterprise scale applications.
• Experience with version control.
• Excellent team working attitude with analytical and creative problem solving abilities.