This job has expired, please see additional jobs below
Data Architect (Big Data)
Entertainment & Media Industry Company
Montreal, , Canada
Job Details - this job has expired, please see similar jobs below
Summary:
The Data Architect is part of the Enterprise data team and is responsible for the Data Management in the Bigdata platform Datalake. The Datalake team aims to centralize all Company raw data in the Datalake in near Real-time and Batch and prepare for various usage (Data warehouse, Data Science and data exploration).The incumbent will analyse the data sources, recommend technical solution, develop and maintain ingestion processes in the Datalake and organize the data efficiently in Datalake (Hadoop). They will also interact with Technical Leads, Programmers, BI developers, Data Scientists, Security Team, IT (GNS), TG, etc., to safeguard the Data is proper, optimized and scalable for various usage.
Responsibilities:
The main and routine tasks of the Data Architect are to:
• Participate in the design, architecture and evolutions of the Datalake;
• Analyze the data sources and perform data exploration;
• Develop and maintain real-time data ingestion pipeline;
• Develop and maintain Batch ingestion processes;
• Develop and maintain Datalake through HDFS, Hive, Impala data structures;
• Develop new benchmarks and tools for Data performance measurement and capacity;
• Analyse and recommend best technologies available for services accompanied with the projects;
• Review, recommend and approve Data management;
• Apply and contribute on development best practices;
• Provide technical mentoring to the other team members;
• Perform deep unitary testing;
• Write technical and process documentation;
Qualifications
Training:
• Degree in computer sciences or certificate with equivalent experience.
Relevant Experience:
• Minimum 7 year experience in the industry with a minimum of 4 years in this role.
Skills and Knowledge:
• Excellent Knowledge and experience of programming Java;
• Excellent knowledge in Distributed Streaming Platform Apache Kafka and it’s advanced features;
• Excellent knowledge in stream processing with Storm or Spark Streaming or similar technology;
• Excellent knowledge and experience of Cloudera Bigdata technologies (Hbase, HDFS, Hive, Impala);
• Good Knowledge and experience of programming C# and Python;
• Good knowledge and experience Linux Server administration;
• Good knowledge of algorithm analysis and optimization;
• Good in collaboration & communication;
• Knowledge on ETL tool is an asset;
• Knowledge on other emerging open source project is an asset (like Druid etc);
• Capacity to work under pressure and solve multifaceted problems;
• English fluent, French is an asset.