Specialist Software Engineer - Bigdata

We are seeking an experienced Big Data Lead Developer to lead our data engineering efforts. In this role, you will design, develop, and maintain large-scale data processing systems. You will work with cutting-edge technologies to deliver high-quality solutions for data ingestion, storage, processing, and analytics. Your expertise will be critical in driving our data strategy and ensuring the reliability and scalability of our big data infrastructure.

  3 to 8 years of experience on application development with Spark/Scala

·       Good hands-on experience of working on the Hadoop Eco-system ( HDFS, Hive, Spark )

·       Good understanding of the Hadoop File Formats

·       Good Expertise on Hive / HDFS, PySpark, Spark, JupiterNotebook, ELT Talend, Control-M, Unix/Script, Python, CI/CD, Git / Jira, Hadoop, TOM, Oozie, Snowflake

·       Expertise in the implementation of  the Data Quality Controls

·       Ability to interpret the Spark UI and identify the bottlenecks in the  Spark process and provide the optimal solution.

Tools

·       Ability to learn and work with various tools such as IntelliJ, GIT, Control M, Sonar Qube and also on board the new frameworks into the project.

·       Should be able to independently handle the projects.

Agile

·       Good to have exposure to CI/CD processes

·       Exposure to Agile methodology and processes

Others

·       Ability to understand complex business rules and translate into technical specifications/design.

·       Write highly efficient and optimized code which is easily scalable.

·       Adherence to coding, quality and security standards.

·       Effective verbal and written communication to work closely with all the stakeholders

·       Should be able to convince the stakeholders on the proposed solutions

BIG DATA

Skills

Description

MANDATORY

Spark Scala

Proficiency in coding and transforming raw requirements into technical solution.

Hadoop

Experience with HDFS infrastructure and platform. Proficiency in the HDFS commands

SQL

Strong skills in data analysis, including querying and data retrieval.

GitHub

Expertise in code management throughout the entire development lifecycle.

GOOD TO HAVE

Hive

Familiarity with this tool for data viewing and computation.

SonarQube

Knowledge of code quality monitoring practices.

Shell Scripting / Python

Ability to utilize these languages to execute Spark Scala code within a big data environment.

We are committed to creating a diverse environment and are proud to be an equal opportunity employer. All qualified applicants receive consideration for employment without regard to race, color, religion, gender, gender identity or expression, sexual orientation, national origin, genetics, disability, age, or veteran status

At Société Générale, we are convinced that people are drivers of change, and that the world of tomorrow will be shaped by all their initiatives, from the smallest to the most ambitious. Whether you’re joining us for a period of months, years or your entire career, together we can have a positive impact on the future. Creating, daring, innovating, and taking action are part of our DNA. If you too want to be directly involved, grow in a stimulating and caring environment, feel useful on a daily basis and develop or strengthen your expertise, you will feel right at home with us!

Still hesitating?
You should know that our employees can dedicate several days per year to solidarity actions during their working hours, including sponsoring people struggling with their orientation or professional integration, participating in the financial education of young apprentices, and sharing their skills with charities. There are many ways to get involved.

We are committed to support accelerating our Group’s ESG strategy by implementing ESG principles in all our activities and policies. They are translated in our business activity (ESG assessment, reporting, project management or IT activities), our work environment and in our responsible practices for environment protection.