Specialist Software Engineer - Python & Databricks

Develop, optimize, and maintain scalable data pipelines using Python and Apache Spark on Databricks to process large volumes of data from multiple sources.
Design and implement ETL/ELT workflows to ingest, cleanse, transform, and aggregate metrics according to business requirements.
Collaborate with data scientists, analysts, and business stakeholders to understand data needs and translate them into technical solutions.
Implement data quality checks, validation rules, and monitoring frameworks to ensure accuracy and reliability of metrics.
Build and maintain automated reporting and dashboarding solutions leveraging Databricks and associated BI tools.
Manage version control, code reviews, and CI/CD pipelines to ensure robust and maintainable codebase.
Troubleshoot and resolve performance issues and optimize Spark jobs for cost and speed efficiency.
Stay current with regulatory requirements and integrate relevant data sources and compliance checks into the metrics factory.
Ability to work in agile mode in complex contexts, with a sense of autonomy and sharing of best practices within a team.
Ability to work with users to translate functional requirements into a technical solution.
Business analyst support for functional testing and diagnostics.

4+ Years of Python programming experience in data manipulation libraries (Pandas, NumPy).
Strong experience with Apache Spark (PySpark) on Databricks platform (job optimization, partitioning, memory management, Spark Structured Streaming)
Databricks expertise in Delta Lake, Unity Catalog, Databricks Workflows, Cluster Management
Expertise in building ETL/ELT pipelines and data workflows in Databricks.
Strong Python experience applied to data using frameworks like PySpark, pandas, polars, ETL/ELT pipeline management, code best practices (clean code, unit tests, packaging)
Experience in data architecture design and implementation (Lakehouse, Medallion architecture, Batch & Streaming)
Familiarity with SQL and NoSQL databases for data storage and retrieval.
Experience with cloud platforms (preferably Azure or AWS) and data lake architectures.
Strong experience in version control systems (Git) and CI/CD tools preferrably Jenkins (Azure DevOps will be good to have).
Experience with data quality frameworks and monitoring tools.
Experience with containerization (Docker) and orchestration (Kubernetes).
Strong analytical skills and ability to work in an agile development environment.
Excellent analytical, communication and collaboration skills.

“We are committed to creating a diverse environment and are proud to be an equal opportunity employer. All qualified applicants receive consideration for employment without regard to race, color, religion, gender, gender identity or expression, sexual orientation, national origin, genetics, disability, age, or veteran status”

At Société Générale, we are convinced that people are drivers of change, and that the world of tomorrow will be shaped by all their initiatives, from the smallest to the most ambitious.

Whether you’re joining us for a period of months, years or your entire career, together we can have a positive impact on the future. Creating, daring, innovating and taking action are part of our DNA.

If you too want to be directly involved, grow in a stimulating and caring environment, feel useful on a daily basis and develop or strengthen your expertise, you will feel right at home with us!

Still hesitating?

You should know that our employees can dedicate several days per year to solidarity actions during their working hours, including sponsoring people struggling with their orientation or professional integration, participating in the financial education of young apprentices and sharing their skills with charities. There are many ways to get involved.

We are committed to support accelerating our Group’s ESG strategy by implementing ESG principles in all our activities and policies. They are translated in our business activity (ESG assessment, reporting, project management or IT activities), our work environment and in our responsible practices for environment protection.