Lead Software Engineer - Databricks (Mosaic AI)

MSA Role: Lead Software Engineer – Databricks (Mosaic AI)
Experience: 8–11 Years
• Lead design and development of GenAI‑enabled data and AI solutions using Databricks, Mosaic AI, LLMs, RAG pipelines, and AI agents.
• Own end‑to‑end architecture and implementation of scalable, production‑grade data pipelines using Databricks (PySpark, SQL, Delta Lake).
• Design, implement, and govern Delta Live Tables (DLT) pipelines for trusted, auditable, and resilient data processing.
• Define and enforce Unity Catalog–based data and AI governance, including catalogs, schemas, access control, lineage, and ownership.
• Architect and manage datasets, features, and views to support LLM inference, embeddings, RAG, and AI agent workflows.
• Design and integrate LLM‑based capabilities (chat, Q&A, summarization, automation) using Azure OpenAI / OpenAI.
• Lead usage of vector databases and embeddings to build scalable search, retrieval, and RAG solutions.
• Drive experiment tracking, lineage, and lifecycle management using MLflow integrated with Unity Catalog.
• Ensure strong data quality, observability, performance, reliability, and cost optimization across Databricks workloads.
• Establish monitoring, alerting, and operational best practices for data and AI pipelines.
• Collaborate closely with architects, data scientists, product owners, and business stakeholders to shape GenAI solution design.
• Translate business objectives into robust Databricks and Mosaic AI technical solutions.
• Own production stability, incident response, root‑cause analysis, and continuous improvement.
• Lead code reviews and enforce Databricks, Spark, and GenAI engineering best practices.
• Mentor Specialist and Junior engineers; provide technical direction and hands‑on guidance.
• Plan, estimate, and deliver work aligned with Agile delivery practices and sprint goals.
• Support global collaboration and be flexible to align with Paris business hours when required.
• Own delivery of GenAI features and data capabilities end‑to‑end, from ingestion to business consumption.

Profile Required:
Senior‑level Software Engineer with deep Databricks, Mosaic AI, and GenAI data platform expertise, capable of designing and leading enterprise‑scale AI‑enabled data solutions.

Minimum Qualifications:

• Deep hands‑on experience with Databricks (PySpark, SQL, Delta Lake) in large‑scale production environments.
• Proven experience designing and leading end‑to‑end data pipelines on Databricks.
• Strong hands‑on experience with Delta Live Tables (DLT) for governed data processing.
• Advanced experience with Unity Catalog for data governance, access control, lineage, and metadata.
• Proven experience delivering GenAI‑enabled data solutions, including LLM integration and RAG pipelines.
• Strong experience with MLFlow for experiment tracking, lineage, and lifecycle management.
• Hands‑on experience with vector databases and embedding workflows.
• Expertise in Spark performance tuning, scalability, and cost optimization.
• Experience managing Databricks workflows, monitoring, and production operations.
• Strong knowledge of cloud‑based data platforms (AWS preferred).
• Proven ability to mentor engineers and lead technical delivery.

Preferred Qualifications:

• Hands‑on experience with Mosaic AI platform capabilities.
• Experience building and operating AI agents for workflow automation and decision support.
• Strong understanding of data observability, platform reliability, and FinOps practices in Databricks.
• Experience working in regulated or financial services domains.
• Exposure to enterprise‑scale AI governance, compliance, and audit requirements.
• Ability to influence architecture decisions and align stakeholders.

• Excellent communication skills with both technical and non‑technical audiences.

“We are committed to creating a diverse environment and are proud to be an equal opportunity employer. All qualified applicants receive consideration for employment without regard to race, color, religion, gender, gender identity or expression, sexual orientation, national origin, genetics, disability, age, or veteran status”

At Société Générale, we are convinced that people are drivers of change, and that the world of tomorrow will be shaped by all their initiatives, from the smallest to the most ambitious.

Whether you’re joining us for a period of months, years or your entire career, together we can have a positive impact on the future. Creating, daring, innovating and taking action are part of our DNA.

If you too want to be directly involved, grow in a stimulating and caring environment, feel useful on a daily basis and develop or strengthen your expertise, you will feel right at home with us!

Still hesitating?

You should know that our employees can dedicate several days per year to solidarity actions during their working hours, including sponsoring people struggling with their orientation or professional integration, participating in the financial education of young apprentices and sharing their skills with charities. There are many ways to get involved.

We are committed to support accelerating our Group’s ESG strategy by implementing ESG principles in all our activities and policies. They are translated in our business activity (ESG assessment, reporting, project management or IT activities), our work environment and in our responsible practices for environment protection.