Senior Databricks Engineer

Senior Databricks Engineer (Open to Canada and USA - West Coast) Fractal is a strategic AI partner to Fortune 500 companies with a vision to power every human decision in the enterprise. Fractal is building a world where individual choices, freedom, and diversity are the greatest assets. An ecosystem where human imagination is at the heart of every decision. Where no possibility is written off, only challenged to get better. We believe that a true Fractalite is the one who empowers imagination with intelligence. Fractal has been featured as a Great Place to Work by The Economic Times in partnership with the Great Place to Work® Institute and recognized as a ‘Cool Vendor’ and a ‘Vendor to Watch’ by Gartner. Please visit Fractal | Intelligence for Imagination for more information about Fractal. Position Overview : Fractal is looking for a proactive and driven Databricks Engineer to join our Consulting team for a client on the West coast. In this role, you will be working on a new Data platform build. You will work designing the system architecture and solution, ensuring the platform is scalable while performant, and creating automated data pipelines in a medallion architecture. Assignment: Design and implement data pipelines using Databricks, Azure Functions, AWS Lambda and/or Spark. Build and maintain data lake solutions and pipelines that can scale. Develop and maintain ETL workflows, including data transformation, validation, and loading. Implement and maintain data security policies and procedures to ensure the confidentiality, integrity, and availability of data. Have expertise in building data pipelines in medallion architecture. Optimize data pipelines for performance, scalability, and cost. Keep up to date with emerging trends and technologies in the Big Data ecosystem. Azure and AWS expertise are most relevant from a cloud perspective. Skills / Experience: Experience with software and cloud engineering and architecture in building enterprise data analytics platforms Experience in implementing solutions using Azure Data Lake, Azure Databricks and S3 Experience creating data ingestion and data pipelines using databricks Experience configuring and deploying Azure services. Expertise with Databricks Experienced in creating reports in Power BI Additional skills: Excellent communication skills verbal and written Experienced in facilitating meetings and working sessions Experience in Azure DevOps and working in an Agile methodology Experience with infrastructure as code concepts, CI/CD, ARM templates, bicep, Terraform, and other automation frameworks. Certified in Data Engineering on Databricks. Experience DLT pipelines is a plus. Able to lead the direction on design decisions. Pay: The wage range for this role takes into account the wide range of factors that are considered in making compensation decisions, including but not limited to skill sets; experience and training; licensure and certifications; and other business and organizational needs. The disclosed range estimate has not been adjusted for the applicable geographic differential associated with the location at which the position may be filled. At Fractal, it is not typical for an individual to be hired at or near the top of the range for their role and compensation decisions are dependent on the facts and circumstances of each case. A reasonable estimate of the current range is: $115,000 - $165,000. In addition, you may be eligible for a discretionary bonus for the current performance period. Benefits: As a full-time employee of the company or as an hourly employee working more than 30 hours per week, you will be eligible to participate in the health, dental, vision, life insurance, and disability plans in accordance with the plan documents, which may be amended from time to time. You will be eligible for benefits on the first day of employment with the Company. In addition, you are eligible to participate in the Company 401(k) Plan after 30 days of employment, in accordance with the applicable plan terms. The Company provides for 11 paid holidays and 12 weeks of Parental Leave. We also follow a “free time” PTO policy, allowing you the flexibility to take the time needed for either sick time or vacation. Fractal provides equal employment opportunities to all employees and applicants for employment and prohibits discrimination and harassment of any type without regard to race, color, religion, age, sex, national origin, disability status, genetics, protected veteran status, sexual orientation, gender identity or expression, or any other characteristic protected by federal, state or local laws.