Sr. Data Engineer

Responsibilities

• 3+ years of experience developing data pipelines using Python/PySpark, with proficiency in Spark SQL and Azure Databricks.
• 4+ years of experience with Azure data engineering technologies, including ADLS, Unity Catalog, Azure Data Factory, Azure Databricks, and Azure SQL/Synapse.
• 4+ years of experience with SQL.
• Experience in data modelling, including dimensional modelling, star schema and snowflake schemas.
• Proficiency in version control tools such as Git and project management tools such as Jira.

Qualifications

• Excellent verbal and written communication skills.
• UG: B.Sc, BCA, or B.Tech/B.E. in any specialization.
• Reliable internet connectivity is required.

Good to Have

• Experience working with financial data or finance-related datasets.
• Experience with Structured Streaming and Azure Event Hubs.
• Experience with Declarative Automation Bundles and Spark Declarative Pipelines (SDP).
• Azure certifications.

This Senior Data Engineer role focuses on building data pipelines and modeling data using Python/PySpark, SQL, and Azure data services. The work also involves version control and project tracking, with finance datasets and streaming technologies listed as helpful experience.

Role at a glance

Skills mentioned

What you will likely work on

• Develop data pipelines with Python/PySpark and Spark SQL.
• Use Azure data engineering services, including ADLS, Unity Catalog, Azure Data Factory, Databricks, and Azure SQL/Synapse.
• Apply dimensional data modeling, including star and snowflake schemas.
• Use version control and project management tools to support delivery.
• Communicate clearly in writing and verbally with colleagues.

Good fit signals

• You have at least four years of experience with SQL and Azure data engineering technologies.
• You have at least three years developing data pipelines with Python/PySpark and are proficient in Spark SQL and Azure Databricks.
• You have hands-on data modeling experience, including dimensional, star, and snowflake schemas.
• Experience with financial datasets, streaming, Event Hubs, or the listed Azure certification options would be useful, but is not stated as required.
• An undergraduate degree in B.Sc, BCA, or B.Tech/B.E. in any specialization is specified.