Data Engineer

Job Description

• Design, develop, and maintain enterprise-scale data pipelines for batch, incremental, and real-time/near-real-time data integration.
• Integrate data from Dynamics 365, Salesforce, SQL databases, REST APIs, JSON/CSV files, file systems, and other enterprise applications.
• Build scalable data solutions using SQL, Python, PySpark, Databricks, Delta Lake, Microsoft Fabric, Azure Data Factory, and Notebooks.
• Design and maintain modern Data Lake, Lakehouse, One Lake, Delta Lake, and Enterprise Data Warehouse solutions.
• Implement CDC, incremental loads, MERGE/UPSERT, data quality, validation, reconciliation, monitoring, logging, error handling, and data observability.
• Optimize SQL queries, Spark jobs, Databricks workloads, Delta Lake, and data pipelines for performance, scalability, and reliability.
• Own the complete lifecycle: requirement → architecture → development → testing → deployment → monitoring → troubleshooting → optimization.
• Build trusted data foundations for Power BI, Semantic Models, enterprise analytics, AI, and business applications.
• Work with Power BI Semantic Models, datasets, relationships, measures, and DAX as an added capability.
• Contribute to AI/Generative AI solutions using Azure OpenAI, Azure AI Search, LLMs, RAG, embeddings, vector/semantic search, AI Agents, Data Agents, and enterprise chatbots.
• Integrate AI solutions with enterprise Data Warehouses, Lakehouse’s, APIs, databases, and Semantic Models.
• Work closely with Data Teams, BI Developers, Application Teams, IT, and Business stakeholders.
• Establish reusable engineering frameworks, coding standards, data quality practices, and production support processes.
• Provide technical guidance and mentoring to other team members.
  

Required Skills & Experience

• 8+ years of professional experience in Data Engineering, Data Integration, or Data Warehousing.
• Very strong hands-on expertise in SQL and Python.
• Strong experience with SQL Server, Databricks, Delta Lake, PySpark.
• Hands-on experience with Microsoft Fabric and/or Azure Data Factory.
• Strong understanding of Enterprise Data Warehouse, Data Lakehouse, and modern data architecture.
• Experience with REST APIs, CDC, real-time/near-real-time integration, and large-scale data processing.
• Experience working with multiple enterprise source systems and complex data integration requirements.
• Strong production support, troubleshooting, performance tuning, and problem-solving skills.
• Ability to take end-to-end ownership of enterprise data engineering solutions.
  

Additional / Advantage Skills

• Power BI Semantic Models, datasets, relationships, DAX, and enterprise BI modelling.
• Azure OpenAI, Azure AI Search, LLM, RAG, embeddings, vector search, semantic search, and AI Agents.
• Experience developing AI-powered chatbots, Data Agents, or enterprise AI applications.
• Git, CI/CD, Azure DevOps, data governance, and data observability.
• Knowledge of modern data architecture and AI-ready data platforms.