Data Engineer (AU, IT, Hybrid)

Schedule: 7 am to 4 pm

Location: Hybrid/Office-based

Must-Have Skills

What We're Looking For

• Minimum 2 years' experience in data engineering using Microsoft Fabric and legacy SQL Server environments
• Strong SQL expertise for complex data transformation across Fabric and traditional platforms
• Experience with Python or PySpark for data pipeline development and transformation
• Proven ability to develop and optimise data pipelines using Microsoft Fabric toolkit (Data Factory, Synapse, Notebooks)
• Knowledge of ETL/ELT design patterns and real-time vs. batch processing
• Experience integrating with legacy systems including SSIS, SSRS, and OLAP cubes
• Strong performance tuning and optimisation skills across hybrid environments
• Collaborative mindset for working with Data Analysts on gold table creation and model design
• Experience coordinating with vendors and providing technical oversight during build projects
  
  

Highly Valued:

• Microsoft certifications: DP-600, DP-203, or related Azure/Fabric credentials
• Familiarity with version control (e.g., Git) and CI/CD processes for data workflows
• Exposure to real-time processing or data streaming technologies
• Background in analytics engineering or business intelligence
• Experience implementing data quality monitoring and alerting systems
  
  

What You'll Do

As a Data Engineer in the Microsoft Fabric Squad, you will design, develop, and maintain robust data pipelines and gold tables that fuel analytics and business insights. You will manage data performance across both modern cloud tools and legacy systems, playing a critical role in enabling scalable and accurate data products.

Responsibilities Include

Pipeline Development & Integration

• Build end-to-end pipelines using Microsoft Fabric toolkit (Data Factory, Synapse, Notebooks)
• Develop ETL/ELT processes for ingesting data from modern and legacy sources
• Create and maintain gold tables and semantic models in collaboration with Data Analysts
• Implement real-time and batch data solutions with performance optimization
• Integrate with and maintain legacy SQL Server environments, including SSIS packages and cube processing
• Automate data validation and quality checks
  
  

Data Transformation & Performance Optimisation

• Apply dimensional modeling principles to transform raw data into analysis-ready structures
• Implement complex business logic and reusable data assets
• Optimize queries across Fabric compute engines and traditional systems
• Apply incremental loading and tuning strategies for large data sets
• Monitor and enhance data performance across platforms
  
  

Business Collaboration & Vendor Support

• Collaborate with Data Analysts and business stakeholders to understand data needs
• Provide technical direction during vendor-led data product development
• Ensure vendor pipelines follow established performance, integration, and security standards
• Assist with data model design to support both BAU and strategic analytics
• Contribute to legacy report maintenance and transitions (Excel, SSRS, Power BI)
• Investigate and resolve data quality issues
  
  

Quality Assurance & Monitoring

• Develop unit and integration tests for pipelines
• Implement monitoring and alerting systems for data quality and job performance
• Troubleshoot failures, document processes, and maintain runbooks
• Support deployment and change management procedures
  
  

Why Join Us

• Day 1 HMO, Dental, and Life Insurance
• Vacation & Sick Leave with Quarterly Conversion
• Financial Assistance Program
• 13th Month Pay and Government Benefits
• Career development and Microsoft certification support
• Agile, small-team environment with high autonomy and direct leadership exposure
• Be part of a certified Great Place to Work company
  
  

About the Role & Team

You will be part of an integrated Microsoft Fabric Squad, composed of:

• Data Platform Engineer: Provides technical foundation, governance, and platform scalability
• Data Engineer (You): Builds pipelines, gold tables, and ensures data quality and reliability
• Data Analyst: Delivers business insights, enables self-service, and supports user transitions
  
  

Operational Collaboration:

• Jointly support legacy and modern systems
• Align efforts for vendor project delivery, data governance, and architectural standards
• Share ownership of data performance, platform stability, and user enablement
  
  

Environment and Methodology:

• Agile delivery using the Scrum framework
• Hybrid environment spanning Microsoft Fabric, SQL Server, SSRS, SSIS, Excel, and Power BI
• Direct involvement with external vendors and business stakeholders
• Continuous learning focus in a fast-evolving data ecosystem
  
  

Success Metrics

• High reliability and performance of data pipelines
• Optimised cost and compute usage on the Fabric platform
• Business user adoption of self-service capabilities
• Smooth delivery and handover of vendor-built data products
• Migration progress from legacy to modern data tooling
  
  

Deliver high-quality data solutions, drive platform evolution, and be a core part of our Microsoft Fabric team. Apply now as a Data Engineer at ConnectOS.

#ConnectOSCareers #JoinConnectOS