Senior Data Engineer

Job Description

Our client is seeking a highly skilled Senior Data Engineer to support advanced data processing and engineering initiatives within an enterprise automotive environment. The role focuses on designing and optimizing robust data pipelines, handling large-scale test data, and contributing to distributed system architectures.

The ideal candidate will bring strong expertise in Python-based development, data engineering practices, and cloud infrastructure, along with experience working in complex, production-grade environments.

Key Responsibilities

• Design, develop, and maintain scalable ETL/ELT pipelines for large-scale data processing
• Manage ingestion and transformation of high-volume test bed data across multiple formats
• Build and optimize distributed data processing systems with a focus on performance and reliability
• Implement object-oriented solutions using advanced programming patterns and concurrency techniques
• Collaborate with cross-functional teams to support data-driven applications and workflows
• Deploy and manage applications within Windows Server environments
• Develop and maintain CI/CD pipelines using tools such as GitHub Actions
• Monitor system performance and implement logging solutions, including integrations with monitoring tools like Grafana
• Handle file system operations, including network drive configurations and NAS interactions
• Ensure efficient database integration and management across Oracle, SQL, and MongoDB systems

Required Experience

• Senior-level experience in data engineering or related roles
• Strong hands-on experience with production-grade ETL pipeline development
• Proven expertise in designing distributed systems and handling large-scale data processing
• Experience working in enterprise environments, preferably within the automotive or testing domain
• Solid understanding of concurrency, multiprocessing, and performance optimization techniques
• Familiarity with Windows-based deployment and system environments

Skills Required

• Advanced proficiency in Python (3.9–3.11)
• Strong knowledge of Object-Oriented Programming principles
• Experience with ETL/ELT pipelines and large-scale data ingestion
• Proficiency in SQL and database technologies including Oracle and MongoDB
• Hands-on experience with Microsoft Azure
• Familiarity with CI/CD practices and tools (e.g., GitHub Actions)
• Knowledge of file processing across multiple data formats
• Experience with PowerShell scripting and Windows service deployment

Preferred / Additional Skills

• Experience with domain-specific data frameworks (e.g., Sympathy for Data)
• Knowledge of automotive test bed systems and related workflows
• Understanding of network file systems (SMB/CIFS, NAS)
• Experience with logging and monitoring solutions at file system and infrastructure level

Education

• Bachelor’s or Master’s degree in Computer Science, Data Engineering, or a related field

Language Requirement

• English – Professional Proficiency

Application Method: Interested candidates can apply by sending their profile to hr@semiconservicenordic.com