How to Apply:

Please submit your application to [email protected]

Job Title: Data Engineer

Location: Bangalore, India

Department: IT

Position Summary:

Penn Engineering is seeking a skilled and motivated Data Engineer who builds and maintains the pipelines that move and transform data from PEM’s source systems into the PEM Data Lake on AWS and prepares refined, analytics-ready datasets for Qlik and other consumers. The Data Engineer must be strong in SQL and Python, eager to learn cloud data services and motivated to write reliable, well-tested data pipelines.

Key Responsibility

• Develop and maintain ingestion pipelines that load data from source systems — JDE (IBM DB2/400), IQMS (Oracle), CEBOS, SAP, SQL Server, Salesforce, HubSpot and file/SFTP feeds — into Amazon S3.
• Build SQL and Python transformations across the Raw → Refined refinement layers, producing clean, human-readable and dimensional/fact datasets.
• Implement transformations using dbt with the DuckDB engine and AWS services such as Glue and Athena; load curated data (QVDs) to Qlik Cloud for application development.
• Develop and schedule jobs in the orchestration framework, monitoring runs and resolving failures to keep daily batch processing on time.
• Support the manufacturing data pipeline — build and maintain Node-RED / dbt transformations of machine tag data (e.g., run rate, cycle time, machine status) landing in TimescaleDB for OEE reporting in Grafana and Qlik.
• Write data quality checks, validations and reconciliations to ensure accuracy and integrity of delivered datasets.
• Support 85+ data models and the applications that depend on them; respond to data issues and fulfill analyst data requests.
• Document pipelines, data sources and transformations to support cataloging and self-service use of the lake.
• Apply data governance standard least-privilege access, handling of sensitive data, and the application approval workflow in everyday work.
• Use AI-assisted development tools (e.g., Kiro, Amazon Q) as part of standard workflows, applying feedback loops to improve and validate output.

Requirements:

• Bachelor’s degree in computer science, Engineering, Information Systems or a related technical field.
• 2–4 years of professional experience in data engineering, ETL/ELT development or analytics engineering.
• Strong SQL skills and working proficiency in Python (including data libraries such as pandas/PySpark).
• Hands-on experience with at least one cloud data service stack, preferably AWS (S3, Glue, Athena, Redshift, Lambda).
• Understanding of data warehousing fundamentals — dimensional modeling, facts and dimensions, slowly changing dimensions.
• Experience working with relational databases (SQL Server, Oracle, PostgreSQL or DB2) and reading from source systems.
• Familiarity with version control (Git/Bitbucket) and CI/CD pipeline concepts.

Preferred Qualifications:

• Exposure to dbt, DuckDB, or modern transformation tooling.
• Experience with a workflow orchestrator (Apache Airflow, AWS EventBridge, or similar).
• Experience with Qlik Sense / Qlik Cloud, Grafana, or another BI/visualization tool.
• Familiarity with manufacturing or ERP data (JDE, SAP, IQMS) and a manufacturing/operations environment.
• Exposure to change data capture (CDC), streaming, or time-series data (TimescaleDB).
• Relevant cloud certification (e.g., AWS Certified Data Engineer / Solutions Architect Associate).

Key Technologies:

AWS S3, Glue, Athena, Redshift, Lambda, Lake Formation · Qlik Cloud / Qlik Data Integration · dbt · DuckDB · Python / PySpark / SQL · Git / Bitbucket · SQL Server, Oracle, DB2/400 · TimescaleDB / MQTT / Node-RED · Grafana.