Data Engineer II
Actively Reviewing the ApplicationsSmart IMS Inc.
Job Description
Job Title: Data Engineer II
Experience: 3–5 Years
Location: Bangalore
Team: Data Platform & Engineering
Job Overview
We are seeking a highly skilled Data Engineer II to join our Data Platform team. In this role, you will be responsible for designing, building, and optimizing scalable data infrastructure capable of handling large-scale datasets. You will work on high-throughput distributed systems and contribute to modern cloud-native and AI-driven data solutions.
Key Responsibilities
Data Pipeline Development & Optimization
- Design, develop, and maintain scalable ETL/ELT pipelines using Scala and Apache Spark/Flink (Core, SQL, Streaming).
- Optimize Spark jobs and SQL queries for performance, scalability, and efficient resource utilization.
- Build and manage data pipelines for processing large-scale datasets with low latency.
- Implement modern Lakehouse architectures using Apache Iceberg, Hudi, or Delta Lake.
Data Management & Quality
- Design and implement data models using Medallion Architecture (Bronze, Silver, Gold layers).
- Ensure data accuracy and reliability through validation frameworks such as Deequ or Great Expectations.
- Monitor data pipelines with observability tools to track data freshness, lineage, and quality.
Cloud-Native Engineering
- Deploy and manage data workloads on GCP (DataProc, GCS, BigQuery) or equivalent cloud platforms.
- Work with Kubernetes (K8s) and Docker for containerized data processing.
- Contribute to Infrastructure as Code (IaC) and automation for scalable deployments.
Collaboration & Innovation
- Collaborate with cross-functional teams including architects, product managers, and data scientists in an Agile environment.
- Participate in code reviews to maintain high coding standards and best practices.
- Explore and implement GenAI and automation workflows to enhance data engineering processes.
Required Qualifications
Experience
- 3–5 years of hands-on experience in Data Engineering.
Technical Skills
- Strong proficiency in Scala and Apache Spark (Batch & Streaming).
- Solid understanding of SQL and distributed computing concepts.
- Experience with GCP (DataProc, GCS, BigQuery) or equivalent cloud platforms (AWS/Azure).
- Hands-on experience with Kubernetes and Docker.
Architecture & Data Modeling
- Experience with Lakehouse table formats (Iceberg, Hudi, Delta Lake).
- Strong understanding of data warehousing concepts (Star schema, Snowflake schema).
Soft Skills
- Strong problem-solving and analytical skills.
- Ability to work independently and in a team environment.
- Effective communication and collaboration skills.
Education
- Bachelor’s or Master’s degree in Computer Science, Information Technology, Engineering, or a related field.
Required Skills
Quick Tip
Customize your resume and cover letter to highlight relevant skills for this position to increase your chances of getting hired.
Related Similar Jobs
View All
Senior Consultant
PwC India
User Interface Designer
Motadata
Senior Data Analyst
Orange Business
Application Security Engineer
Notion
Material Handler 1
DENSO
Quick Apply
Upload your resume to apply for this position