Big Data Engineer
Actively Reviewing the ApplicationsAditi Consulting
Job Description
Summary:
Analytics Solutions Data Engineers are responsible for designing, implementing, automating, and maintaining large-scale enterprise ETL processes that are a vital component of Company’s solutions to client business challenges. The team works with a variety of clients across industries ranging from retail to financial services to telecommunications and more. The Engineers maintain and organize massive, complex datasets and develop cutting-edge tools to enable data loading as well as extract data from the Company Data Warehouse, transforming it and then analyzing it to develop insights that can be exported back to the client.
Responsibilities:- Coordinating with PM and teammates to determine prioritization of existing and upcoming tasks.
- Assist in creating, running, and optimizing queries on Impala and Spark to extract data from the Company Data Warehouse.
- Convert business requirements from client stakeholders into technical specs.
- Build PySpark jobs based on the technical specs for multiple projects in parallel.
- Monitor and troubleshoot the execution of jobs.
- Ingest data into SQL Server and transform it for custom analytics models.
- Generate reports/dashboards to be then exported back to the client.
- Brainstorm and collaborate with stakeholders to solve blockers, if any.
- Ensure timely completion of daily deliverables.
- Experience in tech skills required for building/maintaining scalable pipelines for data extraction and critical soft skills like requirement translation, cross-team collaboration, and problem-solving.
- PySpark / Spark Development
- Building and optimizing PySpark jobs
- Running queries on Impala/Spark
- Troubleshooting distributed jobs
- SQL & Data Warehousing
- Writing and optimizing SQL queries
- Extracting/transforming data from Company Data Warehouse
- Working with SQL Server for ingest transformations
- Hadoop Ecosystem Tools
- Working with distributed storage & compute tools
- Understanding execution and job behavior across clusters
- Critical Thinking & Problem-Solving: Ability to assess blockers, debug issues, and propose solutions.
- Collaboration Independent Ownership: Can drive tasks independently but works well with PMs, teammates, and stakeholders.
- Business Intuition & Requirements Translation: Ability to understand business context and convert requirements into technical specs.
#AditiConsulting
# 26-01628
Required Skills
Quick Tip
Customize your resume and cover letter to highlight relevant skills for this position to increase your chances of getting hired.
Related Similar Jobs
View All
Oracle Fusion Lead
H&R Block India
Application Developer
Cargill
Software Engineer I (.NET with React)
Extensible IT Solutions Private Limited is Now Hexure
Entry Level Project Manager
Lensa
Repair Engineer
Capgemini
Share
Quick Apply
Upload your resume to apply for this position