Big Data Engineer

Skill: Big Data Engineer (Pyspark + SQL)

Exp: 5 to 8 years

Loc: Pune & Chennai

JD

• Lead end to end analysis of largescale multisource datasets using Big Data technologies driving insights for strategic and operational decision making
• Design develop and optimize complex analytical queries using Starburst Trino Presto to enable fast federated access across enterprise data sources
• Build manage and optimize analytics workflows in Databricks using SQL PySpark and Python for scalable data processing
• Partner closely with Data Engineering Architecture and Platform teams to influence data models analytics architecture and performance optimization
• Translate ambiguous business problems into structured analytical approaches and clearly communicate findings trends and recommendations to senior stakeholders
• Define and enforce best practices for data quality validation reconciliation and governance within analytics solutions
• Mentor junior and midlevel analysts providing technical guidance code reviews and analytical best practices
• Support executive reporting dashboards and advanced adhoc analysis as required
• Contribute to continuous improvement of analytics processes standards and reusable frameworks710 years of experience as a Data Analyst Senior Data Analyst working in largescale enterprise data environments
• Strong handson experience with Big Data ecosystems Hadoop Hive Spark distributed data platforms
• Proven experience using Starburst TrinoPresto for complex high performance analytical queries
• Extensive experience with Databricks including Databricks SQL Spark based analytics and notebook development
• Expert level SQL skills query optimization performance tuning complex joins
• Strong proficiency in Python and or PySpark for data analysis and transformation
• Solid understanding of data warehousing data modeling and analytics architecture concepts
• Excellent analytical problem solving and stakeholder communication skills Experience working on cloud platforms such as AWS Azure or GCP
• Exposure to BI and visualization tools Power BI Tableau or equivalent
• Knowledge of enterprise data governance metadata management and data quality frameworks
• Experience working in Agile DevOps delivery models Strong ownership and ability to lead quality outcomes end to end
• Clear communication and stakeholder management
• Mentoring mindset and collaboration across QA Dev and DevOps teams