Arcgate - Data Engineer - Python/SQL

Job Description

Description About : Arcgate is a dynamic and rapidly growing team of 2500+ professionals passionate about data and technology. Responsibilities We deliver cutting-edge solutions to some of the worlds most innovative startups to market leaders across application development, quality engineering, AI data preparation, data enrichment, search relevance, and content: Design, build, and optimize Python based data pipelines that handle large, complex, and messy datasets efficiently. Develop and manage scalable data infrastructures, including databases and data warehouses such as Snowflake, Azure Data Factory etc ensuring reliability and performance. Build, maintain, and optimize CDC processes that integrate data from multiple sources into the data warehouse. Collaborate closely with data scientists, analysts, and operations teams to gather requirements and deliver high-quality data solutions. Perform data quality checks, validation, and verification to ensure data integrity and consistency. Support and optimize data flows, ingestion, transformation, and publishing across various systems. Work with AWS infrastructure (ECS, RDS, S3), manage deployments using Docker, and package services into containers. Use tools like Prefect, Dagster and dbt to orchestrate and transform data workflows. Implement CI/CD pipelines using Harness and GitHub Actions. Monitor system health and performance using DataDog. Manage infrastructure orchestration with Terraform and Terragrunt. Stay current with industry trends, emerging tools, and best practices in data engineering. Coach and mentor junior team members, promoting best practices and skill development. Contribute across diverse projects, demonstrating flexibility Bachelors degree in Computer Science, Engineering, Mathematics, Physics, or a related field. 5+ years of demonstrable experience building reliable, scalable data pipelines in production environments. Strong experience with Python, SQL programming, and data architecture. Hands-on experience with data modeling in Data Lake or Data Warehouse environments (Snowflake preferred). Familiarity with Prefect, Dagster, dbt, and ETL/ELT pipeline frameworks. Experience with AWS services (ECS, RDS, S3) and containerization using Docker. Knowledge of TypeScript, React, Node.js is a plus for collaborating on the application platform. Strong command of GitHub for source control and Jira for change management. Strong analytical and problem-solving skills, with a hands-on mindset for wrangling data and solving complex challenges. Excellent communication and collaboration skills; ability to work effectively with crossfunctional teams. A proactive, start-up mindset, adaptable, ambitious, responsible, and ready to contribute wherever needed. Passion for delivering high-quality solutions with meticulous attention to detail. Enjoy working in an inclusive, respectful, and highly collaborative environment where every voice matters. Benefits Competitive salary package. Opportunities for growth, learning, and professional development. Dynamic, collaborative, and innovation-driven work culture (ref:hirist.tech)