BigStep Technologies - Data Engineer - Python/SQL/ETL

Job Description

Key Responsibilities Build, scale, and maintain robust data solutions to support business objectives. Design, implement, and optimize high-performance data pipelines (extraction, loading, transformation, and orchestration) with scalability, reliability, and speed. Lead end-to-end software development projects involving large language models (LLMs), retrieval-augmented generation (RAG) frameworks, and other AI technologies. Champion modern software engineering practices such as CI/CD, infrastructure-as-code, containerization, and cloud-native deployments. Collaborate with business stakeholders to transform use cases into production-ready services and solutions, owning systems from concept to production. Implement rigorous testing and monitoring practices to ensure superior data quality and integrity. Mentor and guide junior team members, fostering a culture of excellence and continuous & Certifications : Bachelors degree or higher in a STEM field (Computer Science, Math, Physics, Engineering, or related), required. Advanced degree in Computer Science or related field, Experience : 5 to 7 years of total professional experience. 3 to 5 years of experience in Data Engineering or a related discipline with a proven track record of success. Experience in financial services or product development environments is a strong Skills & Competencies : Expertise in Python and SQL with strong data manipulation and analysis skills. Proficiency with Databricks/PySpark and dbt for data warehousing and transformation. Hands-on experience with workflow orchestration tools (e.g., Airflow, Temporal). Experience working with LLMs, prompt engineering, RAG frameworks, and vector databases. Knowledge of machine learning fundamentals, feature engineering, and knowledge graphs is an advantage. Strong experience in designing and implementing complex data systems from scratch. Ability to handle large-scale data projects including ETL, data cleaning, and information retrieval. Strong communication skills (verbal and written). (ref:hirist.tech)