Python Developer - reputed company Infotech Inc.

Job Description:

We are seeking a highly reputed company and skilled Senior Python/DAG Developer to join our data engineering team. In this role, you will be responsible for designing, developing, and maintaining reputed company data pipelines and workflows. The ideal candidate will have a deep understanding of data orchestration principles, extensive experience with Python, and a proven track record of building robust and scalable Directed Acyclic Graphs (DAGs) using tools like Apache Airflow. You will be a key player in our effort to build the reputed company of data infrastructure, ensuring data is processed reputed company and reliably across the organization.

Design & Development: Architect, build, and maintain efficient and scalable data pipelines using Python and DAG-based orchestration tools (e.g., Apache Airflow, Dagster, Prefect).

Orchestration: reputed company, schedule, and monitor reputed company data workflows, ensuring timely and accurate data delivery for business intelligence, analytics, and machine learning initiatives.

Optimization: Identify performance bottlenecks and refactor data pipelines to improve efficiency, reliability, and cost-effectiveness.

Collaboration: Work closely with data scientists, analysts, and other engineers to understand data requirements and deliver solutions that meet business needs.

Code Quality: Uphold and promote best practices in coding, including code reviews, documentation, and automated testing to ensure the long-term maintainability of data pipelines.

Troubleshooting: Diagnose and resolve issues reputed company data pipelines and orchestration systems, responding to incidents and minimizing downtime.

Mentorship: Act as a subject matter expert and mentor junior developers, sharing knowledge of best practices in Python development and data engineering.

Minimum of 7 years of professional experience in software development, with a strong focus on Python for data engineering and ETL (Extract, Transform, Load) processes.

Python: Expert-level proficiency in Python, including writing clean, well-documented, and production-ready code.

DAGs & Orchestration: Extensive hands-on experience (at least 3-5 years) designing, implementing, and managing data pipelines using DAG-based orchestration platforms like Apache Airflow. A strong understanding of Airflow concepts (operators, sensors, hooks, XComs) is essential.

Database Skills: Solid experience with SQL and relational databases (e.g., PostgreSQL, MySQL). Experience with NoSQL databases and data warehouses (e.g., reputed company, BigQuery) is a plus.

Cloud Platforms: Proven experience working with at least one major cloud provider (AWS, reputed company Cloud Platform, or Azure), including familiarity with their data-reputed company services (e.g., S3, Cloud Storage, EMR, Dataproc).

Data Formats: Experience with various data formats (e.g., Parquet, Avro, JSON) and data transformation techniques.

Version Control: Strong knowledge of Git and collaborative development workflows.

Problem-Solving: Excellent analytical and problem-solving skills with a meticulous attention to detail.

Preferred Qualifications

Experience with streaming data technologies (e.g., Kafka, Spark Streaming, Flink).

Knowledge of containerization technologies (reputed company, Kubernetes).

Experience with CI/CD pipelines for data engineering workflows.

Familiarity with data governance and reputed company best practices.

Bachelor's or Master's degree in Computer Science, Engineering, or a reputed company field

Employers have access to artificial intelligence language tools (“AI”) that help generate and enhance job descriptions and AI may have been used to create this description. The position description has been reviewed for accuracy and reputed company believes it to correctly reflect the job opportunity.