About the Role We are searching for a skilled Data Engineer to join our team. The ideal candidate will have hands‑on experience in development and production implementation with Azure Databricks and Spark, proficiency in Python (PySpark) and SQL coding, and a strong understanding of data lake architecture, ETL processes, and data quality management. Experience with Azure data analytics services and excellent technical writing skills are also required. Responsibilities Develop and implement data processing pipelines using Databricks and Spark. Write efficient and optimized code in Python (PySpark) and SQL for TB-scale data processing. Design and maintain data lake architecture and ETL processes. Ensure data quality and integrity through robust data quality management practices. Utilize Azure data analytics services for data storage, processing, and analysis. Create comprehensive technical documentation, including requirements and testing documents. Collaborate with cross‑functional teams to understand data requirements and deliver solutions. Maintain version control using Git. Requirements 5+ years of hands‑on experience in development and production implementation with Azure Databricks and Spark. Proficiency in Python (PySpark) and SQL coding. Proven ability to optimize performance for TB-scale data processing. Strong understanding of data lake architecture, ETL processes, and data quality management. Experience with Azure data analytics services. Excellent technical writing skills for creating requirements and testing documents. Ability to communicate technical concepts clearly and concisely. Knowledge of Git. Additional Details Seniority level: Mid‑Senior level Employment type: Contract Job function: Information Technology Industries: IT Services and IT Consulting Location: Bogota, D.C., Capital District, Colombia #J-18808-Ljbffr
Azure Data Engineer
PYRAMID CONSULTING, INC
Remote, Remote
Publicado hace 12 días
Denunciar empleo