Data Engineer

Company Description

Smart Development is part of the CRD Group and is a forward-thinking think tank based in the vibrant city of Dubai. Our aim is to leverage the power of data and innovative technologies to solve complex challenges and drive sustainable growth. We believe in fostering a culture of continuous learning, collaboration, and impact.

Role Description

We are seeking a diligent and innovative Data Engineer to join our team on a full-time, on-site basis in Dubai. The Data Engineer will play a crucial role in building, optimizing, and maintaining our data architecture, pipelines, and infrastructure to support the analytical and machine learning needs of our organization.

Key Responsibilities

Data Pipeline Construction: Design, construct, install, test and maintain highly scalable data management systems to meet business requirements and goals.

Database Management: Ensure systems meet business requirements and industry practices for data integrity, quality, and reliability. Experience with databases, SQL, and NoSQL technologies are essential.

Data Pre-processing: Work closely with data scientists to ensure the quality and accuracy of data through cleaning, transformation, and validation tasks.

Big Data Technologies: Utilize big data technologies such as Hadoop, Spark, and Hive to handle large-scale data processing and analytics tasks.

ETL Process Development: Develop and manage ETL processes to move and transform data as necessary.

Performance Optimization: Monitor, optimize, and troubleshoot data pipelines to ensure smooth and reliable data flow, and quick availability of data.

Collaboration and Communication: Collaborate with cross-functional teams, including machine learning engineers, data scientists, and software engineers, to understand data needs and deliver on those needs.

Stay Updated: Stay abreast of the latest developments in data engineering, data management, and related technologies, participating in professional groups, forums, and other avenues of learning.

Qualifications

· Bachelor’s degree or higher in Computer Science, Mathematics, Statistics, or a related field.

· 5+ years of experience in data engineering, database management, or related fields.

· Proficiency in SQL, Python, and other data-related programming languages.

· Experience with big data technologies such as Hadoop, Spark, and Hive.

· Strong understanding of ETL processes and data warehousing principles.

· Ability to work efficiently in a fast-paced environment and deliver on tight deadlines.

· Excellent verbal and written communication skills, with a proven ability to collaborate effectively across various teams.