Data Engineer

Note: This recruitment is based on the availability of anticipated funding.

Job Description

Data Friendly Space is looking for a highly skilled and experienced Data Engineer to join in our Natural Language Processing (NLP) projects. In this role, you will play a crucial part in building a data architecture that collects, scrapes, manages and processes large amount of raw data from selected websites, various sources and databases into usable, and structured data ingestion for NLP engineers to use in the NLP models while ensuring reliable, up-to-date and secure data exploitation. You will collaborate closely with our NLP engineers, Project and Product managers to efficiently build the data pipeline in our Machine Learning operations. As a Senior Data Engineer, you’ll need to have a deep understanding of data architectures, managing the integration of data sources, web scraping, creating the APIs that will make the data usable and supervise the entire NLP data infrastructure to ensure optimal performance.

Duties and Responsibilities:

• Design, build and optimise real-time data architecture in Cloud platforms specially for NLP Models to support the data production, storage and operations pipeline.
• Perform and oversee tasks such as writing scripts, calling APIs, web scraping, and writing SQL queries.
• Profile and understand the quality of data sources and propose data clean up or review methods.
• Preprocessing acquired data, performing NLP tasks and converting text data for machine training.
• Specialist in the Cloud environment, infrastructure administration and software development.
• Experience in working in product development teams and cross functional teams.
• Knowledge of continuous integration and delivery.
• Conduct code reviews to ensure adherence to best practices.
• Identifies and pursues novel technology to support future strategic opportunities. Language: Fluent in written and spoken English. Knowledge of additional languages is a plus.

What we are looking for in you:

• 3-5 years of relevant experience in data engineering.
• Bachelor’s degree or higher in computer science, data science, or a related field.
• Expertise in web scraping: Scrapy, BeautifulSoup, proxies
• Strong proficiency in DataBase systems such as SQL, Mongodb, PostgreSQL
• At least 3 years of experienced in Python
• Proficiency in Cloud architectures, ETL and batch processes: AWS, Cassandra, AWS Glue, AWS Batch
• Expertise in vectorized databases and embedding vectors such as Pinecone, Weaviate, PGvector
• Experience in open source real-time data streaming platforms: Hadoop, Kafka and Spark.
• Excellent communication skills in English (both written and spoken)
• Excellent analytical and problem-solving skills
• Ability to work independently and as part of a remote team
• Expertise in data ethics and security best practices.

Additional Assets

• Experience with agile development methodologies
• Familiarity with and interest in climate change, humanitarian aid or development context
• Exposure to NLP and Machine Learning models including Large Language Models.

Additional Information:

Contract Duration: Initial contract is four (4) months with the possibility of extension

Start Date: 1 December 2023

End Date: 31 March 2023

Level of Effort (LOE): Full-time (100%) Fixed rate

Salary: 5000 USD per month

Language: Fluent in written and spoken English

This is a remote job that can be completed from anywhere in the world.

About Us

Data Friendly Space (DFS) is a 501(c)(3) global non-profit organization with guiding principles to improve information management and analysis capacity, tools, and processes in the humanitarian and development community to enable better informed and more targeted assistance.

DFS staff is composed of experts from the humanitarian information management and analysis field who specialize in real-time secondary data review and build humanitarian applications that support the fast extraction of information from large volumes of unstructured data.

DFS also focuses on the creation of data-centric web applications, websites, and mobile applications to support humanitarian organizations. When building software, DFS focuses on the intersection between data automation processes powered by Artificial intelligence and human knowledge and skills, in particular when one can help the other to execute the analysis.

DFS is closely linked to the DEEP project, with several of its staff members have contributed to its foundation and development.

Data Friendly Spaces (DFS) core principles include:

• Technology for humans, by humans: Understanding user design process and developing interfaces that facilitate interaction with your data, at the speed of thought.
• Technological responsibility. While we are passionate about new technologies, we are also realistic about how much can be expected and achieved from them. DFS takes a forward-thinking approach to reviewing and selecting tools and approaches that are the most appropriate to solve given problems. We firmly believe that technology is best used when it enhances human expert capabilities.
• Creating sustainable solutions. The ultimate success of DFS comes when its services are no longer needed and humanitarian organizations are able to make the best of their data without external support. DFS aims to empower organizations with self-sufficient methodologies, technologies, and workstreams through appropriately tailored solutions and capacity building. There is nothing more rewarding for us than seeing projects we supported still producing value years after we have stopped supporting them.