Under the supervision of the Information Management Adviser in the Data Management and Risk Analysis Hub, this consultancy aims to support the efficient consolidation and breakdown of historical IDMC data through thorough data analysis, geographical segmentation, and hazard alignment. The goal is to address data gaps, streamline data management processes, and improve the organization’s capacity to accurately analyze and interpret historical data. Since the coding infrastructure for these tasks has already been developed, the consultant will mostly utilize the existing code and methodologies, adapting them to specific cases ensuring its integration with the existing data processing pipelines.

As a Data Scientist Consultant, you will be responsible for supporting the following tasks:

1. Adapt a generic classification algorithm based on keywords for specific regions, sources and displacement types.

2. Help to develop a methodology to distinguish flow and stock displacement data from text.

3. Extract and harmonise displacement information from historical Excel files.

4. Adapt data extraction pipelines using Large Language Models via prompt engineering for conflict.

5. Develop a methodology to match conflict and disaster events to existing displacement figures.

6. Develop a methodology to rank the relevance of displacement sources.

7. Build UX annotator modules to highlight extracted information for specific sources.

8. Implementing automated quality control measures to ensure the reliability of the extracted data.

Through these tasks, the consultant will play a crucial role in enhancing the accuracy and completeness of IDMC’s historical data, contributing to improved data analysis and informed decision-making processes.