Subject

Natural Language Explanations (NLEs) represent a promising frontier in explainable artificial intelligence, particularly within the context of agriculture. This research aims to develop AI systems that can not only predict outcomes, such as classifying crop images or diagnosing plant diseases, but also provide detailed textual explanations that justify these predictions. By enhancing the interpretability of AI models, farmers can gain better insights into their operations, fostering trust and facilitating more informed decision-making. Despite the potential benefits, the availability of datasets tailored for NLEs in agricultural applications remains limited. However, the advent of large language models (LLMs) and multimodal large models (MLMs) provides novel opportunities to generate relevant datasets and improve model performance in this domain.

Kind of work

The primary tasks in this thesis involve harnessing the capabilities of LLMs and MLMs to create and utilize new datasets for NLEs in agricultural applications. First, the student will leverage the extensive knowledge base of LLMs to generate detailed responses to various agricultural queries, forming a rich dataset to train NLE models. Additionally, the student will utilize LLMs to restructure existing agricultural reports, extracting question-explanation pairs for model training. The student will also explore the use of MLMs to generate textual annotations from agricultural images, even when no textual reports are available. These tasks will culminate in the fine-tuning and evaluation of AI models on the newly created datasets, focusing on both prediction accuracy and the quality of explanations.

Framework of the Thesis

The framework of this master thesis will be organized into several key phases. Initially, a thorough literature review will be conducted to understand the current state of NLEs and the application of LLMs and MLMs in agriculture. Following this, the student will design a methodology for dataset creation, involving both the generation of agricultural queries and explanations using LLMs and the annotation of agricultural images using MLMs. Alternative, the student will explore datasets from partner organisations provided to VUB.

The next phase will involve implementing these methodologies, including data collection, preprocessing, and developing model training pipelines using Python and PyTorch. The student will then evaluate the performance of the trained models using metrics that assess both the accuracy of predictions and the quality of the explanations provided. The final phase will involve documenting the findings, discussing the implications of the results, and proposing future research directions.

Number of Students

1

Expected Student Profile

The ideal candidate for this master thesis should have a strong background in Python programming, with specific expertise in PyTorch and Transformer models, both in theory and practical implementation. Knowledge of natural language processing and machine learning principles is essential, as the project involves advanced techniques in these areas.