Finetuning of Large Language Models (LLM)

Assignment:

LLM finetuning refers to the process of adapting a pre-trained LLMs for specific tasks or domains through further training with data-specific material. The term “finetuning” suggests that the model is already developed a general ability to understand language (through the initial training on a large and diverse dataset), and now it is being finetuned to a narrower domain or specific tasks. Common use cases include for example generating SQL queries or summarizing and reproducing questions and answers (Q&A) based on existing text which however were not used for the original LLM training.

Tasks: In the thesis, the following questions will be examined in more detail:

• Under what circumstances is it advisable to finetune LLMs?
• What methods are available to finetune an LLM, especially considering instruction-based approaches?
• How can we systematically evaluate, whether the finetuned LLM performs better than the general model in a given domain?

Contact person: Odej Kao (odej.kao@tu-berlin.de)

Start: immediately

Resources:

• https://arxiv.org/abs/2401.02115
• https://arxiv.org/abs/2305.02301
• https://dl.acm.org/doi/abs/10.1145/3622896.3622906
• Overview paper: https://arxiv.org/abs/2308.15363