TrustLLM – Trustworthiness and characteristics of LLMs

Assignment: A recent TrustLLM study presents a comprehensive investigation of the properties of LLMs and eight categories to evaluate the properties of Large Language Models (LLMs). An established benchmark, evaluations and analyses of the characteristics of mainstream LLMs as well as discussions on open challenges and future development directions are also part of the paper.

Tasks: The bachelor thesis aims to create a public repository where users can select different criteria in order to see the most suitable LLM for their needs.

• Questionnaire about use cases and the characteristics of the use cases
• Mapping the characteristics to the relevant categories of the TrustLLM paper
• Calculating feasibility score and ranking the results

Contact person: Thorsten Wittkopp (t.wittkopp@tu-berlin.de)

Start: immediately

Resources:

• Paper: https://arxiv.org/pdf/2401.05561.pdf
• Github: https://github.com/HowieHwong/TrustLLM