LLM Routing (Query Complexity Estimation)


The size of LLMs is largely determined by the number of their parameters. It is generally accepted that LLMs with a higher parameter count tend to be more powerful. However, this is also accompanied by increased computing power and correspondingly higher costs. In parallel, there are smaller LLMs that are either optimized in specific domains or designed for less demanding tasks. Interacting with an LLM is through requests, also known as “queries”. To make the most of the resources of LLMs, it is crucial to estimate the complexity of a request. This assessment allows for the targeted assignment of queries to models of different sizes.


The main idea of this thesis is the development of a heuristic for the evaluation of the complexity of queries to Language Models (LLMs) in the context of increasing parameter size:

Contact person: Odej Kao (odej.kao@tu-berlin.de)

Start: immediately