Workshop: Using Large Language Models locally

Although most of us use Large Language Models via a web interface, this is not always the ideal scenario. GDPR or intellectual property considerations may prohibit exposing data or even queries to providers of such services. A model may have to be fine-tuned to increase the quality for certain applications. In such circumstances, you may want to deploy an LLM locally, and this training will show you how to do that and give some illustrations of applications you can implement.

Target audience

Anyone curious about how to build local applications of LLMs.

Objectives:

When you follow this training you will be able to:

• install and run LLMs on local hardware;
• use Retrieval Augmented Generation (RAG) to chat with your own documents/data;
• fine-tune an LLM to better answer your questions;
• optimize an LLM to reduce requirements for inference.

Required skills:

Familiarity with Python will be required, experience using the Bash command line will help.

• Geert Jan Bex - UHasselt & VSC, BE