llm-workshop-2025

Running Models Locally

Session objectives: Privacy concerns regarding data usage have pushed a portion of LLM users to move to locally run LLMs that do not require an internet connection. This session introduces the privacy benefits of running LLMs locally, options to download and run them locally on your computer.

⏳	Topic
10 min	Presentation `Tommy`

Steps to Install and Run Local Models

Download AnythingLLM

Initial Opening of AnythingLLM

This screen contains a simple welcome page.

LLM Preference

This screen contains our LLM Preference. For now, keep AnythingLLM selected in the providers view. If you want to start installing one of the models in the Suggested Models section, feel free (I would start with something on the smaller side).

Data Handling and Privacy

This screen displays the efforts that AnythingLLM takes to ensure that your data is stored only on this machine.

This screen asks if you would like to add your email and answer some questions so that AnythingLLM can get some feedback. You can skip this section if you would like.

Create Your First Workspace

This screen enables you to create your first workspace, an area that’s dedicated for a specific task where you can easily organize documents or customize settings.

Home Page

After creating your first workspace, you will be taken to the homescreen (if it puts you into an interface like ChatGPT, click on the AnythingLLM logo to go the Home Page).

Here you can navigate to your workspaces and embed documents so that the LLM can look through them (there are many more features, but these are the most commonly used ones).

Embedding a Document

Clicking on embed a document will bring us to the screen below. Here, we can upload documents from our computer onto the AnythingLLM documents folder. After adding a file, we can click on it, click on move to workspace, and then save and embed into the workspace. Now the LLM can reference the document. For smaller documents, after clicking save and embed, you can click on the pin icon in the file in your workspace to have it be pinned to the chat you have with an LLM (this prepends the document onto your chat, like you prompted it).

Workspaces

If we click on a workspace in the left tab, we can access it. This interface looks similar to ChatGPT, and also works similarly. Threads are like new chats. At the bottom of the Send a message block, we can modify the LLM we are talking to, attach documents, execute user-defined commands, or even turn on audio! The settings are the small wrench at the bottom left if you would like to change anything specifically. Once in settings, you can click on LLM under AI Providers to download/switch to another LLM.

Note

You can always click the AnythingLLM logo to go back to the home page.