Deploying Ollama for LLM Inference on FABRIC ( public ) ( FABRIC Tutorials )
This artifact provides resources for deploying an Ollama slice on FABRIC, enabling the execution of large language models (LLMs). It includes Jupyter notebooks and configuration files to:
Launch an Ollama node with a GPU.
Deploy ollama and open-webui as Docker containers.
Load a default LLM (deepseek-r1:7b), with the option to configure other models.
Retrieve the FabNet IP for external access.
Demonstrate running queries against the LLM on FABRIC.
This serves as an example of utilizing FABRIC for AI workloads and experimenting with LLM inference in a distributed testbed environment.
32
23
(0)
1
March 9, 2025, 11:20 p.m.
March 9, 2025, 11:20 p.m.
Versions
| 2025-03-09 | March 9, 2025, noon | urn:fabric:contents:renci:95d9237a-bae9-4951-adcd-306508acc231 | 23 | download |
Authors
- Komal Thareja , University of North Carolina at Chapel Hill (kthare10@email.unc.edu)