FRCE GPU resources can be used to run open-weight large language models for tasks such as text extraction, classification, summarization, coding, and research workflows that use an LLM. Several options are available depending on whether you prefer an interactive notebook, an LLM endpoint for another application, or a full development environment.

Ollama + Jupyter through Open OnDemand

Launch JupyterLab and Ollama together as an interactive GPU job through Open OnDemand. This is the simplest option for experimenting with a model from Python or R, using Jupyter AI, or adding LLM inference to an existing notebook workflow.

Using Ollama with Jupyter through Open OnDemand

Ollama as an FRCE Endpoint

Run Ollama as a Slurm GPU job and make the model available through an OpenAI-compatible or native Ollama API. This is useful when another application running within FRCE, such as Biomni, LangChain, LiteLLM, or your own code, needs an LLM endpoint.

Using Ollama as an LLM Endpoint on FRCE

VS Code on an FRCE Compute Node

Use VS Code on your desktop while the terminal, extensions, code, and tools run inside a Slurm allocation on FRCE. A GPU session provides a full development environment for building and testing inference workflows, CUDA applications, Ollama, and other model-serving tools.

Running VS Code on an FRCE Compute Node