Model Repo testing

Model Repo is currently in alpha and is available on Mac and Linux only. Windows support is coming soon.

Why use Model Repo

Model Repo lets you upload your own models to private storage on Runpod and attach them directly to Serverless endpoints. Key benefits:

Faster cold starts: Models are pre-cached on the worker host rather than downloaded at runtime.
No HuggingFace dependency: Your models are stored in Runpod’s infrastructure, so endpoints don’t require an outbound download on every cold start.
Private storage: Models are stored in your account and are not accessible to other users.

Manual testing

Prerequisites

Your email is feature-flagged for Model Repo access.
jq is installed for parsing JSON output.

Set environment variables

Export the following before running any commands. Make sure to set your actual API key — missing this is the most common source of auth errors later.

export RUNPOD_API_URL="https://rest.runpod.io/v1"
export RUNPOD_GRAPHQL_URL="https://api.runpod.io/graphql"
export RUNPOD_API_KEY="your-api-key"   # replace with your actual API key

export MODEL_NAME="model_name"  # unique name per test run — reusing the same name uploads a new version, not a new model
export MODEL_PATH="/path/to/model"               # local path to the model files you want to upload

MODEL_NAME must be unique for each test run. If you reuse the same name, the upload creates a new version of the existing model rather than a new model.

Step 1: Install runpodctl

Option A: Install via Homebrew (recommended)

brew install runpod/runpodctl/runpodctl

Option B: Build from source

brew install go                              # install Go, required to build runpodctl
git clone git@github.com:runpod/runpodctl.git
cd runpodctl
make                                         # builds the binary to ./bin/runpodctl

If you build from source, the binary is at ./bin/runpodctl. Either run it with that path, or add ./bin to your PATH. The steps below use runpodctl — adjust accordingly.

Step 2: Upload the model

# --name: the name to register the model under in your repo
# --model-path: local path to the model files
# --create-upload: creates the upload session and transfers files
runpodctl model add \
  --name "$MODEL_NAME" \
  --model-path "$MODEL_PATH" \
  --create-upload

This outputs a JSON string listing all uploaded files.

Step 3: Wait for the model to be hashed

After upload, the model must be hashed by an asynchronous background process. This typically completes in a few minutes but can take up to 10–15 minutes. Poll until the hash field is non-null:

runpodctl model list --name "$MODEL_NAME" | jq -r '.[0].versions[0].hash'

While hashing is in progress, the command returns null:

% runpodctl model list --name "$MODEL_NAME" | jq -r '.[0].versions[0].hash'
null

Once hashing is complete, it returns the hash value:

% runpodctl model list --name "$MODEL_NAME" | jq -r '.[0].versions[0].hash'
71a311bdf0ca44119ed74dbef8cf573bc89b58cbc48a10fe508f756ebb1922dc

Step 4: Get your user ID and model hash

export USER_ID="$(runpodctl user | jq -r '.id')"        # your Runpod user ID
export MODEL_HASH="$(runpodctl model list --name "$MODEL_NAME" | jq -r '.[0].versions[0].hash')"   # the hash from step 3

Step 5: Deploy a Serverless endpoint with the model attached

# --name: name for the endpoint
# --hub-id: the Hub template to deploy
# --gpu-id: GPU type
# --workers-max: maximum number of active workers
# --workers-min: minimum number of workers kept warm
# --model-reference: attaches your model to the endpoint
# --env: sets the model path on the worker
# --min-cuda-version: works around a bug in runpodctl
runpodctl serverless create \
  --name "my_worker" \
  --hub-id "cm8h09d9n000008jvh2rqdsmb" \
  --gpu-id "AMPERE_24" \
  --workers-max 3 \
  --workers-min 1 \
  --model-reference "https://local/$USER_ID/$MODEL_NAME:$MODEL_HASH" \
  --env MODEL_NAME="/runpod/model-store/modelrepo-local/models/$USER_ID/$MODEL_NAME/$MODEL_HASH" \
  --min-cuda-version "13.0"

--model-reference is only supported with --hub-id and GPU endpoints. It is repeatable if you need to attach multiple models to the same endpoint.

Step 6: Verify the model is working

Send a test request to confirm the endpoint is live and the model is accessible. Replace ENDPOINT_ID with the ID returned in the previous step:

curl -s -X POST "https://api.runpod.ai/v2/${ENDPOINT_ID}/runsync" \
  -H "Authorization: Bearer $RUNPOD_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"input": {"prompt": "hello"}}' | jq

A successful response confirms the endpoint is running and the model is attached. If the request fails with an auth error, verify that RUNPOD_API_KEY is set correctly. If you prefer a graphical interface to curl, you can also send requests to the worker from the web UI.

Step 7: Clean up

Delete the endpoint after testing to stop accruing spend. Use the web UI or:

runpodctl serverless delete <endpoint-id>

Get started

Flash

Serverless

Pods

Storage

Public Endpoints

Instant Clusters

Accounts and billing

Integrations

Hub

Reference

Why use Model Repo

Manual testing

Prerequisites

Set environment variables

Step 1: Install runpodctl

Step 2: Upload the model

Step 3: Wait for the model to be hashed

Step 4: Get your user ID and model hash

Step 5: Deploy a Serverless endpoint with the model attached

Step 6: Verify the model is working

Step 7: Clean up

​Why use Model Repo

​Manual testing

​Prerequisites

​Set environment variables

​Step 1: Install runpodctl

​Step 2: Upload the model

​Step 3: Wait for the model to be hashed

​Step 4: Get your user ID and model hash

​Step 5: Deploy a Serverless endpoint with the model attached

​Step 6: Verify the model is working

​Step 7: Clean up

Why use Model Repo

Manual testing

Prerequisites

Set environment variables

Step 1: Install runpodctl

Step 2: Upload the model

Step 3: Wait for the model to be hashed

Step 4: Get your user ID and model hash

Step 5: Deploy a Serverless endpoint with the model attached

Step 6: Verify the model is working

Step 7: Clean up