UVOO Tech Wiki - Recent changes [en]

Sed

Busk — Wed, 03 Jun 2026 20:07:09 GMT

Hugo

Busk — Mon, 01 Jun 2026 15:07:51 GMT

Created page with "``` nohup hugo server --source website_hugo --disableFastRender > hugo-server.log 2>&1 & ```"

New page

```
nohup hugo server --source website_hugo --disableFastRender > hugo-server.log 2>&1 &
```

Rsync

Busk — Sun, 31 May 2026 22:35:42 GMT

Amdgpu rocm

Busk — Tue, 26 May 2026 02:27:30 GMT

Amdgpu rocm

Busk — Tue, 26 May 2026 02:23:14 GMT

Created page with "# Make sure keyring dir exists sudo mkdir --parents --mode=0755 /etc/apt/keyrings # Refresh AMD key, same as you already did wget https://repo.radeon.com/rocm/rocm.gpg.key -O..."

New page

# Make sure keyring dir exists
sudo mkdir --parents --mode=0755 /etc/apt/keyrings

# Refresh AMD key, same as you already did
wget https://repo.radeon.com/rocm/rocm.gpg.key -O - | \
gpg --dearmor | sudo tee /etc/apt/keyrings/rocm.gpg > /dev/null

# Add the missing AMDGPU kernel-driver repo for Ubuntu 24.04 / noble
sudo tee /etc/apt/sources.list.d/amdgpu.list > /dev/null <<'EOF'
deb [arch=amd64 signed-by=/etc/apt/keyrings/rocm.gpg] https://repo.radeon.com/amdgpu/30.30.3/ubuntu noble main
EOF

sudo apt update

# Confirm apt now sees it
apt-cache policy amdgpu-dkms

# Install prerequisites + driver
sudo apt install "linux-headers-$(uname -r)" "linux-modules-extra-$(uname -r)"
sudo apt install amdgpu-dkms

# Add yourself for ROCm access, then reboot
sudo usermod -a -G render,video "$LOGNAME"
sudo reboot

Rocm

Busk — Tue, 26 May 2026 02:04:13 GMT

Created page with "https://rocm.docs.amd.com/projects/install-on-linux/en/latest/install/install-methods/package-manager/package-manager-ubuntu.html"

New page

https://rocm.docs.amd.com/projects/install-on-linux/en/latest/install/install-methods/package-manager/package-manager-ubuntu.html

Zfs optimization

Busk — Mon, 25 May 2026 15:56:52 GMT

Created page with "# ZFS Optimization Get & Set ## Get current settings for pool ``` #!/bin/bash if [ -z "$1" ]; then echo "Usage: $0 <pool_name>" exit 1 fi POOL="$1" if ! zpool list..."

New page

# ZFS Optimization Get & Set

## Get current settings for pool
```
#!/bin/bash

if [ -z "$1" ]; then
echo "Usage: $0 <pool_name>"
exit 1
fi

POOL="$1"

if ! zpool list "$POOL" > /dev/null 2>&1; then
echo "Error: Pool '$POOL' does not exist."
exit 1
fi

echo "=== ZFS Pool Alignment ==="
zpool get ashift "$POOL"
echo ""

echo "=== Dataset Optimizations ==="
zfs get recordsize,atime,compression "$POOL"
echo ""

echo "=== TRIM Status ==="
zpool get autotrim "$POOL"
echo ""
zpool status -t "$POOL"
```

# Set optimization

```
#!/bin/bash

if [ -z "$1" ]; then
echo "Usage: $0 <pool_name>"
exit 1
fi

POOL="$1"

if ! zpool list "$POOL" > /dev/null 2>&1; then
echo "Error: Pool '$POOL' does not exist."
exit 1
fi

zfs set recordsize=1M "$POOL"
zfs set atime=off "$POOL"
zpool set autotrim=on "$POOL"

echo "Optimization settings applied to pool '$POOL'."
```

Codex use local openai api

Busk — Fri, 22 May 2026 15:57:28 GMT

Created page with "To point the **Codex CLI** to your local `llama-server`, you primarily need to override the environment variables that the tool uses to locate the OpenAI API. Since the Codex..."

New page

To point the **Codex CLI** to your local `llama-server`, you primarily need to override the environment variables that the tool uses to locate the OpenAI API.

Since the Codex CLI (and most OpenAI-compatible tools) follows standard OpenAI SDK conventions, you can redirect its traffic by pointing `OPENAI_BASE_URL` to your local endpoint.

### 1. Set the Environment Variables

You need to set these in your shell configuration (e.g., `~/.zshrc`, `~/.bashrc`) or for the specific session where you run the command.

* **`OPENAI_BASE_URL`**: Set this to your `llama-server` address, ensuring you include the `/v1` path.
* **`OPENAI_API_KEY`**: Even if your `llama-server` doesn't require authentication, the CLI usually expects this variable to exist. You can set it to any dummy string (e.g., `sk-no-key`).

**Example for your shell configuration:**

```bash
export OPENAI_BASE_URL="http://localhost:8080/v1"
export OPENAI_API_KEY="sk-not-needed"

```

*After saving this, run `source ~/.zshrc` (or your relevant shell config file) to apply the changes.*

---

### 2. Configure via `config.toml` (Optional/Permanent)

If you prefer a more permanent configuration—or if you need to use a specific model name that `llama-server` might not report by default—you can define a provider profile in the Codex CLI configuration file, typically located at `~/.codex/config.toml`.

Add a custom provider section like this:

```toml
# In ~/.codex/config.toml

[model_providers.local]
name = "Llama Server"
base_url = "http://localhost:8080/v1"
env_key = "OPENAI_API_KEY"

```

---

### 3. Verify the Connection

Once the environment variables are set, test the connection by running a simple command:

```bash
codex "What is the capital of Utah?"

```

### Important Tips for Local Models

* **Model Name:** If the CLI complains that the "model is not found," you may need to explicitly specify the model name that `llama-server` is reporting. You can check what models are available by hitting the `/v1/models` endpoint directly:
`curl http://localhost:8080/v1/models`
* **Compatibility:** Some "agentic" features of high-end CLI tools rely on OpenAI-specific function calling or vision capabilities. Depending on the model you are running in `llama.cpp` (e.g., a standard Llama 3 vs. a specialized coding model), some advanced agentic behaviors might have varying success compared to using a proprietary model like `gpt-4o`.
* **SSL/HTTPS:** Ensure your `llama-server` is running on `http` (or that you handle any certificate issues if you've enabled HTTPS), as CLI tools often fail with self-signed local certificates.

Zfs nvme log and cache

Busk — Fri, 22 May 2026 15:42:26 GMT

Zfs nvme log and cache

Busk — Fri, 22 May 2026 15:36:45 GMT

Created page with "``` sudo nvme list Node Generic SN Model Namespace Usage Format..."

New page

```
sudo nvme list
Node Generic SN Model Namespace Usage Format FW Rev
--------------------- --------------------- -------------------- ---------------------------------------- ---------- -------------------------- ---------------- --------
/dev/nvme0n1 /dev/ng0n1 CVFT439300062P0EGN INTEL SSDPEDMD020T4 0x1 2.00 TB / 2.00 TB 512 B + 0 B 8DV101H0

# 1. Create a new GPT partition table
sudo parted /dev/nvme0n1 mklabel gpt

# 2. Create the first 1TB partition (for SLOG)
sudo parted -a optimal /dev/nvme0n1 mkpart primary 0% 1TB

# 3. Create the second 1TB partition (for L2ARC)
sudo parted -a optimal /dev/nvme0n1 mkpart primary 1TB 100%

lsblk /dev/nvme0n1

# Add to log
sudo zpool add <pool_name> log /dev/nvme0n1p1

# Add to cache
sudo zpool add <pool_name> cache /dev/nvme0n1p2

```

Llama optimization

Busk — Wed, 13 May 2026 17:32:49 GMT

Created page with "Yes, that is a good **stable baseline**: ```bash ./build/bin/llama-server \ -hf ggml-org/gpt-oss-20b-GGUF \ -ngl 999 \ -c 2048 \ -np 1 \ -t 8 \ --no-warmup \ --..."

New page

Yes, that is a good **stable baseline**:

```bash
./build/bin/llama-server \
-hf ggml-org/gpt-oss-20b-GGUF \
-ngl 999 \
-c 2048 \
-np 1 \
-t 8 \
--no-warmup \
--cache-ram 0 \
--host 0.0.0.0 \
--port 8080
```

For your **Threadripper + 128 GB RAM + Tesla P40**, I’d optimize for **stability first**, not max speed. Your previous log showed GPT-OSS 20B fully loaded, with the P40 detected as compute capability 6.1 and about 22.9 GiB VRAM available, so the hardware is basically working.

## My recommended P40-stable launcher

Use this instead:

```bash
cd ~/llama.cpp

GGML_CUDA_DISABLE_GRAPHS=1 ./build/bin/llama-server \
-hf ggml-org/gpt-oss-20b-GGUF \
-ngl 999 \
-c 2048 \
-np 1 \
-t 12 \
-b 512 \
-ub 256 \
--no-warmup \
--cache-ram 0 \
--host 0.0.0.0 \
--port 8080
```

Why:

```text
GGML_CUDA_DISABLE_GRAPHS=1 Avoids CUDA graph weirdness on older GPUs.
-c 2048 Lower context = lower KV/cache pressure.
-np 1 One request/slot at a time.
-t 12 Enough CPU threads without pegging Threadripper.
-b 512 -ub 256 Smaller batches; slower maybe, but less crashy.
--cache-ram 0 Avoids prompt cache save/load path that appeared near your crash.
--no-warmup Avoids extra startup GPU work.
```

CUDA graphs can improve performance, but they are also an area where llama.cpp has had CUDA-specific edge cases and workarounds; disabling them is a reasonable stability choice on Pascal/P40. NVIDIA’s CUDA-graphs writeup explains why they are used for performance, while llama.cpp issue/discussion threads show cases where disabling CUDA graphs is used as a workaround. ([NVIDIA Developer][1])

## Auto-detect launcher script

Create this:

```bash
cat > ~/run-llama-gpt-oss.sh <<'EOF'
#!/usr/bin/env bash
set -euo pipefail

MODEL_REPO="${MODEL_REPO:-ggml-org/gpt-oss-20b-GGUF}"
HOST="${HOST:-0.0.0.0}"
PORT="${PORT:-8080}"

# Detect CPU threads.
CPU_THREADS="$(nproc)"

# Use about half the CPU threads, capped. Good for responsiveness.
if (( CPU_THREADS >= 32 )); then
THREADS=12
elif (( CPU_THREADS >= 16 )); then
THREADS=8
else
THREADS=$(( CPU_THREADS / 2 ))
(( THREADS < 4 )) && THREADS=4
fi

# Detect system RAM in GiB.
RAM_GB="$(awk '/MemTotal/ { printf "%d", $2/1024/1024 }' /proc/meminfo)"

# Detect NVIDIA VRAM in MiB.
if command -v nvidia-smi >/dev/null 2>&1; then
VRAM_MIB="$(nvidia-smi --query-gpu=memory.total --format=csv,noheader,nounits | head -n1 | tr -d ' ')"
GPU_NAME="$(nvidia-smi --query-gpu=name --format=csv,noheader | head -n1)"
else
VRAM_MIB=0
GPU_NAME="none"
fi

# Conservative defaults for older GPUs.
CTX=2048
NP=1
BATCH=512
UBATCH=256
NGL=999
CACHE_RAM=0
NO_WARMUP="--no-warmup"
CUDA_ENV="GGML_CUDA_DISABLE_GRAPHS=1"

# If VRAM is very small, reduce context/batch.
if (( VRAM_MIB > 0 && VRAM_MIB < 16000 )); then
CTX=1024
BATCH=256
UBATCH=128
fi

# If VRAM is >= 32GB, allow larger context.
if (( VRAM_MIB >= 32000 )); then
CTX=4096
BATCH=1024
UBATCH=512
fi

# Detect Pascal/P40-ish GPU and keep safer settings.
if echo "$GPU_NAME" | grep -Eiq 'P40|P100|Pascal|GTX 10'; then
CTX=2048
NP=1
BATCH=512
UBATCH=256
CACHE_RAM=0
CUDA_ENV="GGML_CUDA_DISABLE_GRAPHS=1"
fi

echo "Detected:"
echo " CPU threads: $CPU_THREADS"
echo " RAM: ${RAM_GB} GiB"
echo " GPU: $GPU_NAME"
echo " VRAM: ${VRAM_MIB} MiB"
echo
echo "Launching llama-server:"
echo " model repo: $MODEL_REPO"
echo " ctx: $CTX"
echo " parallel: $NP"
echo " threads: $THREADS"
echo " batch: $BATCH"
echo " ubatch: $UBATCH"
echo " cache-ram: $CACHE_RAM"
echo

cd "$HOME/llama.cpp"

exec env $CUDA_ENV ./build/bin/llama-server \
-hf "$MODEL_REPO" \
-ngl "$NGL" \
-c "$CTX" \
-np "$NP" \
-t "$THREADS" \
-b "$BATCH" \
-ub "$UBATCH" \
$NO_WARMUP \
--cache-ram "$CACHE_RAM" \
--host "$HOST" \
--port "$PORT"
EOF

chmod +x ~/run-llama-gpt-oss.sh
```

Run it:

```bash
~/run-llama-gpt-oss.sh
```

## Test command

In another terminal:

```bash
curl -N http://127.0.0.1:8080/v1/chat/completions \
-H 'Content-Type: application/json' \
-d '{
"model": "gpt-oss-20b",
"stream": true,
"messages": [
{
"role": "user",
"content": "Output only valid Python code. No markdown. Create a small Dog class with name, breed, age, tricks, add_trick(), and __str__()."
}
],
"temperature": 0.2,
"max_tokens": 250
}'
```

## About auto-detecting `max_tokens`

`max_tokens` is **per request**, not really a server setting. The server can control context size with `-c`, but each API request should still set `max_tokens`.

Simple rule:

```text
Small answer: max_tokens 50-150
Small code: max_tokens 200-500
Medium code: max_tokens 800-1500
Long file/design: max_tokens 2000+
```

For the P40, I’d keep most tests at:

```json
"max_tokens": 250
```

Then increase only when needed.

## Add a simple curl wrapper

Create this:

```bash
cat > ~/ask-llama.sh <<'EOF'
#!/usr/bin/env bash
set -euo pipefail

PROMPT="${*:-Say hello.}"
MAX_TOKENS="${MAX_TOKENS:-300}"

curl -s http://127.0.0.1:8080/v1/chat/completions \
-H 'Content-Type: application/json' \
-d "$(jq -n \
--arg prompt "$PROMPT" \
--argjson max_tokens "$MAX_TOKENS" \
'{
model: "gpt-oss-20b",
messages: [
{
role: "user",
content: $prompt
}
],
temperature: 0.2,
max_tokens: $max_tokens
}')" | jq -r '.choices[0].message.content'
EOF

chmod +x ~/ask-llama.sh
```

Use it:

```bash
~/ask-llama.sh "Output only valid Python code. Create a hello world script."
```

With more output:

```bash
MAX_TOKENS=800 ~/ask-llama.sh "Output only valid Python code. Create a Dog class with name, breed, age, tricks, add_trick(), and __str__()."
```

## My final recommendation

For your P40, use:

```bash
GGML_CUDA_DISABLE_GRAPHS=1
-c 2048
-np 1
-t 12
-b 512
-ub 256
--cache-ram 0
--no-warmup
```

After it is stable for a while, try performance increases one at a time:

```text
1. Increase -t from 12 to 16
2. Increase -c from 2048 to 4096
3. Increase -b 512 to 1024
4. Re-enable CUDA graphs only if everything else is stable
```

Do not tune everything at once, because then you will not know what caused the next crash.

[1]: https://developer.nvidia.com/blog/optimizing-llama-cpp-ai-inference-with-cuda-graphs/?utm_source=chatgpt.com "Optimizing llama.cpp AI Inference with CUDA Graphs"

Llama 1

Busk — Wed, 13 May 2026 16:32:10 GMT

Created page with "`model.gguf` was just a placeholder. You need to download an actual `.gguf` file, such as: ```text gpt-oss-20b-mxfp4.gguf ``` For GPT-OSS 20B, the Hugging Face repo is: ```..."

New page

`model.gguf` was just a placeholder. You need to download an actual `.gguf` file, such as:

```text
gpt-oss-20b-mxfp4.gguf
```

For GPT-OSS 20B, the Hugging Face repo is:

```text
ggml-org/gpt-oss-20b-GGUF
```

The model page shows `gpt-oss-20b-mxfp4.gguf` as the file used by `llama.cpp` / `llama-cpp-python`, and it also supports direct `llama-server -hf ggml-org/gpt-oss-20b-GGUF`. ([Hugging Face][1])

## Easiest: let `llama.cpp` download it

From your `llama.cpp` directory:

```bash
cd ~/llama.cpp

./build/bin/llama-server \
-hf ggml-org/gpt-oss-20b-GGUF \
-ngl 999 \
-c 4096 \
--host 0.0.0.0 \
--port 8080
```

The official quick start for this repo is basically:

```bash
llama-server -hf ggml-org/gpt-oss-20b-GGUF -c 0 --jinja
```

([Hugging Face][2])

For your P40, I would start with:

```bash
-c 4096
```

rather than unlimited/context auto mode.

## Manual download to a real file

Install the Hugging Face CLI:

```bash
sudo apt install -y python3-pip
python3 -m pip install --user -U huggingface_hub
```

Make sure your user-local Python bin path is active:

```bash
export PATH="$HOME/.local/bin:$PATH"
```

Create a model directory:

```bash
mkdir -p ~/models/gpt-oss-20b
```

Download the GGUF file:

```bash
huggingface-cli download ggml-org/gpt-oss-20b-GGUF \
gpt-oss-20b-mxfp4.gguf \
--local-dir ~/models/gpt-oss-20b \
--local-dir-use-symlinks False
```

Then run it:

```bash
cd ~/llama.cpp

./build/bin/llama-server \
-m ~/models/gpt-oss-20b/gpt-oss-20b-mxfp4.gguf \
-ngl 999 \
-c 4096 \
--host 0.0.0.0 \
--port 8080
```

## Test it

```bash
curl http://127.0.0.1:8080/v1/chat/completions \
-H 'Content-Type: application/json' \
-d '{
"model": "gpt-oss-20b",
"messages": [
{
"role": "user",
"content": "Write a minimal Go HTTP health check server."
}
],
"temperature": 0.2
}'
```

## For Qwen later

Same idea, but choose a Qwen GGUF repo instead. For example, Qwen’s docs show running Qwen models through `llama.cpp` using GGUF files. ([Hugging Face][3])

For now, get GPT-OSS 20B working first with:

```bash
-hf ggml-org/gpt-oss-20b-GGUF
```

or with the downloaded file:

```bash
-m ~/models/gpt-oss-20b/gpt-oss-20b-mxfp4.gguf
```

[1]: https://huggingface.co/ggml-org/gpt-oss-20b-GGUF?utm_source=chatgpt.com "ggml-org/gpt-oss-20b-GGUF"
[2]: https://huggingface.co/ggml-org/gpt-oss-20b-GGUF/resolve/main/README.md?download=true&utm_source=chatgpt.com "285 Bytes"
[3]: https://huggingface.co/docs/inference-endpoints/engines/llama_cpp?utm_source=chatgpt.com "llama.cpp"

Tesla P40 llama.cpp

Busk — Wed, 13 May 2026 16:25:46 GMT

Tesla P40 llama.cpp

Busk — Wed, 13 May 2026 16:19:32 GMT

Created page with "``` sudo apt update sudo apt install -y git cmake build-essential curl libcurl4-openssl-dev git clone https://github.com/ggml-org/llama.cpp cd llama.cpp cmake -B build -DGGM..."

New page

```
sudo apt update
sudo apt install -y git cmake build-essential curl libcurl4-openssl-dev

git clone https://github.com/ggml-org/llama.cpp
cd llama.cpp

cmake -B build -DGGML_CUDA=ON
cmake --build build --config Release -j"$(nproc)"
```

Yes — for a **Tesla P40**, I would use **`llama.cpp` first** for GPT-OSS, Qwen, DeepSeek, Llama, Gemma, etc.

Your P40 is a great cheap 24 GB VRAM card, but it is **Pascal**, so it lacks Tensor Cores and newer datatype acceleration. That makes `llama.cpp` a better fit than heavier stacks like vLLM/TGI for most local single-GPU use.

## Recommendation for your P40

Use this stack:

```text
Best low-friction stack:
llama.cpp + GGUF models + CUDA build

Optional convenience layer:
Ollama or LM Studio

Avoid as first choice on P40:
vLLM, TensorRT-LLM, exllamav2 for GPT-OSS
```

`llama.cpp` has active support for GPT-OSS and GGUF models, and there are ready GGUF builds for `gpt-oss-20b`. Hugging Face’s GGUF page for `ggml-org/gpt-oss-20b-GGUF` shows direct `llama.cpp`, `llama-cpp-python`, Ollama, LM Studio, and Jan usage paths. ([Hugging Face][1])

For Qwen, the official Qwen docs also document running Qwen3/Qwen3MoE with `llama.cpp` and GGUF, with support starting from specific llama.cpp builds. ([Qwen][2])

## What models fit your P40?

With 24 GB VRAM:

| Model type | Recommendation |
| -------------------------------- | ------------------------------------------- |
| GPT-OSS 20B | Good target |
| GPT-OSS 120B | Not realistic on one P40 |
| Qwen 7B / 14B / 30B-A3B MoE | Good targets depending quant |
| DeepSeek Coder / Qwen Coder GGUF | Good for coding |
| Llama 3.x 8B / 70B quantized | 8B easy, 70B partially/offload or CPU spill |
| Mixtral / MoE GGUF | Can work, speed varies |

For your use — Go, SQL, Python, React, infra code — I would start with:

```text
1. Qwen coder GGUF model
2. GPT-OSS 20B GGUF
3. DeepSeek coder/distill GGUF
```

For the P40 specifically, **GGUF quantized models** are the practical path.

## Build `llama.cpp` with CUDA

On Ubuntu:

```bash
sudo apt update
sudo apt install -y git cmake build-essential curl libcurl4-openssl-dev

git clone https://github.com/ggml-org/llama.cpp
cd llama.cpp

cmake -B build -DGGML_CUDA=ON
cmake --build build --config Release -j"$(nproc)"
```

Check CUDA offload works:

```bash
./build/bin/llama-cli --help | grep -i gpu
```

## Run GPT-OSS 20B

Example using llama.cpp’s Hugging Face download support:

```bash
./build/bin/llama-server \
-hf ggml-org/gpt-oss-20b-GGUF \
-ngl 999 \
-c 8192 \
--host 0.0.0.0 \
--port 8080
```

Then test OpenAI-compatible API:

```bash
curl http://127.0.0.1:8080/v1/chat/completions \
-H 'Content-Type: application/json' \
-d '{
"model": "gpt-oss-20b",
"messages": [
{"role": "user", "content": "Write a small Go HTTP server with health check."}
],
"temperature": 0.2
}'
```

If VRAM gets tight, reduce context:

```bash
-c 4096
```

Or reduce GPU layers:

```bash
-ngl 60
```

But on a 24 GB P40, for many 20B-ish quantized GGUFs, this should be okay.

## Run a Qwen coder model

For coding, Qwen is probably where I would spend most of my time. Example:

```bash
./build/bin/llama-server \
-hf unsloth/Qwen3-Coder-Next-GGUF \
-ngl 999 \
-c 8192 \
--host 0.0.0.0 \
--port 8080
```

The Unsloth Qwen GGUF page notes recent llama.cpp fixes for Qwen output/tool-calling behavior, so use a fresh llama.cpp build rather than an old distro package. ([Hugging Face][3])

## Should you use Ollama instead?

Ollama is easier, but `llama.cpp` gives you more direct control.

Use **Ollama** if you want simple:

```bash
ollama run hf.co/ggml-org/gpt-oss-20b-GGUF
```

Use **llama.cpp directly** if you care about:

```text
GPU layer tuning
context size tuning
exact GGUF file selection
server flags
benchmarking
performance debugging
```

Given how you work, I’d use **llama.cpp directly** first.

## Should you use vLLM?

For your **P40**, probably no — not as your first runtime.

vLLM is excellent for newer GPUs and serving many users, but it tends to assume newer CUDA paths and benefits heavily from Tensor Cores / modern attention kernels. On Pascal P40, `llama.cpp` is usually simpler and more forgiving.

I would use vLLM when you have something like:

```text
RTX 3090
RTX 4090
RTX PRO 4500 Blackwell
A10/A40/A100/H100/L40S
```

For the P40:

```text
llama.cpp > Ollama > exllamav2/vLLM experiments
```

## My practical recommendation

Start with:

```text
Runtime: llama.cpp
Driver: 580-server
Model format: GGUF
First model: gpt-oss-20b-GGUF
Coding model: Qwen coder GGUF
Context: 4096 or 8192 first
GPU layers: -ngl 999
```

Then benchmark:

```bash
./build/bin/llama-bench -m /path/to/model.gguf -ngl 999
```

For a single Tesla P40, **don’t chase the newest serving framework**. Use stable `580-server`, fresh `llama.cpp`, and GGUF models. That will give you the fewest headaches and the best compatibility.

[1]: https://huggingface.co/ggml-org/gpt-oss-20b-GGUF?utm_source=chatgpt.com "ggml-org/gpt-oss-20b-GGUF"
[2]: https://qwen.readthedocs.io/en/latest/run_locally/llama.cpp.html?utm_source=chatgpt.com "llama.cpp - Qwen - Read the Docs"
[3]: https://huggingface.co/unsloth/Qwen3-Coder-Next-GGUF?utm_source=chatgpt.com "unsloth/Qwen3-Coder-Next-GGUF"

Tesla P40

Busk — Wed, 13 May 2026 16:16:18 GMT

Created page with " You must reboot after update so in kernel ```bash nvidia-srv/580.126.20, 6.8.0-111-generic, x86_64: installed ``` I would **stay on `580-server`** for the Tesla P40. Do *..."

New page

You must reboot after update so in kernel

```bash
nvidia-srv/580.126.20, 6.8.0-111-generic, x86_64: installed
```

I would **stay on `580-server`** for the Tesla P40.

Do **not** move to `590` or `595` for this card unless you are testing on a disposable install. The P40 is Pascal, and the newer 590+ branch is where Pascal support becomes problematic/legacy. NVIDIA’s 580 data-center release notes still list Tesla P40 support, while 595 is a newer data-center branch focused on newer supported platforms and has newer compatibility requirements like DCGM 4.3.x+ for DCGM users. ([NVIDIA Docs][1])

Your best option:

```bash
sudo apt install nvidia-driver-580-server nvidia-utils-580-server
```

Then pin/hold it so Ubuntu does not “helpfully” move you to 590/595 later:

```bash
sudo apt-mark hold nvidia-driver-580-server nvidia-utils-580-server
```

Also check whether you have mixed normal and server 580 packages. You currently show:

```text
nvidia-utils-580/noble-updates 580.142...
nvidia-utils-580-server ... 580.126... [installed]
```

That is okay as long as only the `*-server` package is installed. Verify:

```bash
dpkg -l | grep -E 'nvidia-driver|nvidia-utils|libnvidia|nvidia-dkms' | awk '{print $2, $3}'
```

For a Tesla P40 compute box, I would prefer this branch order:

```text
Best: 580-server
Okay: 535-server / 570-server if 580 gives issues
Avoid: 590 / 595 for P40
```

For GPT-OSS/local LLM use, newer `590`/`595` is unlikely to give you anything meaningful on a P40. The bigger limitations are the P40’s Pascal architecture: no Tensor Cores, no BF16, no FP8/MXFP4 acceleration. Stability matters more than newest driver here.

[1]: https://docs.nvidia.com/datacenter/tesla/tesla-release-notes-580-126-20/index.html?utm_source=chatgpt.com "Version 580.126.20(Linux) :: NVIDIA Data Center GPU ..."

```

Codex cli

Busk — Thu, 07 May 2026 23:08:50 GMT

Codex cli

Busk — Thu, 07 May 2026 23:07:48 GMT

Created page with "2. Trusted Binaries The risk is considered low because bubblewrap is a small, heavily audited, and "security-first" piece of software. Its entire purpose is to provide isolati..."

New page

2. Trusted Binaries
The risk is considered low because bubblewrap is a small, heavily audited, and "security-first" piece of software. Its entire purpose is to provide isolation. Most Linux security experts consider it safer to allow bwrap to function (so it can sandbox other dangerous code) than to leave it broken.

3. Targeted vs. Global
This is significantly safer than the alternative method of running sudo sysctl -w kernel.unprivileged_userns_clone=1.

The Global Method: Allows every program on your computer (including a malicious script or a compromised web browser) to create namespaces.

The AppArmor Method: Limits that power strictly to the bwrap utility.

Summary
This configuration is the "middle ground" of security. It restores functionality to your development tools while keeping the rest of the operating system locked down. The risk is that you are placing absolute trust in the integrity of the bwrap developers.

← Older revision		Revision as of 20:07, 3 June 2026
(One intermediate revision by the same user not shown)
Line 2:		Line 2:

	grep -rl oldtext . \| xargs sed -i 's/oldtext/newtext/g'		grep -rl oldtext . \| xargs sed -i 's/oldtext/newtext/g'
		+
		+
		+	find . -type f -name "*.txt" -exec sed -i 's/old_text/new_text/g' {} +
		+
		+	find . -type f -name "*.txt" -exec sed -i '' 's/old_text/new_text/g' {} +
		+
		+	find . -type f -name "*.txt" -print0 \| xargs -0 sed -i 's/old_text/new_text/g'v
		+
		+	## Linux
		+
		+	grep -rlZ --exclude-dir='.git' 'oldtext' . \| xargs -0 sed -i 's/oldtext/newtext/g'
		+
		+
		+	## Macos/freebsd
		+	grep -rl --null --exclude-dir='.git' 'oldtext' . \| xargs -0 sed -i '' 's/oldtext/newtext/g'

← Older revision		Revision as of 22:35, 31 May 2026
(One intermediate revision by the same user not shown)
Line 1:		Line 1:
	# Backup Command		# Backup Command
		+
		+	rsync -axSH --exclude='.locks/' --dry-run --progress ~/.cache/huggingface/hub/ /ai/hf_cache/hub/

	rate limit 10000 Kbytes per second on zfs		rate limit 10000 Kbytes per second on zfs

← Older revision		Revision as of 02:27, 26 May 2026
Line 23:		Line 23:
	sudo usermod -a -G render,video "$LOGNAME"		sudo usermod -a -G render,video "$LOGNAME"
	sudo reboot		sudo reboot
		+
		+
		+
		+	# Test
		+	```
		+	dkms status \| grep -i amdgpu
		+	lsmod \| grep amdgpu
		+	ls /dev/kfd /dev/dri/render*
		+	rocminfo \| head
		+	amd-smi list
		+	```

@@ Line 23: / Line 23: @@
 # Add to cache
 sudo zpool add <pool_name> cache /dev/nvme0n1p2
 ```

@@ Line 1: / Line 1: @@
 ```
 sudo apt update
 sudo apt install -y git cmake build-essential curl libcurl4-openssl-dev
@@ Line 9: / Line 10: @@
 cmake --build build --config Release -j"$(nproc)"
 ```

@@ Line 1: / Line 1: @@
-. Trusted Binaries
+This AppArmor profile is a **targeted exception**. Instead of turning off security restrictions for the entire operating system, it grants a specific permission to only one program: `bubblewrap` (`bwrap`).
-. Targeted vs. Global
+### Breakdown of the Code
-The Global Method: Allows every program on your computer (including a malicious script or a compromised web browser) to create namespaces.
+* **`abi <abi/4.0>,`**: This tells AppArmor to use the feature set and syntax rules introduced in version 4.0.
-The AppArmor Method: Limits that power strictly to the bwrap utility.
+---
-Summary
+### What does it do?
 This configuration is the "middle ground" of security. It restores functionality to your development tools while keeping the rest of the operating system locked down. The risk is that you are placing absolute trust in the integrity of the bwrap developers.