Blackwell FP4, the Thinking-Token Paradox, and Splitting MCP into Read and Write

At the end of the August update to the local LLM post, I mentioned I had a replacement shortlist lined up for Gemma 4, and that the real test wasn’t benchmark numbers - it was whether the new model could chain half a dozen MCP tool calls together without making up an argument.

At the end of August, I finally retired Gemma4-26B-A4B. What replaced it - along with an overhaul of the MCP servers it talks to - made the local stack both noticeably faster and much safer to run.

Part 1: From Gemma 4 AWQ to Swift-Qwen3.8 NVFP4

Gemma 4 was a Mixture-of-Experts model (26B total parameters, 4B active per token) quantised with 4-bit AWQ. It fit easily in the RTX 5090’s 32 GB of VRAM, ran quick, and had one annoying habit: its <channel|> reasoning tags needed a custom patch in vLLM’s streaming parser after every upgrade so they wouldn’t leak into tool-call JSON.

When Qwen3.8-27B landed at the end of August, I moved from a sparse MoE model to a dense 27B model quantised for NVIDIA Blackwell’s native 4-bit floating-point (NVFP4) tensor cores.

Instead of uniform 4-bit quantisation across every layer, the checkpoint uses a mixed-precision TensorRT ModelOpt layout (--quantization modelopt_mixed on vLLM 0.28.0):

• Attention projections (~208 layers) stay in FP8 to keep long-context recall and instruction following sharp.
• MLP feed-forward blocks (~193 layers) run in hardware NVFP4, keeping the weights around 20 GB and leaving plenty of room in 32 GB VRAM for an FP8 KV cache at a 65,536-token context window.

The Thinking-Token Paradox

Moving from Gemma 4 to Qwen 3.8 broke multi-turn tool calling on day one - for the exact opposite reason Gemma 4 had broken it back in July.

On Gemma 4, leaving internal reasoning turned on during tool calls caused the model to leak reasoning markers into the JSON payload. The rule on Gemma 4 was simple: turn thinking off when calling tools.

When I applied that exact same rule (enable_thinking: false in the chat template kwargs) to Qwen 3.8, turn one worked fine: the model emitted a clean tool call, the MCP server ran it, and the result came back into the conversation. Then on turn two - when the model was supposed to read the tool output and answer my question - it returned an empty string (content: ""). Every single time.

Qwen 3.8’s training expects a <think>...</think> block between receiving a <tool_response> and writing its final answer. Suppressing the thinking block on a conversation that contains tool results effectively mute-buttons the second turn.

Turning thinking back on fixed the empty replies immediately, at a new cost: the base Qwen3.8-27B model liked to ponder for ~600 tokens (~5.4 seconds) before answering a simple status question.

Three weeks later, switching to Swift-Qwen3.8-27B-NVFP4 - a fine-tune trained to trim redundant chain-of-thought steps without removing the <think> block - cut thinking tokens by 58% and brought turn-two tool synthesis down to 1.8 seconds.

Three Systemd Units and an Outbound Guard

Because upgrading vLLM or swapping quantised checkpoints always carries a small risk of a parser regression, WSL2 keeps three mutually exclusive systemd service units on port 8000:

1. vllm-swift-qwen38.service (active default)
2. vllm-qwen38.service (disabled fallback)
3. vllm-gemma-moe.service (disabled legacy fallback)

Each unit declares Conflicts= against the other two, so starting a fallback unit cleanly stops the active model and frees VRAM before loading the replacement. A companion vllm-network-guard.service applies local firewall rules blocking the inference user from making outbound internet connections - keeping local inference strictly local.

Part 2: Splitting MCP into Read and Write

While the local model was getting faster at calling tools, the MCP server fleet was outgrowing its original setup.

Back in May, I had four custom MCP servers with a handful of tools each. By September, I had five infrastructure MCP servers (Juniper, OPNsense, Aruba, Synology, plus a fifth server for security camera and alarm telemetry) exposing more than 110 tools in total - including tools that create firewall rules, edit DNS zone records, reboot access points, and commit switch configs.

Loading all 110+ tools into every default coding session caused two headaches:

1. Context bloat - Dumping over a hundred JSON tool schemas into the system prompt before you even type a word burns tens of thousands of tokens on every turn.
2. Unnecessary blast radius - Ninety-five percent of the time, when I ask an agent to check the network, I’m diagnosing something (show interfaces, get_firewall_log, get_aps). An agent checking why a port is flapping doesn’t need load_and_commit_config or delete_firewall_rule sitting in its belt.

Five Read-Only Defaults, Four Opt-In -write Servers

The fix was a clean split across every mutable infrastructure server:

Server Family	Default Instance (Read-Only)	Opt-In Admin Instance (--admin)
OPNsense	35 read tools (rules, states, ARP, leases, WireGuard status)	50 write tools (aliases, NAT, rules, service restarts)
Synology	34 read tools (health, SMART, VMM guests, DNS records, shares)	22 write tools (DNS zone edits, file ops, download tasks)
Juniper	7 read tools (show CLI, PFE commands, config diff, facts)	11 write tools (load_and_commit_config, Jinja2 templates)
Aruba	10 read tools (APs, clients, radio stats, rogue APs, show)	11 write tools (SSID toggle, AP reboot, PoE cycling via SSH CLI)
Cameras / Alarm	11 read tools (camera status, snapshots, motion, alarm zones)	None - strictly read-only by design

This split isn’t just hiding tool definitions in Python - each half logs in with a different account:

• The default read-only servers use a read-only service account whose permissions are enforced by the appliance itself (like the read-only Junos login class on the switch).
• The -write servers only load when I launch the CLI with --admin (or task flags like --net or --nas), and authenticate with a separate write-enabled automation account so every change is clearly tagged in the audit logs.

Part 3: Killing the 20-Second Aruba RADIUS Penalty

Splitting the MCP servers also pushed me to finally fix the most annoying bug on the August architecture page: the 20-second Aruba login penalty.

In August, Authentik’s RADIUS outpost had been dropped from the Juniper core switch completely, and on the Aruba wireless controller it was timing out on every login before falling back to a local account - adding twenty seconds to every Aruba MCP call.

Digging through packet captures and Authentik outpost logs in September showed why RADIUS had been failing, and how to fix it properly across all three boxes (Juniper core, OPNsense, and Aruba 7010):

• Vendor-Specific Attributes (VSAs) are required for role mapping - Authenticating a username and password over RADIUS only tells the appliance who you are. Without the vendor’s custom RADIUS attribute in the Access-Accept packet, the box either drops you into an unprivileged default role or rejects the management login outright.
• Aruba 7010 - Adding a custom Authentik RADIUS property mapping that returns Aruba-Admin-Role (root for admin accounts, read-only for the default MCP account) made the controller accept the very first RADIUS packet in milliseconds. The 20-second timeout disappeared immediately.
• Juniper EX3400 virtual chassis - Turning RADIUS back on initially uncovered a subtle privilege-escalation trap: if all RADIUS users map to a single generic user RADIUS template on Junos, then a read-only service account logging in over RADIUS inherits whatever permissions that shared template has. Configuring Authentik to return the Juniper-Local-User-Name VSA let Junos map read-only accounts to a RADIUS template (class mcp-readonly) and admin accounts to a separate RADIUS-ADMIN template (class super-user).

By early October, all three network appliances were authenticating against Authentik RADIUS on the first packet, with zero fallback delays and a hard wall between read-only queries and admin writes.