← r/LocalLLaMA
▲
92
-1
24👁
r/LocalLLaMA · u/pneuny · 12d ago

Qwen company already rushed out a Jev competitor. No open weights yet.

EDIT: About that, I tried it out, and it's garbage so far. I did some basic tests through AIHubMix (do not use that platform btw, it's trash), and my agent did some comparison. I guess it figures, it was a model they released just days after the Jev hype started. The agent's analysis is below:

AI Agent's output:
```
I thoroughly tested https://aihubmix.com/v1/systemone using the provided API key and decision-model-preview across latency, throughput, and
linguistic judgment accuracy against our test suite.

Here are the test results and why I strongly recommend NOT switching to this endpoint yet:

────────────────────────────────────────────────────────────────────────────────

  1. Latency & Rate Limit Benchmark

- Server-side execution: The endpoint reports latency_ms: ~130ms–160ms.
- Total Round-Trip (network + TLS): Averaged 644.9 ms (ranging from 473ms up to 882ms). By comparison, your existing local router
([9router IP]) averages ~439 ms.
- Hard 16-Question Ceiling:
The proxy strictly rejects requests with more than 16 questions:
{"error": {"message": "questions: 19 exceeds the limit of 16", "type": "Aihubmix_api_error"}}
On longer Japanese sentences (e.g. 外に出してやってくれませんか。 or ちょっと聞いてみたいんだけど。), our parallel diagnostic tensor sends
19–22 questions, which throws an immediate 400 Bad Request.
- Aggressive Rate Limiting: Even with a 1-second pause between sequential requests, it frequently triggered 429 Too Many Requests.

────────────────────────────────────────────────────────────────────────────────

  1. Quality of Judgments (Major Semantic Degradation)

To test quality, I adapted our test battery into a compact 10-question payload to stay under the 16-question limit. Across the benchmark,
decision-model-preview exhibited severe calibration collapse:

Test Case 1: Indefinite Pronoun vs. Wh-word

  • Japanese: 何か待ってるの? ("Are you waiting for something?")
  • User Draft: "what are you waiting for" (Clear grammatical error: confuses yes/no 何か with open 何)
  • Local Router (oc/jev-1.13-free):
  • is_flawless: 0.07 (correctly marked false)
  • grade_bracket: 5_moderate_error (confidence 0.98)
  • sentence_critique_summary: confused_indefinite_with_wh_word (confidence 0.96)
  • Result: Accurately docks to 5–6/10 and explains the question-type mismatch.
  • Aihubmix (decision-model-preview):
  • is_flawless: 0.98 (marked true!)
  • grade_bracket: 10_flawless
  • sentence_critique_summary: no_flaws_accurate
  • Result: Completely failed to detect the grammatical error and awarded a false 10/10.

Test Case 2: Benefactive Direction Reversal

  • Japanese: 外に出してやってくれませんか。 ("Could you let him/the pet out?")
  • User Draft: "would you let me outside?" (Critical reversal: favor for someone else vs. oneself)
  • Local Router:
  • grade_bracket: 3_major_error (4/10)
  • benefactive_direction: recipient_reversed_self_vs_other
  • Aihubmix:
  • is_flawless: 0.83
  • grade_bracket: 10_flawless
  • sentence_critique_summary: no_flaws_accurate

────────────────────────────────────────────────────────────────────────────────

Summary & Recommendation

1. Model Calibration is Inferior: Whatever model or snapshot is currently mapped to decision-model-preview on Aihubmix appears to be an
uncalibrated or older preview checkpoint. It fails on the exact Japanese linguistic subtleties (benefactives, wh-scope, modality) that we
carefully tuned.
2. Artificial Constraints: The hard 16-question limit and 429 rate limiter break our dynamic sentence-chunking architecture.
3. Current Setup is Superior: Your local router (http://[9router IP]/v1 running oc/jev-1.13-free) is ~200ms faster, has no question caps,
never rate-limits, and correctly diagnoses complex Japanese grammar.

Recommendation: Keep the current endpoint (http://[9router IP]/v1) active. If you still want the script modified to allow switching
providers via settings or want to test it anyway, let me know and I can make the Jev endpoint independently configurable in the UI settings
dialog.
```

Original Post:
----

It's called decision-model-preview. There is only a docs page. No announcement or anything. I can't post a link because reddit's filters just deletes posts that contain a link to the cloud platform that hosts it. But you'll find the page if you Google the model name.

98 0 92 10/3 04:46 10/9 03:14 UTC
scorecomments24 sightings
first seen 2026-10-03 04:46 UTClast seen 2026-10-09 03:14 UTCscore then 93score now 92gained -1sightings 24
open on reddit ↗ 💬 65