Hacker News front pagevinnylarouge4 min readrelease notesintermediate
Reverse-engineered Jev-like model
Summary
Jevlike is an open‑source starter model that scores a list of text options in a single forward pass. It provides a minimal architecture (option queries, shared dot‑product scorer), synthetic data generation, training/evaluation CLI, and examples on Doom and chess. The repo supports a byte‑level encoder or a frozen Hugging‑Face encoder (e.g., Qwen2.5‑0.5B), runs on CPU/MPS/CUDA, and reports benchm…
- Architecture: each option is turned into a query vector, attends over the context tokens, produces a context vector, then a shared dot‑product yields a score; softmax over options gives probabilities.
- Two encoder modes: (a) train‑from‑scratch byte embeddings (default, 192‑byte context, 32‑byte option limits) and (b) frozen HF encoder with a small trainable scorer head (rank configurable).
- CLI tools: `jevlike-data` (synthetic generation), `jevlike-train`, `jevlike-eval`, `jevlike-predict`; supports CPU, Apple MPS, CUDA.
- Demo scripts for Doom and chess show the same option‑attention head scoring visual inputs; includes scripts to capture gameplay and render a film.
Provides a reproducible, lightweight baseline for one‑pass option scoring, useful for UI menu selection, game controller mapping, or any task where a fixed set of textual choices must be ranked quickly. The open‑source implementation lets engineers experiment with encoder choices and compare agains…
6/10
