Deep Dive
Introducing AnyJev
The video begins by introducing AnyJev, a new library developed by Nokia's research team. It can convert open models into decision models like Jev without additional training. The presenter tests AnyJev using a dataset of 500 headlines to determine its ability to distinguish satire from real news. AnyJev's main appeal is its ability to run locally, offering a free alternative to Jev's hosted API, which costs 4.2 cents per million input tokens.
Understanding Decision Models
The video explains decision models, which make quick judgments rather than generating text. Jev, a hosted decision model, charges per token but offers free output. AnyJev, however, adds a decision layer to open models on personal machines, supporting various platforms like Hugging Face and Llama CPP. AnyJev's L0 mode addresses biases by averaging results across different option orders and adjusting for label lean, all without needing labeled data.
Performance and Limitations
The presenter tests Jev and AnyJev using a dataset from Kaggle, revealing Jev's superior accuracy and speed. AnyJev's L0 mode corrects bias but doesn't calibrate confidence. In mixed datasets, AnyJev's content-free prior method helps remove model lean, but it sometimes overshoots, affecting accuracy. Despite running locally for free, AnyJev's training-free approach couldn't match Jev's accuracy on the Quen 8 billion parameter model, highlighting its limitations.