my experiments with so called "truth"

A language model can use a sentence you just typed without learning new weights from it. The prompt changes the calculation's input. During ordinary generation, the model's learned numbers stay fixed. To see how that works, give it the words “The kettle is on the” and follow one prediction. We will follow a model in the GPT family, short for Generative Pre-trained Transformer. It generates text using already-learned parameters: the numbers its calculations use. We will trace one prediction through that model, then see how repeating the step produces more text. The second article explains how training learns those numbers. Start with one prediction After “The kettle is on the”, “stove” is a possible continuation. So is “table”. The model assigns scores to the possible next tokens, which we can turn into probabilities. Choosing a token from those probabilities is a separate step. A token is a unit of text defined by a tokenizer. It might be a word, part of a word, or punctua...