How GenAI actually works, from one small JavaScript file

I've used LLMs at work for a while, but mostly through someone else's wrapper. So I opened an empty file called token.js, with the plan of making it do something and then asking why it did it.

It started as about twenty lines. By the end of the day it could think in steps and check its own work, and most of what I actually learned came from things going wrong on the way there.

The first call failed twice, neither time because of the code

The first version was the same snippet everyone starts with:

const response = await openai.chat.completions.create({
  model: "gpt-4o-mini",
  messages: [
    { role: "system", content: "You are a helpful assistant." },
    { role: "user", content: "Hello! Can you help me with a coding problem?" },
  ],
});
console.log(response.choices[0].message.content);

First error: ERR_MODULE_NOT_FOUND. My package.json had tiktoken in it and nothing else, so npm install openai fixed that. Second error:

RateLimitError: 429 You have no credits remaining.

The name is misleading, because I wasn't sending too many requests. There was simply no money on the API account, and a ChatGPT subscription doesn't pay for API calls. It did tell me one useful thing though: the key was fine, since a bad key gets you a 401.

Rather than top up straight away, I tried NVIDIA's hosted models and then OpenRouter, and that turned out to be the first real lesson. I didn't have to rewrite anything. Both speak the same API format, so moving was two lines:

const openai = new OpenAI({
  baseURL: "https://openrouter.ai/api/v1",
  apiKey: process.env.OPENROUTER_API_KEY,
});

To check there was no magic in the SDK, I also replaced it with plain fetch for a while, and got the same answer back. Under the wrapper it's a POST request with some JSON in it. That JSON is the whole interface: a model name and a list of messages.

What the model actually reads

The model doesn't read words. Before it sees anything, your text is split into tokens, and each token is a number. That's what tiktoken was doing in my package.json, so I ran it on my own prompt:

import { encoding_for_model } from "tiktoken";

const enc = encoding_for_model("gpt-4o-mini");
console.log(enc.encode("Hello! Can you help me with a coding problem?"));

Eleven tokens came back:

13225, 0, 4101, 481, 1652, 668, 483, 261, 22458, 4792, 30
Hello  !   Can you help me with a  coding problem ?

Common words get a token each, and notice the spaces belong to the token that follows them. A word the tokenizer hasn't seen much gets broken up instead, so the name of this blog comes out as ho and isted.

From there, generating text is one small operation on repeat. The model looks at all the tokens so far, works out how likely each possible next token is, picks one, adds it to the end, and goes again. A long answer is that loop running a few hundred times. Nobody planned the sentence in advance, and it's why streaming responses arrive a few pieces at a time.

It's also why APIs bill per token rather than per request. A token is the unit of work, and it counts in both directions.

The model remembers nothing

This was the one that changed how I think about all of it. I'd assumed that chat means the model is keeping track of the conversation. It isn't. Each API call starts from nothing.

The way a chat works is that your code keeps the list and resends all of it, every time:

const messages = [
  { role: "system", content: SYSTEM_PROMPT },
  { role: "user", content: question },
];

// after each reply
messages.push({ role: "assistant", content: reply });

The memory lives in my array, not in the model. Once I'd seen that, a few things I'd only half understood made sense. Long chats cost more, because every old message gets paid for again on every call. Context windows matter, because that list has a size limit. And an assistant that "forgets" something early in a long chat has usually just had it trimmed off the list by whatever app it's running in.

A system prompt is only the first message

The system prompt sounds like configuration. In practice it's text at the top of that same list, written to steer whatever comes next. I changed mine to one line:

You are a JS expert and only answer questions about JavaScript.

That one line made it refuse things. I asked for the capital of France and got:

I'm here to answer questions specifically about JavaScript.

No code of mine did that. The behaviour came entirely from text that sits in front of the question.

Making it think in steps

If the model only ever predicts the next token, you can shape its answer by making it produce the reasoning first. By the time it writes the final answer, those steps are in the context and it's building on them. This is called chain of thought, and it's just a prompting pattern. So my system prompt grew some rules:

Work through every question in steps, and reply with exactly one step each time.
- START: say in your own words what the user is asking.
- THINK: work out one small part of the answer. Use 2 to 4 THINK steps.
- EVALUATE: check the THINK step before it. Is it correct and complete?
- OUTPUT: give the final answer to the user. This is always the last step.

Always reply with JSON in this format:
{ "step": "START" | "THINK" | "EVALUATE" | "OUTPUT", "content": "..." }

I also added a worked example to the prompt, because showing the format is far more reliable than describing it. My example has a deliberately wrong THINK that the EVALUATE catches, so the model can see what a correction looks like.

Then the code became a loop. Each pass is one API call that returns one step, gets pushed onto the list, and goes back in, until the model says OUTPUT:

for (let i = 0; i < MAX_STEPS; i++) {
  const response = await openai.chat.completions.create({
    model: "openai/gpt-4o-mini",
    response_format: { type: "json_object" },
    messages,
  });

  const reply = response.choices[0].message.content;
  messages.push({ role: "assistant", content: reply });

  const { step, content } = JSON.parse(reply);
  console.log(`[${step}] ${content}`);

  if (step === "OUTPUT") return;
}

Here's a real run, a bit shortened, on [] + {}:

[START] The user wants to know the result of adding an empty array and an empty object.
[THINK] Using the + operator with two objects can lead to type coercion.
[EVALUATE] This is a correct consideration.
[THINK] [] is converted to an empty string, and {} to '[object Object]'.
[EVALUATE] This is correct.
[THINK] [] + {} will result in '' + '[object Object]'.
[EVALUATE] That is correct.
[OUTPUT] [] + {} results in '[object Object]'.

The answer is right, and watching it arrive one step at a time was the most fun part of the day.

What the steps don't do

It would be easy to stop there and call that an agent that checks its own work, but the runs didn't really support that.

I tried a few more JavaScript questions to see how EVALUATE held up. On typeof NaN it did push back on one of its own steps, which was nice to see, and that answer came out right. Then I asked about the old [1, 2, 3].map(parseInt) trap:

[EVALUATE] The evaluations of parseInt with their respective radices are correct.
           The results would lead to NaN for the first two and 3 for the last.
[OUTPUT] [1, 2, 3].map(parseInt) returns [NaN, NaN, 3].

Node says [1, NaN, NaN]. The reasoning looked careful, every step got approved, and the answer was wrong. That's the same model marking its own homework, and the extra steps made a wrong answer look more trustworthy, not less. Running the code would have settled it in a second, so a real check probably needs to be something outside the model, or at least a different one.

The rules weren't followed all the time either. The prompt says to use 2 to 4 THINK steps, and on the France question it went straight from START to OUTPUT. A good refusal, but not what I'd specified. A system prompt is a strong suggestion, and anything that has to hold every time belongs in code. That's why MAX_STEPS is a plain JavaScript constant and not a line in the prompt.

And it costs more. That [] + {} run took eight API calls, each one resending a longer list than the last, for a question a single call would have answered.

So what I actually have is a small file and a better mental model. Text goes in as numbers, the next token gets predicted over and over, and everything that feels like memory or personality is text being put in front of that loop. I still need to learn how the models are trained, which is a much bigger subject, but I can now look at an AI product and guess what's in its messages array.

First published on my blog. Cover photo: McKay Savage, CC BY 2.0, via Wikimedia Commons (source).