What generative AI is

Generative AI (Gen AI) is the family of AI systems that produce new content - text, code, images, audio, video - rather than a label or a prediction about content that already exists. A spam filter reads an email and answers "spam" or "not spam"; a generative model writes the email. The output is not looked up anywhere. It is composed, piece by piece, to fit patterns the model learned from a very large body of examples.

The word does most of the work. Predictive AI chooses from a fixed set of answers: which category, which number, fraud or not. Generative AI's answer space is open: any sentence, any picture. That one difference explains most of what follows - why it is so useful for drafting and transforming, why the same prompt can give different results, and why it can produce something fluent that is not true.

Nearly everything sold under the AI label in the last few years - chat assistants, code completion, image generators, voice clones - is generative AI. It is one kind of machine learning, which is one kind of AI; section 04 places the terms.

How it works

Two steps, far apart in time. Training happens once, over months, on an enormous amount of data: the model - a very large network of adjustable numbers called parameters - is shown example after example and nudged until it is good at predicting what comes next in each one. It does not store the examples; it stores the regularities in them. Generation happens every time you use it: you give it a prompt, and it produces output that fits those regularities, one small piece at a time.

What "one small piece" means depends on the kind of content:

  • Text and code come out as a sequence of tokens - words or fragments of words. At each step the model scores every possible next token, one is picked, and the process repeats until the answer is complete. This is the mechanism inside every large language model.
  • Images, audio, and video mostly come out of diffusion: the model starts from pure noise and removes it in steps, each step nudging the picture toward something that matches the description. Training taught it the reverse - how images dissolve into noise - so it can run the process backwards.

In both cases the model learned a sense of what is likely and then samples from it. Sampling is deliberate randomness: pick a very likely option most of the time, a less likely one occasionally. It is why the same prompt gives a different reply each time, why you can ask for "another version" - and, because likely and true are different things, why a fluent answer can be wrong.

The modalities

The mechanism is shared; what changes is the kind of content coming out and what it is good for in a builder's day:

  • Text. Drafting, summarising, translating, rewriting, answering questions. The chat assistants - Claude, ChatGPT - are text generation with a conversation wrapped around it.
  • Code. The same token-by-token generation, trained heavily on source code. In the editor it is completion (GitHub Copilot); in the terminal it is an agent that edits files and runs commands (Claude Code).
  • Images. Text-to-image and image editing: Nano Banana and GPT Image are the two families covered on StackNova. Strong at illustration, product shots, and variations; weaker at exact text inside the image and at precise counts of things.
  • Audio. Speech from text, including voices cloned from a short sample, and music or sound effects from a description. The same idea, applied to sound.
  • Video. Short clips from a text description or a still image. The most expensive modality to run and the least mature - diffusion again, extended over time.

Increasingly one model does several of these - reads an image and writes about it, hears speech and answers in text. That is multimodal: several kinds of content in and out of the same model, which is why a chat assistant can now look at a screenshot.

Where the words fit

The terms around generative AI nest inside each other. From the outside in:

  • AI is the widest term: any software doing something that looks like judgement. Most of it is not generative - fraud scoring, recommendation ranking, route planning.
  • Machine learning is AI that learns its behaviour from data instead of hand-written rules. Generative AI is machine learning whose output is new content.
  • Foundation model is one very large generative model, pretrained on broad data and reused for many tasks. An LLM is a foundation model for text. What is a model? puts the whole ladder in order.
  • Assistant or chatbot is a product wrapped around a model: conversation history, a system prompt, safety rules, a subscription.
  • Agent is a generative model running in a loop: it produces an action, a tool carries it out, the result comes back, and it produces the next one. What is agentic AI? covers the loop.

When someone says "AI" in a product meeting they almost always mean one of the last three. Ask which - the answer changes what you are buying, what it costs, and what can go wrong.

What it is good and bad at

A generative model is judged by fit - does the output match the patterns of good output - and it is excellent wherever fit is what you need:

  • Drafting. A first version of anything with a known shape - an email, a test, a function, a summary, a landing page. The blank page is its best use.
  • Transforming. Summarise this, translate that, turn these notes into a table, explain this code, rewrite for a different reader. The input carries the facts; the model changes the form.
  • Variations at volume. Ten headlines, five layouts, twenty test cases. Cheap to ask for, cheap to discard.

It is weak wherever the output is judged by truth rather than fit:

  • Facts it saw rarely or never. Obscure people, small companies, anything after its training cutoff. It produces the shape of an answer with plausible content filled in - a hallucination, and the reason a generated citation is checked before it is trusted.
  • Exactness. Arithmetic, counting, precise quotes, the fourth decimal place. Fluent text is not a calculator; give the model a tool for that and let the tool answer.
  • Knowing when it is wrong. Confidence does not vary with accuracy. Nothing in the output tells you which sentences to check.

The practical rule: use it freely where you can judge the result by looking at it, and put a source, a test, or a person between it and the reader where the result has to be true. Retrieval - handing the model the facts before it writes - is how products do the first half of that.

How we got here

Three ideas, a few years apart, made the current wave possible, and one product launch made it visible:

  • 2014 - GANs. Generative adversarial networks trained two networks against each other, one producing images and one judging whether they were real. The first convincingly realistic generated faces came out of this line of work.
  • 2017 - the Transformer. A network architecture built around attention: every token in a sequence can weigh every other. It scaled with data and compute in a way earlier designs did not, and it is the architecture inside every major language model since.
  • 2020 - diffusion. Denoising diffusion models made image generation stable and controllable by learning to reverse the process of adding noise. Within two years they were behind DALL·E 2, Stable Diffusion, and Midjourney.
  • 2022 - ChatGPT. A conversational interface on a large language model, released in November 2022. The technology was several years old; the product made generative AI a mainstream tool within weeks.
  • Since then. Models became multimodal, context windows grew from a few pages to whole books, and the loop of model plus tools turned assistants into agents. The mechanism in section 02 has not changed; what it is attached to has.

What to keep: generative AI composes; it does not look things up. That is the source of everything it does well - drafts, transformations, variations - and everything it does badly. Design your use of it around the difference.

References