Startup Ideas Bank

Cool demo, weak business: a tiny-chip language model is a party trick unless it solves a pain people pay for.

AI roast score: 28/100 (F)

The idea

You built a language model that lives on an ESP32. Impressive engineering. Unfortunately, “runs on a tiny chip” is not a customer, a budget, or a business.

In the founder's own words

slvDev/esp32-ai

Running a 28.9M parameter LLM on a microcontroller


𝕏 slvDev  · 
LinkedIn



This is a 28.9 million parameter language model that generates text on an ESP32-S3
microcontroller. It runs on the chip itself, with nothing sent to a server, and it
displays generated text at 9.88 tokens per second on a small screen wired to the
chip. It fits because most of the model lives in flash instead of RAM, using
Per-Layer Embeddings, an idea from Google's Gemma 3n.

The numbers







Parameters
28.9M stored (25M of them in a flash lookup table)


Chip
ESP32-S3, 512KB SRAM, 8MB PSRAM and 16MB flash


Speed
9.88 tok/s end to end, 94.9 ms/token of compute


Connectivity
none, everything runs on the device


Model size
14.9MB at 4-bit



Why it is hard, and how it fits anyway

A microcontroller has very little fast memory. The ESP32-S3 gives you 512KB of
SRAM, and only the values touched many times per token can live there:
activations and norm weights. The dense core and output head, scanned once per
position, sit in PSRAM. What is left is the embedding tables, and their size is
what normally decides how big a model can be.

In this model, most parameters sit in an embedding table, which the model reads
from rather than computes on. So that 25-million-parameter table stays in slow
flash, and only the few rows each token needs are pulled from it, about 450
bytes. Most of the model is therefore never loaded to run it: it sits in flash
and is sampled a little at a time.

That idea is Google's Per-Layer Embeddings, from
Gemma 3n . Here it runs
on the memory layout of a microcontroller instead of a phone or a GPU.

Each tier holds whatever is read at its own frequency:

SRAM (fast, tiny) activations and norm weights, touched many times a token
PSRAM (medium) the core and output head, read once per position
FLASH (huge, slow) the 25M-param table, about 6 rows read per token (~450 B)

What it does, and what it does not

The model was trained on TinyStories, so it writes short, simple stories and mostly
keeps them coherent. It will not answer questions, follow instructions, write code,
or know facts. That limit comes from the small part of the model that does the
reasoning, and the memory trick does not change it. What is interesting here is the
architecture, fitting a large model onto a tiny chip, rather than what a 28.9 million
parameter model can say.

Models


Barista - espresso question answering

TinyStories - story generation

Running it yourself

Download and deployment are separate operations: one reaches the network, the
other touches the board.

scripts/fetch_model.sh < tinystories | barista > # download, verify, install into artifacts/
scripts/deploy.sh < tinystories | barista > # generate

The roast

This is clever hardware hacking, not a startup yet. Your pitch says the model “will not answer questions, follow instructions, write code, or know facts.” That wipes out the normal reasons anyone buys a language model. So what exactly are they paying for besides the flex? The only real value you’ve shown is that it runs “with nothing sent to a server” and fits on an ESP32-S3. That is a demo feature, not a market. Developers do not buy parameter counts. They buy lower cost, better results, or a job done faster. You have none of those nailed down, and your pricing is “undecided,” which means there is no business model hiding behind the spectacle. Your own profile makes the risk worse: you are a solo founder, you have no funding, and your biggest unknown is whether you can build it. That is fine for research. It is brutal for a product that depends on deep systems work, a very narrow chip target, and a buyer who is supposedly a developer. Right now this looks like a technical artifact looking for a problem.

Red flags

Verdict

Keep it as a technical showcase unless you can tie it to a paid embedded use case with a real buyer and a hard reason to choose it over cloud inference.

Start with your own startup idea →

More ideas roasted by AI

Browse all startup ideas →