For years “technical PM” meant something vague: you could talk to engineers without embarrassing yourself. That definition is gone. At ProductCon San Francisco on October 7, Carlos, Product School's founder, put a specific list on it, and said Product School now asks PM candidates to demo something they built instead of doing a take-home.
His list, in our words: a technical PM can write evals, choose a model, engineer context, review code that AI wrote, and know what a model call costs. None of that requires writing a parser from scratch. All of it requires having built something.
1. Writing evals
An eval is a test for an AI feature: given this input, the output must (or must never) do that. It is the closest thing to a real spec for anything AI touches, because it says exactly what “good” means. Practice: take any prompt you use at work and write five pass/fail checks for its answers. Run them every time you change the prompt.
2. Choosing a model
Bigger is not always better. Smaller models are faster and cheaper and fine for sorting, tagging and extraction; larger ones earn their cost on reasoning and writing. Practice: run the same task through two models and compare quality, speed and price. Write down which you'd ship and why.
3. Engineering context
Most bad AI output is a context problem, not a model problem. What the model knows (the instructions, the examples, the data you hand it) decides what it does. Practice: keep one living instructions file for a project (many tools read one automatically) and improve it every time the output disappoints you.
4. Reviewing AI-written code
You don't need to write the code to be responsible for it. You do need to read it well enough to ask: what does this touch, what happens when it fails, and where does the data go? Practice: before you accept a change, ask the AI to explain it in plain English and list what could break. If the answer worries you, don't ship.
5. Knowing the cost per call
An AI feature has a unit cost every time it runs. A PM who can't estimate it can't price the feature or know when it stops making sense. Practice: for one thing you've built, work out roughly what a single run costs and what a thousand users a day would cost.
Where to start
Pick the skill that scares you least and practice it this week. The Builder PM Ladder puts these five skills on rungs L4 and L5. Most PMs start at L1 or L2, and that's fine.