Less Code Is Half an Argument

Every framework comparison you have read counts the same thing: how many lines the author wrote. Fewer lines wins. The conclusion follows so naturally that nobody says out loud what it assumes — that the lines you did not write cost nothing.

They cost something. On the task below, one framework executed 1895 lines of its own package to do work that took 37 lines without it. Those lines are not gone; they moved somewhere you cannot see them, cannot step through them, and cannot fix them at three in the morning.

A framework is scaffolding, not architecture. Chosen before the shape of the building is known, it becomes the shape.

So the useful comparison has at least two columns, and the second one is the one nobody prints. Here is what four implementations of one task actually cost — and the constraint that decided the whole thing before a line was written.

Working code: stages/s09_frameworks/ at tag stage-09. 28 checks, 12 of them on failure modes. Runs offline, no API key.

The most expensive finding happened before the first line of code

pip install "crewai>=0.60"
ERROR: Could not find a version that satisfies the requirement crewai>=0.60

Precision matters here, because the imprecise version of this sentence is refuted by one command. pip download crewai on Python 3.14 succeeds and hands you 0.11.2: old releases declare >=3.10,<4.0 and install fine. The <3.14 ceiling starts at 0.14.0.

And 0.14.0 is exactly where crewai.BaseLLM and crewai.tools.BaseTool appear — the extension points through which this stage passes its own client. So the choice is: a version that installs and has nothing to plug your client into, or a version with the API that will not install.

This is not a failure of the exercise. It is the sharpest of the constraints the exercise teaches you to see: it decides the choice first, before elegance is even a question. And no framework comparison shows it, because every one of them is written on a machine where the install worked.

The side effect turned out worse than the incompatibility. The optional dependency group failed as a whole without a marker, taking LangGraph — which installs perfectly — down with it. The instructions punished obedience: a reader who followed them got nothing, and a reader who ignored them got a working stage.

crewai>=0.60,<2; python_version < '3.14'

One marker. The group now installs what can be installed and says what cannot.

What makes the comparison honest

If each implementation is free to define the task, the table measures the author’s willingness to cut corners. The problem is not dishonesty, it is invisibility: a CrewAI implementation naturally wants one more model call to “coordinate”, and nothing in a hand-written comparison notices.

So the task contract is code, and every implementation is asserted against it: same input, same tools, same model, same stopping condition, same result shape. Written as prose in a README it catches nothing, because prose is not run.

One thing the contract deliberately does not fix: the number of model calls. Capping it would hide precisely the cost some frameworks impose — an extra coordination round-trip is a real difference and belongs in a column, not in a rule that forbids it.

The table

Two rows are unrun and stay in the table with their reason. Deleting them would throw away the only honest numbers about CrewAI there are — and “not evaluated” is a state, not a hole.

0 against 1895. That is how many lines of the package executed on this input. Not how many were installed — how many ran. The distinction matters: measuring installed size counts code paths this task never touches, and that number is unfalsifiable.

The measurement is line-level tracing, not package size, and it warms up first: the first call in a process also executes the module bodies of everything imported lazily, which is a per-process cost, not a per-request one. On this bench, skipping the warm-up inflated the number sevenfold.

Explicit against implicit coordination, as a number

“Implicit coordination is cheaper” is half true, and it is the half you pay for later. The other half is measurable:

no framework      0 prose places
LangGraph         0 prose places
CrewAI           10 prose places
Google ADK        1 prose place

It counts named arguments whose values describe behaviour in prose: role, goal, backstory, description, instruction, expected_output. Zero means the next step is decided by code. Ten means that answering “why did this step run” requires reading ten descriptions and imagining how a model read them.

Explicit coordination costs lines. Implicit coordination costs understanding — and it charges you at the worst possible moment, during an incident.

This one is measured from the source, not from a run, which is why the rows that cannot execute still have it: an interpreter constraint does not make code unreadable. A declared number would be a flag the author sets by hand — the exact defect the previous stage is about.

A trap that reported zero and looked correct

The counter that measures invisible lines found the package by module origin. For a namespace package — langgraph is one — origin is None. So the column read 0, the table printed, the numbers were whole integers, and the framework appeared to cost nothing at all.

That is the shape of the dangerous defect: not a crash, a plausible number. The fix looks at submodule_search_locations before origin, and there is now a check that asserts the counter reports non-zero for a package known to execute — because “measured zero” and “failed to measure” must not print the same way.

A second one, subtler: overhead tokens were counted from what the implementation asked for, which is exactly the number that misses the overhead. A framework that quietly appends its own preamble asks for more, and a counter living inside the implementation sees the enlarged request as the baseline. The counter now sits at the provider boundary, and the mutation exercise that moves it back inward shows two of three observer positions undercounting by 100% — both in the direction of “the framework is cheaper”.

There is no combined score, and that is the point

The table has no total column. A weighted score requires weights, and weights are an opinion about whose constraint matters more, baked into a number nobody agreed on.

The conclusion has a different shape:

The baseline row exists for the same reason. Without an implementation that uses no framework at all, the comparison answers “which one” and never “whether any”.

What this deliberately does not prove

• Two of the four were not run. CrewAI with the required API does not install on this interpreter, and Google ADK is in the same position here. Their columns say not evaluated, and everything said about CrewAI’s runtime behaviour is expectation rather than measurement — labelled as such in the table.
• These numbers do not transfer to another task. A two-step task with no branching is the worst possible case for a graph orchestrator, and saying so is part of the result.
• Invisible lines describe this input and this thread. The tracer does not see other threads.
• Nothing here measures answer quality. All four produce the same result by construction; that is what makes the costs comparable.

Numbers

28 checks, 12 of them on failure modes
10 mutation exercises, each red in the check that claims it
37 lines with no framework against 54 + 1895 with one

The four implementations, the executable task contract, the counter at the provider boundary and the 10 mutation exercises are in stage 9 of the course repository, pinned to the tag. It runs offline, without an API key — and it will tell you honestly which rows it could not run on your machine.