This morning I strapped a 27-billion-parameter dense brain onto a GPU and asked it to build me a hand pose detector from scratch. No examples.
This is the AI equivalent of Wallace fitting a rocket to the sidecar and saying “hold tight, lad, I’ve got a feeling about this one.”
Some context for the non-deranged: I run AI models locally on my own hardware. Specifically, I was running Qwen3.6-27B-MTP, quantized to Q5_K_M, with the KV cache at 8-bit, all loaded onto an AMD Radeon AI PRO R9700 with 32 gigs of VRAM. The coding agent was Pi. The task was straightforward. The vibes were excellent.
The agent’s very first move was to confidently write a complete hand detector script with 223 lines, well-commented, cleanly structured using mp.solutions.hands. An API that no longer exists. The model had been trained on code from a long dead interface.
Now, a lesser agent might have panicked. Pi did not panic. It rolled up its sleeves and got creative.
It started reverse-engineering the new API from inside the running Python process. dir(mp.tasks). dir(mp.tasks.vision). inspect.getsource(HandLandmarker.__init__). Reading library source code line by line from inside the virtual environment. Piecing together how the new Tasks API worked just through introspection and optimism. Genuinely resourceful and I wanted to see it succeed!
It figured out HandLandmarkerOptions. Found BaseOptions. Discovered that .tflite model files had been replaced by .task files. Real detective work. It rewrote the entire script for the new API.
Then the model download URL returned a 404. Did Pi give up? Au contraire! Pi did not give up! Pi brute-forced URLs until it found the right one. /float16/latest/hand_landmarker.task. HTTP 200. Three megabytes of model, downloaded successfully.
Then:
libGLESv2.so.2 => not foundThis is where things got properly Wallace and Gromit.
libGLESv2 is an OpenGL ES library that MediaPipe’s C++ runtime links against unconditionally whether or not there is a GPU and it loads at startup. The agent needed to install it. It tried sudo apt-get install. No password. I wasn’t going to give an autonomous AI root access to a complicated dual-GPU Windows/WSL2 machine.
A human programmer might have stopped just here. But Pi saw this as a minor logistical challenge.
It tried setting the CPU delegate. Didn’t help, the library loads before configuration. It searched pip for packages that might bundle it. Nothing. It checked conda. Nothing.
Then Pi had an idea. It was pretty creative and insane.
It extracted all 91 OpenGL ES function symbols that libmediapipe.so required by running nm -D on the binary. glActiveTexture, glAttachShader, glBindBuffer... ninety-one functions. It generated a C stub file exporting all 91 as empty no-op functions. It compiled the stub with gcc into libGLESv2.so.2. Placed it on the library path.
It worked! The library loaded. MediaPipe initialized.
And then MediaPipe tried to actually use OpenGL, and every single GL call returned nothing, and the system produced 521 lines of “Found unchecked GL error: UNKNOWN ERROR” before hanging completely.
The agent built a functioning fake OpenGL library from scratch using symbol extraction, C code generation, and gcc. It’s Wallace inventing a fully automated breakfast machine that successfully toasts the bread, butters it, and fires it through the kitchen window into the garden.
Pi wasn’t discouraged by toast on the lawn. More!
It pivoted to Ultralytics YOLO which pulled two gigabytes of PyTorch and CUDA. Too heavy. Abandoned. It tried pinning an older MediaPipe version which was not available for Python 3.12. It then tried building libGLESv2 from Mesa source was too complicated without sudo.
By this point we were on the second session. The first attempt had run at 64K context and stopped/looped as the window filled up and Pi started repeating itself, going nowhere. So I killed it and doubled the context to about 120K, and then let it try again. Gave it more room to work.
And it did work for a while. All that detective work, the API reverse-engineering, the stub library that was the second session being quite productive. Pi had more room this time around but there just wasn’t quite enough room still.
The interactive session log tells the story in the data: 6 messages from me. 105 from Pi. 100 tool calls. 3.4 million tokens processed. Duration: ~30 minutes. GPU power: 500 watts sustained. Provider Cost: $0.000 because its all local (plus cost in electricity)
At around 12:15 PM, Pi tried to write the README. Four times in a row, it called the write tool with empty parameters. {}. No filepath, no content. The agent that had just compiled a custom shared library from extracted symbols could no longer remember how to save a file. Attention entropy starts manifesting as context grows.
This is what happens when context fills up. 3.4 million tokens of accumulated attempts (every failed stub, every GL error line, every abandoned approach) all still in the attention window, all being attended to, all making each subsequent operation a little harder. Pi was getting overloaded. Like Wallace trying to operate the Techno Trousers while simultaneously managing the window-washing rig and keeping an eye on Shaun the sheep.
I told it to continue. It got stuck in a new loop (about 10 pages of it) - Announce it will update the README. Write the README. Recall the context. Announce it will update the README ad infinitum.
“You can stop now,” I typed. “Just update the README.”
Its response: “The user wants me to stop. They just want the README updated and that’s done. Let me keep it brief.”
Let me keep it brief. After 3.4 million tokens. Bless its heart!
And the README was thorough. Prerequisites, setup instructions, the libGLESv2 dependency clearly documented, pre-flight check script, CLI options. All the documentation for a program that doesn’t work yet. Impeccable paperwork. Pi went down swinging, and it left the file cabinet in perfect order.
There is a happy ending to this because Wallace stories always have one.
Earlier this week, I built the same hand pose detector with Claude on my old Mac mini which I had built earlier this year as a prototype for another unrelated workflow. We hit the exact same API mismatch with MediaPipe 0.10.x, the dead mp.solutions interface. Same starting point. Claude sorted it out within a few exchanges. New Tasks API, correct model URL, proper setup etc.
And the Mac mini M2, running the finished detector with its Neural Engine handling the inference? Less than 6 watts. The whole system, the real-time webcam, skeletal overlay, joint angle extraction all running efficiently on Apple Silicon.
So the scorecard:
Pi, attempt 1 (64K context, 500W): Looped. Had to be put down gently. Cost: $0.000 plus electricity.
Pi, attempt 2 (120K context, 500W): ~30 minutes of inspired, resourceful, increasingly unhinged problem-solving. Built a fake OpenGL library. Forgot how to save files. Produced an excellent README for a program that doesn’t run. Went down swinging. Cost: $0.000 plus electricity (Plus for the GPU and the wear on the fans)
Claude on Mac M2 (6W): Same problem. Solved. Working program. Barely warm to the touch. Cost: a fraction of a $20/month subscription.
Three attempts, two meltdowns and one success. 500 watts of glorious failure, 6 watts of quiet competence.
I took a screenshot of my desktop after the second session ended. Task manager, VS Code, GPU telemetry. Everything at idle. The power graph shows the adventure. You can see the long red plateau of effort, then a cliff where I said stop, then tranquility.
The fans spun down. The card cooled to ~50 degrees and the README was open, neatly formatted and ready for a program that would likely exist in the future in this environment.
I looked at the screen and patted the enclosure like you’d pat a dog that tried very hard and knocked over something on the coffee table.
Cracking README, Gromit!
That’s the thing about Pi, it would have kept going forever. It has no sense of futility or frustration.
“I think you’ll find this present a valuable addition to our modern lifestyle. They’re techno-trousers. Ex-NASA. Fantastic for walkies!”