Go back

From Manual Proto Hunting to Buf Sanity

This post was heavily helped by “Protobuf dependency management with Buf”, so big thanks to Mike for hosting the tutorial.

Repository of my project: CynicDog/pokedex-grpc

In the summer of 2025…

I was sweating over a gRPC xDS server in Python — aiming to dynamically configure network policies via Envoy sidecars, all running inside Docker. It sounded manageable in theory, until I met Protocol Buffers.

I didn’t know much about .proto files — and I knew even less about the sprawling dependency graph behind Envoy’s APIs. Still, I dove in, collecting schemas like:

…and quickly hit a wall. Each proto had imports. Those had their own imports. Suddenly I was juggling files like buf/validate/validate.proto, google/protobuf/descriptor.proto, and files buried in half a dozen different GitHub repos.

It worked — barely. After manually wrangling folders from Google, UDPA, Envoy, and others, I finally compiled Python bindings using protoc. But it was exhausting.

Buf to the Rescue

I knew this couldn’t scale, so I went looking. That’s when I found Buf — a toolkit that makes Protobuf dependency management a first-class citizen.

More importantly, I discovered Buf Schema Registry (BSR) — a centralized place to publish and import proto modules with versioning, docs, and tools. The idea? Stop copying GitHub folders. Just reference remote, versioned protos — and let buf handle the rest.

Building a Pokémon gRPC API

To put it into practice, I built a lightweight gRPC server and client around Pokémon data. The schema was defined in a file called pokedex.proto, which imports types from the pixelperfect/gametypes module on Buf.

I set up the project using uv, a fast Python environment and packaging tool. It handled my dependencies, created a lockfile, and kept everything reproducible.

Inside a proto/ directory, I initialized a Buf module, declared my dependencies in buf.yaml, and added a buf.gen.yaml file to define how Buf should generate Python code — both for protobuf classes and the gRPC service interfaces.

Once everything was wired up, I ran Buf’s generator. The result: a clean set of .py and .pyi files in a gen/ folder, fully ready to import into Python.

Catching Pikachu in gRPC

On the server side, grpc_server.py implements the PokedexService. When it receives a GetPokemon request, it returns a hardcoded Pokemon object — in this case, Pikachu — built using the external types from Buf’s gametypes module.

The client, grpc_client.py, sends a request for a Pokémon and prints the response. It uses the auto-generated PokedexServiceStub class to make the call and unmarshal the result.

To try it out, I set the PYTHONPATH to include the generated code folder, started the server in one terminal, then ran the client in another. The result: a full Pokémon object with name, types, abilities, moves, and base stats — unmarshalled and printed neatly using Python’s protobuf bindings.

uv: Packaging That Doesn’t Fight Back

For me, Python environments have always been, frankly, painful. venv and pip clunky and inconsistent, and don’t even get me started on Conda.

But uv changed that. It’s built in Rust, fast, and reliable. There are experts who know far better than me about uv, but simply put: it uses Rust to deliver fast, reliable Python environment and dependency management with deterministic locking that prevents version conflicts and ensures repeatable installs.

Instead of wrestling with confusing virtualenvs, fragmented package installs, and managing multiple config files, uv bundles all that pain into one simple, fast, and reliable tool.

Final Thoughts

This project started with chaos: manually downloading proto files, untangling dependencies, and praying that protoc would run without exploding.

But it ended with something clean and sustainable: remote proto modules via Buf and fast environment management via uv has truly enhanced my development joy.

Takeaway: If you’re building gRPC services in Python, don’t fight your protos and Python project setups. Let Buf and uv do the heavy lifting.


Share this post on:

Next Post
GitHub as a Web Application for Agile Development