Skip to main content

What is an RPC

note

If you're familiar with RPCs, especially with Google Protocol Buffers, you can safely skip this page.

RPC is short for Remote Procedure Call.

"In distributed computing, a remote procedure call (RPC) is when a computer program causes a procedure (subroutine) to execute in a different address space (commonly on another computer on a shared network), which is coded as if it were a normal (local) procedure call, without the programmer explicitly coding the details for the remote interaction."

Wikipedia - Remote Procedure Calls

A REST endpoint (a specific HTTP endpoint that accepts in, say JSON and returns JSON) is an RPC, albeit a rather primitive one. In the case of REST, the URL itself uniquely identifies the procedure you're trying to execute, like https://example.com/user/add. In the case of REST these endpoints typically accept in JSON of a specific shape.

Google Protocol Buffers#

Maglev uses Google Protocol Buffers. Protobuf (sometimes shorted to just proto) is four separate things:

  • A formal language to define not only RPC names, but also the EXACT structure of the arguments each expects to take in, and will return. It also supports things like streaming, and a host of other cool things I won't go into here.
  • A format (binary) encoding for any structure defined in proto.
  • A compiler that can parse the above language (proto3 in Maglev's case). What this compiler does with that information is application specific. In Maglev's case it does two things:
    • Creates a chunk of binary data (encoded as a base64 string) that the Maglev core can use at runtime to encode/decode and dispatch RPCs.
    • Create Typescript typing definitions for strong typing
  • A runtime library that knows how to encode/decode binary protobuf data (often called "serialization") as well as dispatch RPCs.

This is a little confusing when you first using things like Protocol Buffers, but the advantages quickly become apparent.

Maglev defines RPCs in Google Proto3 Format. This is normally defined in files ending with .proto, generally all contained within a single directory called protos at the root of the project directory.

Why Not HTTP/JSON#

There are innumerable issues with HTTP/1, but lets compare the power and flexibility Maglev offers with a traditional HTTP REST endpoint. Let's say we want to build a simple web app that simply responds to server-side events.

We could setup an HTTP endpoint that accepts and returns some JSON and pull that endpoint every N seconds to see if the event has happened. That's obviously not ideal, so we could make this slightly less horrible by long-pulling each request (the server doesn't respond to the request until the event 'fires' or a short timeout expires). Great, problem solved. Except what happens if the server tries to send us events faster than we can round-trip an HTTP call (which are very expensive to make, by the way). Okay let's modify the API to return back N events on each request.

This isn't ideal, and as you add on more and more async two way communication it quickly becomes untenable. For this reason web introduced WebSockets, two-way communication between a server and a client. Except now we need a way to distinguish if the message we just got from the server is the above event, or some other chunk of data it's trying to send us, or even a response to a request we made earlier. Easy enough to solve, add a 'type' field to your JSON and check it in a big if-statement. Congratulations, you just wrote a very primitive RPC system. It's riddled with very serious issues, but that's essentially what an RPC is. But what happens when you instead want to receive events from a machine that can't open a public port like a server can?

Maglev does away with all of this complexity. You define an RPC handler (akin to the code in a specific if-else branch above) and that handler will automatically be called for you with an already deserialized structure. How the bytes got to the machine, you don't need to care. How they were structured, you don't need to care. What endian the other machine is, you don't need to care. Just run your business logic, and return back an object to Maglev, it will handle turning it into bytes and getting it back to the caller. There are no "servers" and "clients" in Maglev, any node can invoke and RPC on another node (assuming it has permission to do so).