Fast and Reliable Swift APIs with gRPC

Fast and Reliable Swift APIs with gRPC
Tim Burks
Google, Inc.

I’m talking about Networked APIs.
Application Programming Interfaces that operate across a network of
computers. They communicate using network protocols including HTTP, and
are frequently produced by different organizations than the ones that consume
them.
Google API Design Guide: Glossary

I hate SDKs.
Closed-Source SDKs:
● Snoop user or developer data.
● Do other unauthorized things.
● Fail to build or run.
Open-Source SDKs:
● Also fail to build or run.
● Introduce dependencies that I don’t want.
● Are just ugly and gross.

but now I make
SDKs…
● google-cloud-python
● google-cloud-node
● google-cloud-ruby
● google-cloud-java
● google-cloud-go
● google-cloud-php
● google-cloud-dotnet
● google-api-python-client
● google-api-nodejs-client
● google-api-ruby-client
● google-api-java-client
● google-api-go-client
● google-api-php-client
● google-api-dotnet-client
● google-api-objectivec-client
● google-api-objectivec-client-for-rest

Protocol Buffers
a language-neutral, platform-neutral, extensible mechanism for serializing structured data.

“Protocol Buffers” means several things
1. A serialization mechanism
2. An interface description language
3. A methodology

Protocol Buffer Serialization
It’s just a stream of bytes
[field_number<<3 + wire_type] [length if necessary] [data]...
$ hexdump /tmp/request.bin
0000000 0a 05 68 65 6c 6c 6f
0a is “0000 1010”, so
field_number = 1 and wire_type = 2

The Protocol Buffer Language
package echo;
message EchoRequest {
string text = 1;
}
message EchoResponse {
string text = 1;
}
service Echo {
rpc Get(EchoRequest) returns (EchoResponse) {}
rpc Update(stream EchoRequest) returns (stream EchoResponse) {}
}

The Protocol Buffer Methodology
$ protoc echo.proto -o echo.out --swift_out=.
$ which protoc-gen-swift
../bin/protoc-gen-swift
$ more echo.pb.swift
// DO NOT EDIT.
//
// Generated by the Swift generator plugin...
// Source: echo.proto

Interface Builder for Data
message Person {
string name = 1;
int32 id = 2;
string email = 3;
message PhoneNumber {
string number = 1;
}
repeated PhoneNumber phone = 4;
}
Interface Builder: Developers specify
their interfaces using a special tool,
tooling compiles and integrates that into
their apps.
Protocol Buffers: Developers specify
their data structures using a special
language, tooling compiles and
integrates that into their apps.

10 billion+
API calls
every second

Interoperability
Java
Service
Python
Service
GoLang
Service
C++
Service
gRPC
Service
gRPC
Stub
gRPC
Stub
gRPC
Stub
gRPC
Stub
gRPC
Service
gRPC
Service
gRPC
Service
gRPC
Stub

Nonstreaming APIs
a.k.a. “Unary”
Client sends one request.
Server sends one
response.
message HelloRequest {
string name = 1;
}
message HelloReply {
string message = 1;
}
service HelloService {
rpc SayHello(HelloRequest) returns (HelloReply);
}

Streaming APIs
Client-Streaming
Client sends multiple
messages.
Server sends one
response.
Server may choose to send
the response before all the
client messages are received.
message Latency {
string name = 1;
double val = 2;
}
message Histogram {
string name = 1
double p99 = 2;
double p999 = 3;
double avg = 4;
}
service MetricsService {
rpc ReportLatency(stream Latency) returns Histogram;
}

Streaming APIs
Server-Streaming
Client sends one request.
Server sends multiple
messages.
message Stock {
string stocksymbol = 1;
}
message Quote {
double price = 1;
}
service StockTickerService {
rpc GetTicker(Stock) returns (stream Quote);
}

Streaming APIs
Bidirectional Streaming
Client and server can
send multiple messages
to each other.
Client and server can send
messages independently;
neither has to wait to receive
a message before sending a
message.
message Message {
string text = 1;
}
service ChatService {
rpc Chat(stream Message) returns (stream Message);
}

gRPC Implementations: Native
C/C++, Java, Go
gRPC Core
HTTP 2.0
SSL
Code Generated API
Application Layer
Framework Layer
Transport Layer

gRPC Implementations: Wrapped
C#, Node.js, Ruby,
PHP, Python, Obj-C,
Swift
Python
Obj-C, C#, C++,
...
Ruby PHPPython
C gRPC Core
HTTP 2.0
SSL
Language Bindings
Code Generated
Ruby PHP
Obj-C, C#,
C++,...
Application Layer
Framework Layer
Transport Layer

gRPC Sample Service
package echo;
message EchoRequest {
// The text of a message to be echoed.
string text = 1;
}
message EchoResponse {
// The text of an echo response.
string text = 1;
}
service Echo {
// Immediately returns an echo of a request.
rpc Get(EchoRequest) returns (EchoResponse) {}
// Splits a request into words and returns each word in a stream of messages.
rpc Expand(EchoRequest) returns (stream EchoResponse) {}
// Collects a stream of messages and returns them concatenated when the caller closes.
rpc Collect(stream EchoRequest) returns (EchoResponse) {}
// Streams back messages as they are received in an input stream.
rpc Update(stream EchoRequest) returns (stream EchoResponse) {}
}

Running the generator(s)
$ ls
echo.proto
$ protoc echo.proto --swift_out=. --swiftgrpc_out=.
$ ls
echo.client.pb.swift echo.proto swiftgrpc.log
echo.pb.swift echo.server.pb.swift
echo.proto Protocol Buffer language source file
echo.pb.swift generated by protoc-gen-swift
echo.client.pb.swift generated by protoc-gen-swiftgrpc
echo.server.pb.swift generated by protoc-gen-swiftgrpc
swiftgrpc.log generated by protoc-gen-swiftgrpc

gRPC Swift server protocol (generated)
// To build a server, implement a class that conforms to this protocol.
public protocol Echo_EchoProvider {
func get(request : Echo_EchoRequest, session : Echo_EchoGetSession) throws
-> Echo_EchoResponse
func expand(request : Echo_EchoRequest, session : Echo_EchoExpandSession) throws
func collect(session : Echo_EchoCollectSession) throws
func update(session : Echo_EchoUpdateSession) throws
}

gRPC Swift server sample (handwritten)
// get returns requests as they were received.
func get(request : Echo_EchoRequest, session : Echo_EchoGetSession) throws
-> Echo_EchoResponse {
return Echo_EchoResponse(text:"Swift echo get: " + request.text)
}
// update streams back messages as they are received in an input stream.
func update(session : Echo_EchoUpdateSession) throws -> Void {
while true {
do {
let request = try session.Receive()
try session.Send(Echo_EchoResponse(text:"Swift echo update: (request.text)"))
} catch Echo_EchoServerError.endOfStream {
break
}
}
try session.Close()
}

gRPC Swift unary client sample (handwritten)
var requestMessage = Echo_EchoRequest(text:message)
let responseMessage = try service.get(requestMessage) // blocking
print("get received: " + responseMessage.text)

gRPC Swift streaming client sample (handwritten)
let sem = DispatchSemaphore(value: 0)
let updateCall = try service.update() // blocking
DispatchQueue.global().async {
while true {
do {
let responseMessage = try updateCall.Receive() // blocking
print("Received: (responseMessage.text)")
} catch Echo_EchoClientError.endOfStream {
sem.signal()
break
}
}
}
let parts = message.components(separatedBy:" ")
for part in parts {
let requestMessage = Echo_EchoRequest(text:part)
try updateCall.Send(requestMessage)
sleep(1)
}
try updateCall.CloseSend()
// Wait for the call to complete.
sem.wait()

Try it!
https://github.com/grpc/grpc-swift/wiki

To Do:
● Build Systems (currently Swift PM only)
○ Wanted: CocoaPods, Carthage
○ Xcode: `swift package generate-xcodeproj`
● gRPC-Core code is vendored. Can this be organized better?
● Need to build and pass gRPC interoperability tests.
● Increased metadata access.
● Other issues at https://github.com/grpc/grpc-swift/issues.

To Do: Wrap Google Datastore API
● Entity-oriented NoSQL data store, heavily used by Google App Engine apps.
● Usually mapped to native structs:
// golang
type Task struct {
Category string
Done bool
Priority int
Description string `datastore:",noindex"`
PercentComplete float64
Created time.Time
Tags []string
}
Swift Mirrors and Decoders?

To Do: Auth
● https://github.com/google/auth-library-swift (available)
○ MacOS: Gets OAuth tokens from a browser
○ Linux on GCP: Gets OAuth tokens from the GCP environment
● To Do: Support Service Accounts
○ Requires JWT signing with RS256

Build a gRPC service, get a free REST API!

Fast and Reliable Swift APIs with gRPC

More Related Content

Fast and Reliable Swift APIs with gRPC