Oracle just introduced its new machine learning model solution, GraphPipe. The product is meant to provide a standard, high-performance protocol for transmitting tensor data over its network and simplify the implementation of clients and servers. Currently, Oracle GraphPipe servers are equipped to serve models built in TensorFlow, PyTorch, mxnet, CNTK, or caffe2. Here’s some more information about the new offering and its relevant features.
Oracle GraphPipe Implementation
The efficient network protocol is designed specifically to standardize machine learning data transmissions and simplify the entire process. Currently, there isn’t really a dominant standard for transmitting tensor-like data between components in deep learning architectures. So it’s common for developers to use protocols built for other purposes, like JSON. And those can be pretty inefficient for those this type of application. The other solution people tend to use today is TensorFlow-serving’s protocol buffers. But those carry the baggage of TensorFlow’s complex software. So GraphPipe is made to combine the efficiency of a binary, memory-mapped format with the simplicity of other solutions.
To do this, GraphPipe includes a set of flatbuffer definitions, guidelines for serving models consistently according to those flatbuffer definitions, examples for serving different models — including those from TensorFlow, ONNX, and caffe2, and client libraries for querying models that have been served with GraphPipe. So basically, Oracle GraphPipe requests act like TensorFlow-serving predict requests, but with flatbuffers as the message format.
To actually perform, the tool compares the serialization and deserialization speeds of float tensor data in python using a custom API, protocol buffers from a TensorFlow predict request, and a GraphPipe remote request. Each request consists of different floating-point values (normally around 19 million), which consist of more than 100 images. That creates millions of floating point values. Then, the tool compares end-to-end throughput with a Python-JSON TensorFlow model server, TensorFlow-serving, and GraphPipe’s TensorFlow model server.
If you want to learn more about Oracle GraphPipe, you can check out Oracle’s GitHub. The company is also providing its flatbuffer spec online and servers to help you implement that spec for both Python and Go.
Featured image: Freerange Stock