TechVidvan

Architecture of TensorFlow – Components and Working

TensorFlow is an open-source platform that works on machine learning and deep learning. It works with graphs, nodes, and it performs computations for different models using it.

TensorFlow allows the implementation of many algorithms while using the existing ones to train the system. In this article, we will learn about TensorFlow Architecture and its components.

 

Architecture of TensorFlow

The system of TensorFlow, which finds its use for machine learning, is also known as TensorFlow Serving. The architecture works in three significant steps:

1. Data pre-processing – Data collection process brings unstructured data. Hence the process of pre-processing makes it structured and brings it under one limiting value.

2. Model building – Build the model for the data.

3. Train and estimate the model – Use the data to train the model. Run the model on epochs to increase the accuracy and reduce the loss. Now test the model with unknown data.

Requirements of TensorFlow

The requirements of the library can be split into two dimensions:

a. Development Phase – Requirements of the system when the training of the data is done.

b. Run Phase – Running the model after training it requires a platform, processing unit, and language support.

The architecture of TensorFlow Serving is quite distinct, which contains many terms of the novel, such as servable, servable versions, streams, models, loaders. We will look into each term with some briefing:

1. TensorFlow Servable

It is the central unit in the architecture of TensorFlow Serving which acts as objects. Clients use it in the process of computations. It offers flexible size and granularity and consists of one lookup table to a tuple of interference models. Typical Servables contain the following:

  • a TensorFlow SavedModelBundle (tensorflow::Session)
  • a lookup table for embedding or vocabulary lookups

It improves the functionality and performance with the help of improvements:

  • streaming results
  • experimental APIs
  • asynchronous modes of operation

2. Servable Versions

The architecture of TensorFlow Serving consists of one or more than one version in a single-server interface. This offers weights, new algorithms, and other data to start loading again over time.

It supports rollout and experimentation by executing multiple versions of servable which load concurrently. Clients can request for the latest version or version id to the servers for a model.

3. Servable Streams

It is the ascensive order of versions put in higher-order versions numbers.

4. Models

The combination of one or more Servables is known as models. A model consists of algorithms and lookups or embedding tables. A composite model consists of :

  • multiple independent Servables
  • one composite Servable

A model may comprise different fractions of Servable. For example, A lookup table can be instantiated to multiple instances of Servable.

5. Loaders

It manages the life cycle of Servable. The loader API enables an infrastructure that is independent of data or product use cases and algorithms. The standardization of loader’s API is for loading and unloading of Servable.

6. Sources

Sources module provides the plugin Sources, which provide one or multiple servable streams. For each stream, Source provides one Loader for each version. Servables can detect servables from random storage systems.

Sources manage the state of servables or versions. Sources come in use where there are delta updates between versions of servable.

7. Managers

The life cycle of Servables is run by Managers starting from loading Servables, serving Servables, unloading Servables. Managers communicate with the Sources to trace the versions. Managers listen to the requests of Source requests.

A manager manages the action of unloading and loading, for e.g., if a new version is still loading, Manager can hold the action of unloading.
Managers provide a GetServableHandle() interface for the clients to access loaded servable instances.

8. Core

Core looks at the lifecycle and metrics of Servables. It treats Servables and Loaders as opaque objects.

9. Lifecycle of Servable

It consists of the following steps:

  1. Source creates a Loader. It contains the metadata to load the Servable.
  2. Source notifies the Manager of the aspired version.
  3. Manager finds the next action on the basis of the aspired version policy to load/unload a version.
  4. If the Manager finds it safe, it provides resources to Loader for loading the new version.
  5. On the request of a Client with a specific version or version id, the Manager returns the relevant servable.

10. Extensibility

TensorFlow provides some extension gateways to append the functionality-

a. Version Policy

In single servable streams, the management of loading and unloading sequence is done by version policy. It provides two policies for use cases.

One is Availability Preserving Policy, which loads the new version before unloading an old version. And another is Resource Preserving Policy, which avoids the loading of two versions simultaneously.

b. Source

TensorFlow allows the creation of new sources for adding algorithm backends and cloud offerings.

c. Batcher

For the better cost performance, TensorFlow allows summing up of all requests as one. It helps to accelerate hardware like GPUs. It allows clients to merge their specific type requests, which can be processed easily by the algorithm.

Summary

The TensorFlow architecture is a surplus system of components that adds up to the platform’s functionality. It contains a variety of functional mechanism elements that act together to bring a high order user-friendly system.

It is a very flexible system that serves the purpose of machine learning, providing its compatibility with the environment. TensorFlow Serving works with many models at a time and allows creating new algorithms with ease.

Its API is a scalable system that can help to deploy models as well as work on various product algorithms.