{"id":86942,"date":"2023-02-10T09:00:13","date_gmt":"2023-02-10T03:30:13","guid":{"rendered":"https:\/\/techvidvan.com\/tutorials\/?p=86942"},"modified":"2023-02-10T09:00:13","modified_gmt":"2023-02-10T03:30:13","slug":"serialization-in-python","status":"publish","type":"post","link":"https:\/\/techvidvan.com\/tutorials\/serialization-in-python\/","title":{"rendered":"Serialization in Python"},"content":{"rendered":"

Serialization, as it relates to data storage, is the act of converting a data model or entity status into a representation that can be transferred and later reassembled, or saved (for instance, in a disk or ram buffer).\u00a0<\/span><\/p>\n

An item is changed into a storable format known as serialization so that it can later be deserialized and restored to its original form using the serialized format.<\/span><\/p>\n

We use the procedures and methods from the Python Pickle module when we wish to serialize and de-serialize a Python object. The process of turning a Py object into a stream of bytes is termed pickling.<\/span><\/p>\n

This is often referred to as “serialization,” “marshaling,” or “flattening.” Pickling is the opposite, changing a binary code or object that resembles bytes into an object from a stream of bytes.<\/span><\/p>\n

Why is Serialization required in Python?<\/span><\/h3>\n

The process of serialization involves putting the entity into a format suitable for storage or transmission. It is quite handy for us to continue utilizing the saved object later on rather than having to recreate it from scratch after transmitting or retaining the serialized data because we can reconstruct the item later and get the same structure\/object.<\/span><\/p>\n

How to perform serialization in Python?<\/span><\/h3>\n

There are numerous serialization assets available in Python. The JSON file format is a famous example of hash maps that can be used with a variety of texts and is accessible to people. It allows us to maintain and rebuild the dictionary using a similar structure. But JSON can only keep texts and numbers, as well as simple structures like a list and dictionary. JSON cannot be asked to retain the type of data.\u00a0 Furthermore, it is unable to discriminate between lists and Python tuples.<\/span><\/p>\n

We will examine pickle a popular Python serialization library, in the section that follows.<\/span>\u00a0<\/span><\/p>\n

The pickle module, which is a component of the Python codebase, offers deserialization and serialization techniques for Python objects.<\/span><\/p>\n

Pickle must be imported into Python to begin:<\/span><\/p>\n

Input:<\/b><\/p>\n

# Importing the library\nimport pickle\n<\/pre>\n
We can then use the pickle’s dump() method to encode a Python object, such as a dictionary, and save the character sequence as a file.<\/span><\/p>\n
Input:<\/b><\/p>\n
dic = {\"Hii\": \"Everyone!\"}\nwith open(\"demo.pickle\", \"wb\") as outfile:\n      \t# Loading the document in binary form\n      \tpickle.dump(dic, outfile)<\/pre>\n
The file “demo.pickle” now contains the byte stream \u201cdic\u201d.We used the pickle’s load() method to read the serialized byte streams from the file to retrieve the original object.<\/span><\/p>\n
with open(“demo.pickle”, “rb”) as infile:<\/span><\/p>\n
Input:<\/b><\/p>\n
dic_new = pickle.load(infile)<\/pre>\n
Be careful of what site you get your information from. The site may have content that is dangerous to your computer. It can do bad things to your computer while you are getting the information.<\/span><\/p>\n
By combining them, the following code demonstrates that Pickle may retrieve the same entity: <\/span><\/p>\n
Input:<\/b><\/p>\n
import pickle\n# Demo entity\ndic = {\"Hii\": \"Everyone!\"}\n# Serializing process\nwith open(\"demo.pickle\", \"wb\") as outfile:\n    pickle.dump(dic, outfile)\nprint(\"Reading entity\", dic)\n# Deserialization\nwith open(\"demo.pickle\", \"rb\") as infile:\n    dic_new = pickle.load(infile)\nprint(\"Rebuilding entity \",dic_new)\nif dic ==dic_new:\n    print(\"Successfully rebuilt\")\n<\/pre>\n
We may acquire the serialized object in Python as an array of bytes type using pickle’s dumps() function in addition to writing the serialized data into a pickle file:<\/span><\/p>\n
Input<\/strong>:<\/span><\/p>\n
array_dic = pickle.dumps(dic)<\/pre>\n
Likewise, we may change from an array of bytes type to the primary object using Pickle’s load method:<\/span><\/p>\n
Input<\/strong>:<\/span><\/p>\n
array_dic_new = pickle.loads(array_dic)<\/pre>\n
Pickle’s ability to serialize nearly any Py entity, including user-defined ones like the ones below, is one of its many advantages.<\/span><\/p>\n
Input:<\/b><\/p>\n
import pickle\n class Demo:\n    def __init__(self, data):\n        print(data)\n        self.data = data\n# Build an object of class Demo\nDemo_obj = Demo(1)\n# Serialization and deserialization\ninfo = pickle.dumps(Demo_obj)\nrebuilt = pickle.loads(info)\n# Confirmation\nprint(\"Data from rebuilt entity:\", rebuilt.data)\n<\/pre>\n
Output:<\/b><\/p>\n
Data from rebuilt entity: 1<\/div>\n
Note that when pickle.loads() is called, the print statement from the class’ function Object() { [native code] } is not executed. This is because the item was rebuilt, not created.<\/span><\/p>\n
Given that Python functions are first-class objects, Pickle can even serialize Python functions:<\/span><\/p>\n
Input:<\/b><\/p>\n
import pickle\ndef fun():\n    return \"Hii Everyone!!\"\n# Serialization and deserialization\np_fun = pickle.dumps(fun)\nfun_new= pickle.loads(p_fun)\n# Confirmation\nprint (fun_new()) #prints \u201cHii Everyone!!\u201d\n<\/pre>\n
What can be pickled and unpickled in Python?<\/span><\/h3>\n
Python’s object structure is processed by a procedure known as pickling. During pickling, a Py entity is transformed into a stream of bytes. Unpickling is the method of recovering the original Python objects from the pickle file’s stored text version. The byte stream is changed into a Py entity.<\/span><\/p>\n
For instance, socket, file handler, database networks, and other items are typically not pickable. By default, pickling is an option for everything that is constructed (recursively) from fundamental Py categories (dicts, lists, primitives, entities, object pointers, even circular).<\/span><\/p>\n
You can write custom pickling code to, for instance, backup and restore the settings of a database server, but this requires unique, proprietary logic.<\/span><\/p>\n
How to ensure security<\/span><\/h3>\n
Pickle can be used to safeguard our work. Instead of retraining the model each time it is used, for instance, a learning algorithm via Keras or scikit-learn could be serialized by pickle and retrieved later. The next section demonstrates how we can use Keras to create a LeNet5 model to recognize the MNIST handwritten digits before serializing the model once it has been trained. The model can then be rebuilt without having to train it once again, and it must yield the same outcome as the initial formulation:<\/span><\/p>\n
\u00a0Input:<\/b><\/p>\n
# Importing Libraries\nimport pickle\nimport numpy as np\nimport tensorflow as tf\nfrom tensorflow.keras.datasets import mnist\nfrom tensorflow.keras.models import Sequential\nfrom tensorflow.keras.layers import Conv2D, Dense, AveragePooling2D, Dropout, Flatten\nfrom tensorflow.keras.utils import to_categorical\nfrom tensorflow.keras.callbacks import EarlyStopping\n \n# Loading the MNIST digits\n(XTrain, YTrain), (XTest, YTest) = mnist.load_data()\n \n# Reshaping the data\nXTrain = np.expand_dims(XTrain, axis=3).astype(\"float32\")\nXTest = np.expand_dims(XTest, axis=3).astype(\"float32\")\n \n# Encoding the output\nYTrain = to_categorical(YTrain)\nYTest = to_categorical(YTest)\n \n# LeNet5 model\nmodel = Sequential([\n    Conv2D(6, (5,5), input_shape=(28,28,1), padding=\"same\", activation=\"tanh\"),\n    AveragePooling2D((2,2), strides=2),\n    Conv2D(16, (5,5), activation=\"tanh\"),\n    AveragePooling2D((2,2), strides=2),\n    Conv2D(120, (5,5), activation=\"tanh\"),\n    Flatten(),\n    Dense(84, activation=\"tanh\"),\n    Dense(10, activation=\"softmax\")\n])\n \n# Training the model\nmodel.compile(loss=\"categorical_crossentropy\", optimizer=\"adam\", metrics=[\"accuracy\"])\nearlystopping = EarlyStopping(monitor=\"val_loss\", patience=4, restore_best_weights=True)\nmodel.fit(XTrain, YTrain, validation_data=(XTest, YTest), epochs=100, batch_size=32, callbacks=[earlystopping])\n \n# Evaluation of the model\nprint(model.evaluate(XTest, YTest, verbose=0))\n \n# Using Pickle for serialization and deserialization\np_model = pickle.dumps(model)\nnew_model = pickle.loads(p_model)\n \n# Evaluation\nprint(new_model.evaluate(XTest, YTest, verbose=0))\n<\/pre>\n
The output from the aforementioned code is as follows. The following two lines precisely tie together the test scores from the actual and recreated models:<\/span><\/p>\n
Output:<\/b><\/p>\n
Epoch 1\/100 1875\/1875 [==============================] – 48s 25ms\/step – loss: 0.1495 – accuracy: 0.9561 – val_loss: 0.0650 – val_accuracy: 0.9796 Epoch 2\/100 1875\/1875 [==============================] – 49s 26ms\/step – loss: 0.0640 – accuracy: 0.9805 – val_loss: 0.0502 – val_accuracy: 0.9835 Epoch 3\/100 1875\/1875 [==============================] – 47s 25ms\/step – loss: 0.0504 – accuracy: 0.9841 – val_loss: 0.0472 – val_accuracy: 0.9860 Epoch 4\/100 1875\/1875 [==============================] – 47s 25ms\/step – loss: 0.0406 – accuracy: 0.9870 – val_loss: 0.0443 – val_accuracy: 0.9861 Epoch 5\/100 1875\/1875 [==============================] – 46s 24ms\/step – loss: 0.0352 – accuracy: 0.9882 – val_loss: 0.0491 – val_accuracy: 0.9835 Epoch 6\/100 1875\/1875 [==============================] – 47s 25ms\/step – loss: 0.0303 – accuracy: 0.9900 – val_loss: 0.0379 – val_accuracy: 0.9868 Epoch 7\/100 1875\/1875 [==============================] – 46s 24ms\/step – loss: 0.0289 – accuracy: 0.9907 – val_loss: 0.0471 – val_accuracy: 0.9860 Epoch 8\/100 1875\/1875 [==============================] – 46s 25ms\/step – loss: 0.0252 – accuracy: 0.9919 – val_loss: 0.0390 – val_accuracy: 0.9872 Epoch 9\/100 1875\/1875 [==============================] – 46s 24ms\/step – loss: 0.0208 – accuracy: 0.9934 – val_loss: 0.0435 – val_accuracy: 0.9869 Epoch 10\/100 1875\/1875 [==============================] – 48s 26ms\/step – loss: 0.0206 – accuracy: 0.9930 – val_loss: 0.0376 – val_accuracy: 0.9890 Epoch 11\/100 1875\/1875 [==============================] – 47s 25ms\/step – loss: 0.0205 – accuracy: 0.9935 – val_loss: 0.0448 – val_accuracy: 0.9865 Epoch 12\/100 1875\/1875 [==============================] – 47s 25ms\/step – loss: 0.0182 – accuracy: 0.9939 – val_loss: 0.0402 – val_accuracy: 0.9884 Epoch 13\/100 1875\/1875 [==============================] – 47s 25ms\/step – loss: 0.0157 – accuracy: 0.9948 – val_loss: 0.0468 – val_accuracy: 0.9871 Epoch 14\/100 1875\/1875 [==============================] – 47s 25ms\/step – loss: 0.0134 – accuracy: 0.9958 – val_loss: 0.0416 – val_accuracy: 0.9877 [0.03760313242673874, 0.9890000224113464]<\/div>\n
Conclusion<\/span><\/h3>\n
Pickle is a strong library, but there are certain restrictions on what should be pickled. Live connections, like database servers and active file handles, for instance, cannot be pickled. Recreating these items creates a challenge because it requires the right credentials and is outside the range of what Pickle is meant for because it cannot establish the interaction with the database or directory for you.<\/span><\/p>\n
You learned what serialization is in this blog and how to employ Python libraries to serialize objects like dictionaries and Tensorflow Keras models. Additionally, you now know the benefits and drawbacks of the Python library(pickle).<\/span><\/p>\n","protected":false},"excerpt":{"rendered":"
Serialization, as it relates to data storage, is the act of converting a data model or entity status into a representation that can be transferred and later reassembled, or saved (for instance, in a...<\/p>\n","protected":false},"author":1,"featured_media":87039,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1053],"tags":[4849],"class_list":["post-86942","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-python","tag-serialization-in-python"],"yoast_head":"\nSerialization in Python - TechVidvan<\/title>\n<meta name=\"description\" content=\"Learn about serialization in Python and how to employ Python libraries to serialize objects like dictionaries and Tensorflow Keras models.\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/techvidvan.com\/tutorials\/serialization-in-python\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Serialization in Python - TechVidvan\" \/>\n<meta property=\"og:description\" content=\"Learn about serialization in Python and how to employ Python libraries to serialize objects like dictionaries and Tensorflow Keras models.\" \/>\n<meta property=\"og:url\" content=\"https:\/\/techvidvan.com\/tutorials\/serialization-in-python\/\" \/>\n<meta property=\"og:site_name\" content=\"TechVidvan\" \/>\n<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/TechVidvan\/\" \/>\n<meta property=\"article:published_time\" content=\"2023-02-10T03:30:13+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/techvidvan.com\/tutorials\/wp-content\/uploads\/2023\/01\/serialization-in-python.webp\" \/>\n\t<meta property=\"og:image:width\" content=\"1200\" \/>\n\t<meta property=\"og:image:height\" content=\"628\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/webp\" \/>\n<meta name=\"author\" content=\"TechVidvan Team\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:creator\" content=\"@vidvantech\" \/>\n<meta name=\"twitter:site\" content=\"@vidvantech\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"TechVidvan Team\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"7 minutes\" \/>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Serialization in Python - TechVidvan","description":"Learn about serialization in Python and how to employ Python libraries to serialize objects like dictionaries and Tensorflow Keras models.","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/techvidvan.com\/tutorials\/serialization-in-python\/","og_locale":"en_US","og_type":"article","og_title":"Serialization in Python - TechVidvan","og_description":"Learn about serialization in Python and how to employ Python libraries to serialize objects like dictionaries and Tensorflow Keras models.","og_url":"https:\/\/techvidvan.com\/tutorials\/serialization-in-python\/","og_site_name":"TechVidvan","article_publisher":"https:\/\/www.facebook.com\/TechVidvan\/","article_published_time":"2023-02-10T03:30:13+00:00","og_image":[{"width":1200,"height":628,"url":"https:\/\/techvidvan.com\/tutorials\/wp-content\/uploads\/2023\/01\/serialization-in-python.webp","type":"image\/webp"}],"author":"TechVidvan Team","twitter_card":"summary_large_image","twitter_creator":"@vidvantech","twitter_site":"@vidvantech","twitter_misc":{"Written by":"TechVidvan Team","Est. reading time":"7 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/techvidvan.com\/tutorials\/serialization-in-python\/#article","isPartOf":{"@id":"https:\/\/techvidvan.com\/tutorials\/serialization-in-python\/"},"author":{"name":"TechVidvan Team","@id":"https:\/\/techvidvan.com\/tutorials\/#\/schema\/person\/e9c26e74dd3d87421f7ada9433b8cd22"},"headline":"Serialization in Python","datePublished":"2023-02-10T03:30:13+00:00","mainEntityOfPage":{"@id":"https:\/\/techvidvan.com\/tutorials\/serialization-in-python\/"},"wordCount":1123,"commentCount":0,"publisher":{"@id":"https:\/\/techvidvan.com\/tutorials\/#organization"},"image":{"@id":"https:\/\/techvidvan.com\/tutorials\/serialization-in-python\/#primaryimage"},"thumbnailUrl":"https:\/\/techvidvan.com\/tutorials\/wp-content\/uploads\/2023\/01\/serialization-in-python.webp","keywords":["Serialization in Python"],"articleSection":["Python Tutorials"],"inLanguage":"en-US","potentialAction":[{"@type":"CommentAction","name":"Comment","target":["https:\/\/techvidvan.com\/tutorials\/serialization-in-python\/#respond"]}]},{"@type":"WebPage","@id":"https:\/\/techvidvan.com\/tutorials\/serialization-in-python\/","url":"https:\/\/techvidvan.com\/tutorials\/serialization-in-python\/","name":"Serialization in Python - TechVidvan","isPartOf":{"@id":"https:\/\/techvidvan.com\/tutorials\/#website"},"primaryImageOfPage":{"@id":"https:\/\/techvidvan.com\/tutorials\/serialization-in-python\/#primaryimage"},"image":{"@id":"https:\/\/techvidvan.com\/tutorials\/serialization-in-python\/#primaryimage"},"thumbnailUrl":"https:\/\/techvidvan.com\/tutorials\/wp-content\/uploads\/2023\/01\/serialization-in-python.webp","datePublished":"2023-02-10T03:30:13+00:00","description":"Learn about serialization in Python and how to employ Python libraries to serialize objects like dictionaries and Tensorflow Keras models.","breadcrumb":{"@id":"https:\/\/techvidvan.com\/tutorials\/serialization-in-python\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/techvidvan.com\/tutorials\/serialization-in-python\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/techvidvan.com\/tutorials\/serialization-in-python\/#primaryimage","url":"https:\/\/techvidvan.com\/tutorials\/wp-content\/uploads\/2023\/01\/serialization-in-python.webp","contentUrl":"https:\/\/techvidvan.com\/tutorials\/wp-content\/uploads\/2023\/01\/serialization-in-python.webp","width":1200,"height":628,"caption":"serialization in python"},{"@type":"BreadcrumbList","@id":"https:\/\/techvidvan.com\/tutorials\/serialization-in-python\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/techvidvan.com\/tutorials\/"},{"@type":"ListItem","position":2,"name":"Serialization in Python"}]},{"@type":"WebSite","@id":"https:\/\/techvidvan.com\/tutorials\/#website","url":"https:\/\/techvidvan.com\/tutorials\/","name":"TechVidvan Blogs","description":"","publisher":{"@id":"https:\/\/techvidvan.com\/tutorials\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/techvidvan.com\/tutorials\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/techvidvan.com\/tutorials\/#organization","name":"TechVidvan","url":"https:\/\/techvidvan.com\/tutorials\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/techvidvan.com\/tutorials\/#\/schema\/logo\/image\/","url":"https:\/\/techvidvan.com\/tutorials\/wp-content\/uploads\/2024\/03\/techvidvan-logo-200x50-1.webp","contentUrl":"https:\/\/techvidvan.com\/tutorials\/wp-content\/uploads\/2024\/03\/techvidvan-logo-200x50-1.webp","width":200,"height":50,"caption":"TechVidvan"},"image":{"@id":"https:\/\/techvidvan.com\/tutorials\/#\/schema\/logo\/image\/"},"sameAs":["https:\/\/www.facebook.com\/TechVidvan\/","https:\/\/x.com\/vidvantech"]},{"@type":"Person","@id":"https:\/\/techvidvan.com\/tutorials\/#\/schema\/person\/e9c26e74dd3d87421f7ada9433b8cd22","name":"TechVidvan Team","description":"The TechVidvan Team delivers practical, beginner-friendly tutorials on programming, Java, Python, C++, DSA, AI, ML, data Science, Android, Flutter, MERN, Web Development, and technology. Our experts are here to help you upskill and excel in today\u2019s tech industry."}]}},"amp_enabled":true,"_links":{"self":[{"href":"https:\/\/techvidvan.com\/tutorials\/wp-json\/wp\/v2\/posts\/86942","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/techvidvan.com\/tutorials\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/techvidvan.com\/tutorials\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/techvidvan.com\/tutorials\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/techvidvan.com\/tutorials\/wp-json\/wp\/v2\/comments?post=86942"}],"version-history":[{"count":0,"href":"https:\/\/techvidvan.com\/tutorials\/wp-json\/wp\/v2\/posts\/86942\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/techvidvan.com\/tutorials\/wp-json\/wp\/v2\/media\/87039"}],"wp:attachment":[{"href":"https:\/\/techvidvan.com\/tutorials\/wp-json\/wp\/v2\/media?parent=86942"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/techvidvan.com\/tutorials\/wp-json\/wp\/v2\/categories?post=86942"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/techvidvan.com\/tutorials\/wp-json\/wp\/v2\/tags?post=86942"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}