Deep Learning Project – Colorize Black & White Images with Python

This Deep Learning Project aims to provide colorizing black & white images with Python.

In image colorization, we take a black and white image as input and produce a colored image. We will solve this project with OpenCV deep neural network.

Deep Learning Project Colorize Images Python

 

Deep Learning Project – Colorize Black & White Images with Python

Lab Color Space:

Like RGB, Lab is another color space. It is also three channel color space like RGB where the channels are:

  • L channel: This channel represents the Lightness
  • a channel: This channel represents green-red
  • b channel: This channel represents blue-yellow

In this color space, the grayscale part of the image is only encoded in L channel. Therefore Lab color space is more favorable for our project.

Follow TechVidvan on Google & Stay updated with latest technology trends

Problem Statement:

deep learning project colorize black white images with python

We can formulate our problem statement as to predict a and b channels, given an input grayscale image.

In this deep learning project, we will use OpenCV DNN architecture which is trained on ImageNet dataset. The neural net is trained with the L channel of images as input data and a,b channels as target data.

Steps to implement Image Colorization Project:

For colorizing black and white images we will be using a pre-trained caffe model, a prototxt file, and a NumPy file.

The prototxt file defines the network and the numpy file stores the cluster center points in numpy format.

1. Make a directory with name models.

mkdir models

2. Open the terminal and run following commands to download the caffemodel, prototxt file and the NumPy file.

wget https://github.com/richzhang/colorization/blob/master/colorization/resources/pts_in_hull.npy?raw=true -O ./pts_in_hull.npy
  
wget https://raw.githubusercontent.com/richzhang/colorization/master/colorization/models/colorization_deploy_v2.prototxt -O ./models/colorization_deploy_v2.prototxt

wget http://eecs.berkeley.edu/~rich.zhang/projects/2016_colorization/files/demo_v2/colorization_release_v2.caffemodel -O ./models/colorization_release_v2.caffemodel

3. Create a python file image_colorization.py and paste the code specified in the below steps

4. Imports:

import numpy as np
import cv2 as cv
import os.path

5. Read B&W image and load the caffemodel:

frame = cv.imread(“__test_image_path__”)

numpy_file = np.load('./pts_in_hull.npy')

Caffe_net = cv.dnn.readNetFromCaffe("./models/colorization_deploy_v2.prototxt", "./models/colorization_release_v2.caffemodel")

6. Add layers to the caffe model:

numpy_file = numpy_file.transpose().reshape(2, 313, 1, 1)
Caffe_net.getLayer(Caffe_net.getLayerId('class8_ab')).blobs = [numpy_file.astype(np.float32)]
Caffe_net.getLayer(Caffe_net.getLayerId('conv8_313_rh')).blobs = [np.full([1, 313], 2.606, np.float32)]

7. Extract L channel and resize it:

input_width = 224
input_height = 224

rgb_img = (frame[:,:,[2, 1, 0]] * 1.0 / 255).astype(np.float32)
lab_img = cv.cvtColor(rgb_img, cv.COLOR_RGB2Lab)
l_channel = lab_img[:,:,0] 

l_channel_resize = cv.resize(l_channel, (input_width, input_height)) 
l_channel_resize -= 50

8. Predict the ab channel and save the result:

Caffe_net.setInput(cv.dnn.blobFromImage(l_channel_resize))
ab_channel = Caffe_net.forward()[0,:,:,:].transpose((1,2,0)) 

(original_height,original_width) = rgb_img.shape[:2] 
ab_channel_us = cv.resize(ab_channel, (original_width, original_height))
lab_output = np.concatenate((l_channel[:,:,np.newaxis],ab_channel_us),axis=2) 
bgr_output = np.clip(cv.cvtColor(lab_output, cv.COLOR_Lab2BGR), 0, 1)

cv.imwrite("./result.png", (bgr_output*255).astype(np.uint8))

Results:

Now execute the deep learning project – run the python file with path of a grayscale image to test our results.

python3 image_colorization.py

After running the above python file, the colorized image will be saved in your working directory as result.png

Deep Learning Project Colorize Images Python

Code for GUI:

Make a new python file gui.py in the present directory. Paste the below code for image colorization interface.

import tkinter as tk
from tkinter import *
from tkinter import filedialog
from PIL import Image, ImageTk
import os
import numpy as np
import cv2 as cv
import os.path


numpy_file = np.load('./pts_in_hull.npy')
Caffe_net = cv.dnn.readNetFromCaffe("./models/colorization_deploy_v2.prototxt", "./models/colorization_release_v2.caffemodel")
numpy_file = numpy_file.transpose().reshape(2, 313, 1, 1)

class Window(Frame):
    def __init__(self, master=None):
        Frame.__init__(self, master)

        self.master = master
        self.pos = []
        self.master.title("B&W Image Colorization")
        self.pack(fill=BOTH, expand=1)

        menu = Menu(self.master)
        self.master.config(menu=menu)

        file = Menu(menu)
        file.add_command(label="Upload Image", command=self.uploadImage)
        file.add_command(label="Color Image", command=self.color)
        menu.add_cascade(label="File", menu=file)

        self.canvas = tk.Canvas(self)
        self.canvas.pack(fill=tk.BOTH, expand=True)
        self.image = None
        self.image2 = None

        label1=Label(self,image=img)
        label1.image=img
        label1.place(x=400,y=370)




    def uploadImage(self):
        filename = filedialog.askopenfilename(initialdir=os.getcwd())
        if not filename:
            return
        load = Image.open(filename)

        load = load.resize((480, 360), Image.ANTIALIAS)

        if self.image is None:
            w, h = load.size
            width, height = root.winfo_width(), root.winfo_height()
            self.render = ImageTk.PhotoImage(load)
            self.image = self.canvas.create_image((w / 2, h / 2), image=self.render)
           
        else:
            self.canvas.delete(self.image3)
            w, h = load.size
            width, height = root.winfo_screenmmwidth(), root.winfo_screenheight()
           
            self.render2 = ImageTk.PhotoImage(load)
            self.image2 = self.canvas.create_image((w / 2, h / 2), image=self.render2)


        frame = cv.imread(filename)
    
        Caffe_net.getLayer(Caffe_net.getLayerId('class8_ab')).blobs = [numpy_file.astype(np.float32)]
        Caffe_net.getLayer(Caffe_net.getLayerId('conv8_313_rh')).blobs = [np.full([1, 313], 2.606, np.float32)]

        input_width = 224
        input_height = 224

        rgb_img = (frame[:,:,[2, 1, 0]] * 1.0 / 255).astype(np.float32)
        lab_img = cv.cvtColor(rgb_img, cv.COLOR_RGB2Lab)
        l_channel = lab_img[:,:,0] 

        l_channel_resize = cv.resize(l_channel, (input_width, input_height)) 
        l_channel_resize -= 50 

        Caffe_net.setInput(cv.dnn.blobFromImage(l_channel_resize))
        ab_channel = Caffe_net.forward()[0,:,:,:].transpose((1,2,0)) 

        (original_height,original_width) = rgb_img.shape[:2] 
        ab_channel_us = cv.resize(ab_channel, (original_width, original_height))
        lab_output = np.concatenate((l_channel[:,:,np.newaxis],ab_channel_us),axis=2) 
        bgr_output = np.clip(cv.cvtColor(lab_output, cv.COLOR_Lab2BGR), 0, 1)

  
        cv.imwrite("./result.png", (bgr_output*255).astype(np.uint8))

    def color(self):

        load = Image.open("./result.png")
        load = load.resize((480, 360), Image.ANTIALIAS)

        if self.image is None:
            w, h = load.size
            self.render = ImageTk.PhotoImage(load)
            self.image = self.canvas.create_image((w / 2, h/2), image=self.render)
            root.geometry("%dx%d" % (w, h))
        else:
            w, h = load.size
            width, height = root.winfo_screenmmwidth(), root.winfo_screenheight()

            self.render3 = ImageTk.PhotoImage(load)
            self.image3 = self.canvas.create_image((w / 2, h / 2), image=self.render3)
            self.canvas.move(self.image3, 500, 0)
 

root = tk.Tk()
root.geometry("%dx%d" % (980, 600))
root.title("B&W Image Colorization GUI")
img = ImageTk.PhotoImage(Image.open("logo2.png"))

app = Window(root)
app.pack(fill=tk.BOTH, expand=1)
root.mainloop()

Summary:

This tutorial guides you how to build a deep learning project to colorize black and white images. It first introduces you to Lab color space and why it is favorable for our problem statement. Then step by step it describes how to implement black and white image colorizer.

Are you looking for more deep learning projects with source code? Please write in comment section, we will develop and publish the project.

Do you like our efforts? Please rate TechVidvan on Google.

Did you like this article? If Yes, please give TechVidvan 5 Stars on Google | Facebook


8 Responses

  1. ali osama says:

    thanks man
    share source code

  2. Esmanur says:

    Thx a lot, it is an amazing project

  3. chaitanya says:

    the caffe model and prototxt files links are not working

  4. sindhu says:

    please send the correct link for the caffe model and prototxt model

  5. Rafi says:

    I want pix2pix Gan code.

  6. Suriya says:

    you can download the model externally from predefined models

Leave a Reply

Your email address will not be published. Required fields are marked *