As we know, there are so many distributed stream processing engines available. The question arises is why Apache spark streaming and what are its unique benefits. Spark Streaming tutorial totally aims at the topic...
Basically, there is a pretty simple concept of a Spark Shared variable. In simple words, these are variables those we want to share throughout our cluster. In this blog, we completely focus on Shared...
Spark DStream (Discretized Stream) is the basic abstraction of Spark Streaming. In this blog, we will learn the concept of DStream in Spark, we will learn what is DStream, operations of DStream such as...
As we know, there are various modules available in Apache Spark. Each module is serving different purposes, streaming API is one of its powerful modules. It provides power to the developer to work with...
For processing real-time streaming data Apache Storm is the stream processing framework, while Spark is a general purpose computing engine. To handle streaming data it offers Spark Streaming. Hence, Streaming process data in near...
Seems like Spark RDDs, input DStream transformations in Apache spark also allow the data to be modified. Many of the spark transformations available on normal spark RDD’s, that Dstreams support. In this blog, we...
This document aims at a Spark Streaming Checkpoint, we will start with what is a streaming checkpoint, how streaming checkpoint helps to achieve fault tolerance. There are two types of spark checkpoint i.e. reliable...
There is always a question tickling in mind that why they should be using datasets rather than Spark RDD. In this tutorial, we will give you answer this question by comparing Spark RDD vs...
Recently, there are two new data abstractions released dataframe and datasets in apache spark. Now, it might be difficult to understand the relevance of each one. Also, not easy to decide which one to...
Spark datasets is a distributed collection of data. It is a new interface, provides benefits of RDDs with Spark SQL’s optimized execution engine. In this blog, we will learn the concept of Spark SQL...