TechVidvan
1. Objective The recovery of failures of machines is already inbuilt in Apache spark streaming. This feature is what we call spark streaming fault tolerance property. In this blog, we will learn the whole...
1. Objective As we know, there are so many distributed stream processing engines available. The question arises is why Apache spark streaming and what are its unique benefits. Spark Streaming tutorial totally aims at...
1. Objective Basically, there is a pretty simple concept of a Spark Shared variable. In simple words, these are variables those we want to share throughout our cluster. In this blog, we completely focus...
1. Objective Spark DStream (Discretized Stream) is the basic abstraction of Spark Streaming. In this blog, we will learn the concept of DStream in Spark, we will learn what is DStream, operations of DStream...
1. Objective As we know, there are various modules available in Apache Spark. Each module is serving different purposes, streaming API is one of its powerful modules. It provides power to the developer to...
1. Objective For processing real-time streaming data Apache Storm is the stream processing framework, while Spark is a general purpose computing engine. To handle streaming data it offers Spark Streaming. Hence, Streaming process data...
1. Objective Seems like Spark RDDs, input DStream transformations in Apache spark also allow the data to be modified. Many of the spark transformations available on normal spark RDD’s, that Dstreams support. In this...
1. Objective This document aims at a Spark Streaming Checkpoint, we will start with what is a streaming checkpoint, how streaming checkpoint helps to achieve fault tolerance. There are two types of spark checkpoint...
1. Objective There is always a question tickling in mind that why they should be using datasets rather than Spark RDD. In this tutorial, we will give you answer this question by comparing Spark...
1. Objective Recently, there are two new data abstractions released dataframe and datasets in apache spark. Now, it might be difficult to understand the relevance of each one. Also, not easy to decide which...
