Skip to content

Commit 706e38f

Browse files
joyyojpwendell
authored andcommitted
[SPARK-1998] SparkFlumeEvent with body bigger than 1020 bytes are not re...
flume event sent to Spark will fail if the body is too large and numHeaders is greater than zero Author: joyyoj <sunshch@gmail.com> Closes apache#951 from joyyoj/master and squashes the following commits: f4660c5 [joyyoj] [SPARK-1998] SparkFlumeEvent with body bigger than 1020 bytes are not read properly (cherry picked from commit 2966044) Signed-off-by: Patrick Wendell <pwendell@gmail.com>
1 parent cc95d97 commit 706e38f

File tree

1 file changed

+2
-2
lines changed

1 file changed

+2
-2
lines changed

external/flume/src/main/scala/org/apache/spark/streaming/flume/FlumeInputDStream.scala

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -69,12 +69,12 @@ class SparkFlumeEvent() extends Externalizable {
6969
for (i <- 0 until numHeaders) {
7070
val keyLength = in.readInt()
7171
val keyBuff = new Array[Byte](keyLength)
72-
in.read(keyBuff)
72+
in.readFully(keyBuff)
7373
val key : String = Utils.deserialize(keyBuff)
7474

7575
val valLength = in.readInt()
7676
val valBuff = new Array[Byte](valLength)
77-
in.read(valBuff)
77+
in.readFully(valBuff)
7878
val value : String = Utils.deserialize(valBuff)
7979

8080
headers.put(key, value)

0 commit comments

Comments
 (0)