[SPARK-12413] Fix Mesos ZK persistence

Michael Gummelt · sarutak · commit df0231952e55 · 2015-12-18T20:21:42.000+09:00
I believe this fixes SPARK-12413. I'm currently running an integration test to verify. Author: Michael Gummelt <mgummelt@mesosphere.io> Closes apache#10366 from mgummelt/fix-zk-mesos. (cherry picked from commit 2bebaa3) Signed-off-by: Kousuke Saruta <sarutak@oss.nttdata.co.jp>
diff --git a/core/src/main/scala/org/apache/spark/deploy/rest/mesos/MesosRestServer.scala b/core/src/main/scala/org/apache/spark/deploy/rest/mesos/MesosRestServer.scala
@@ -99,7 +99,11 @@ private[mesos] class MesosSubmitRequestServlet(
     // look for files in SPARK_HOME instead. We only need the ability to specify where to find
     // spark-submit script which user can user spark.executor.home or spark.home configurations
     // (SPARK-12345).
-    val environmentVariables = request.environmentVariables.filterKeys(!_.equals("SPARK_HOME"))
+
+    // Do not use `filterKeys` here to avoid SI-6654, which breaks ZK persistence
+    val environmentVariables = request.environmentVariables.filter { case (k, _) =>
+      k != "SPARK_HOME"
+    }
     val name = request.sparkProperties.get("spark.app.name").getOrElse(mainClass)
 
     // Construct driver description
