[SPARK-12520] [PYSPARK] [1.5] Ensure the join type is inner for equi-Join.

gatorsmile · davies · commit 42286feb676f · 2015-12-27T23:23:57.000-08:00
This PR is to add `assert` to ensure the join type is `inner` for equi-Join. JIRA: https://issues.apache.org/jira/browse/SPARK-12520 In the JIRA, users specify the join type `outer` when using the equi-join. However, the result we returned is the `inner` join, which is the only type Spark 1.5 supports. (Note, starting from Spark 1.6, we can support the other types for equi-join). For example, ```scala joined_table = left_table.join(right_table, "joining_column", "outer") ``` Should we also back port it to 1.4? davies JoshRosen Thanks! Author: gatorsmile <gatorsmile@gmail.com> Closes apache#10484 from gatorsmile/pythonEquiOuterJoin.
diff --git a/python/pyspark/sql/dataframe.py b/python/pyspark/sql/dataframe.py
@@ -570,6 +570,7 @@ def join(self, other, on=None, how=None):
         if on is None or len(on) == 0:
             jdf = self._jdf.join(other._jdf)
         elif isinstance(on[0], basestring):
+            assert how is None or how == 'inner', "Equi-join does not support: %s" % how
             jdf = self._jdf.join(other._jdf, self._jseq(on))
         else:
             assert isinstance(on[0], Column), "on should be Column or list of Column"
