IB/hfi1: Fix a latency issue for small messages

mjruhl · dledford · commit 90b2620e6a8a · 2018-12-03T16:05:19.000-05:00
A recent performance enhancement introduced a latency issue in the HFI message path. The new algorithm removed a forced call send for PIO messages and added a forced schedule event for messages larger than the MTU. For PIO, the schedule path can introduce thrashing that can significantly impact the throughput for small messages. If a message size is within the PIO threshold, always take the send path. Fixes: 0b79b27 ("IB/{hfi1, qib, rdmavt}: Schedule multi RC/UC packets instead of posting") Reviewed-by: Mike Marciniszyn <mike.marciniszyn@intel.com> Signed-off-by: Michael J. Ruhl <michael.j.ruhl@intel.com> Signed-off-by: Dennis Dalessandro <dennis.dalessandro@intel.com> Signed-off-by: Doug Ledford <dledford@redhat.com>
diff --git a/drivers/infiniband/hw/hfi1/qp.c b/drivers/infiniband/hw/hfi1/qp.c
@@ -340,6 +340,13 @@ int hfi1_setup_wqe(struct rvt_qp *qp, struct rvt_swqe *wqe, bool *call_send)
 	default:
 		break;
 	}
+
+	/*
+	 * System latency between send and schedule is large enough that
+	 * forcing call_send to true for piothreshold packets is necessary.
+	 */
+	if (wqe->length <= piothreshold)
+		*call_send = true;
 	return 0;
 }
 

Original file line number	Diff line number	Diff line change
`@@ -340,6 +340,13 @@ int hfi1_setup_wqe(struct rvt_qp qp, struct rvt_swqe wqe, bool *call_send)`
`340`	`340`	`default:`
`341`	`341`	`break;`
`342`	`342`	`}`
	`343`	`+`
	`344`	`+ /*`
	`345`	`+ * System latency between send and schedule is large enough that`
	`346`	`+ * forcing call_send to true for piothreshold packets is necessary.`
	`347`	`+ */`
	`348`	`+ if (wqe->length <= piothreshold)`
	`349`	`+ *call_send = true;`
`343`	`350`	`return 0;`
`344`	`351`	`}`
`345`	`352`