Cool. Would that work also for batched matmul (per slice matmul) ?

@WilliamTambellini I think that must be ensured, though I haven't added this specific test yet in the published commits. The goal is that all of the existing behavior of af_matmul will not change (which includes its current batching behavior), but we will have the added ability to reuse an existing af_array. Thanks for bringing it up!

@WilliamTambellini This change is to the C-API, we will be adding a new gemm API that we have been discussing in the issue #2446 . It will look different from regular matmul. Something like the following

void af::gemm(af::array& C, const array &lhs, const array &rhs,
const matProp optLhs = AF_MAT_NONE, const matProp optRhs = AF_MAT_NONE,
const float alpha = 1, const float beta = 1);

C would be your pre-allocated output array. After each call to this new API, C will have aplha*(lhs x rhs) + beta*C. What happens when lhs and rhs are batched inputs is a design choice we haven't made yet. Once this PR is merged, we will raise the new function PR and we can continue this discussion on that.

ok tks @9prady9
@mark-poscablo Could you consequently add a unitest for accumulated batched gemm :
Example:
A= 3 x 2 x 2
0 1
1 2
2 3
0 1
1 2
2 3

B= 2 x 3 x 2
0 1 2
1 2 3
0 1 2
1 2 3

After looping over the slices with 1 gemm per slice on C shaped 3 x 3 (single slice) with beta = 1, C should be :
C=3 x 3
2 4 6
4 10 16
6 16 26

Why just matmul ? None of the other functions support using an existing af_array.

Why just matmul ? None of the other functions support using an existing af_array.

matmul specifically because this is a part of the approach we're taking regarding #2446 (see @9prady9's comment above). We have done this also with a few other functions such as af_approx1 (#2324)

Coming back to this after some time. Working on leaving the old behavior intact and separating the new behavior into new functions.

@9prady9 Yes I have more coming in. Pushed some changes for now to see how they fare in different systems though.