Is there a way to ensure that if we add new functionality, we HAVE to remove the corresponding lines from the skips file? Basically treat them as xfail and use this: scrict xfails

Test Results

5 923 tests  ±0   5 891 ✅ +1   6m 52s ⏱️ - 5m 9s
    1 suites ±0      32 💤  - 1 
    1 files   ±0       0 ❌ ±0 

Results for commit 6170535. ± Comparison against base commit 0308ed0.

sparse.numba_backend.tests.test_compressed ‑ test_reductions_float16[f4-1-sum-kwargs0]
sparse.numba_backend.tests.test_compressed ‑ test_reductions_float16[i8-None-sum-kwargs0]

sparse.numba_backend.tests.test_compressed ‑ test_reductions_float16[f8-None-sum-kwargs0]
sparse.numba_backend.tests.test_coo ‑ test_reductions_float16[i4-None-sum-kwargs0]
sparse.numba_backend.tests.test_coo ‑ test_reductions_float16[i8-None-sum-kwargs0]

♻️ This comment has been updated with latest results.

I don't think array-api-tests supports it. Skips file just defines which functions are missing. We did it this way in NumPy and we successfully got to 99% compatibility after a few months. One needs to remember to remove specific lines from this file.