Skip to content

Commit d86a88d

Browse files
Andrew OrJoshRosen
authored andcommitted
[SPARK-12165][ADDENDUM] Fix outdated comments on unroll test
JoshRosen Author: Andrew Or <andrew@databricks.com> Closes apache#10229 from andrewor14/unroll-test-comments. (cherry picked from commit 8770bd1) Signed-off-by: Josh Rosen <joshrosen@databricks.com>
1 parent 9bc6a27 commit d86a88d

File tree

1 file changed

+9
-4
lines changed

1 file changed

+9
-4
lines changed

core/src/test/scala/org/apache/spark/memory/StaticMemoryManagerSuite.scala

Lines changed: 9 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -163,15 +163,20 @@ class StaticMemoryManagerSuite extends MemoryManagerSuite {
163163
assertEvictBlocksToFreeSpaceNotCalled(ms)
164164
assert(mm.storageMemoryUsed === 860L)
165165
// `spark.storage.unrollFraction` is 0.4, so the max unroll space is 400 bytes.
166-
// Since we already occupy 60 bytes, we will try to evict only 400 - 60 = 340 bytes.
166+
// As of this point, cache memory is 800 bytes and current unroll memory is 60 bytes.
167+
// Requesting 240 more bytes of unroll memory will leave our total unroll memory at
168+
// 300 bytes, still under the 400-byte limit. Therefore, all 240 bytes are granted.
167169
assert(mm.acquireUnrollMemory(dummyBlock, 240L, evictedBlocks))
168-
assertEvictBlocksToFreeSpaceCalled(ms, 100L)
170+
assertEvictBlocksToFreeSpaceCalled(ms, 100L) // 860 + 240 - 1000
169171
when(ms.currentUnrollMemory).thenReturn(300L) // 60 + 240
170172
assert(mm.storageMemoryUsed === 1000L)
171173
evictedBlocks.clear()
174+
// We already have 300 bytes of unroll memory, so requesting 150 more will leave us
175+
// above the 400-byte limit. Since there is not enough free memory, this request will
176+
// fail even after evicting as much as we can (400 - 300 = 100 bytes).
172177
assert(!mm.acquireUnrollMemory(dummyBlock, 150L, evictedBlocks))
173-
assertEvictBlocksToFreeSpaceCalled(ms, 100L) // 400 - 300
174-
assert(mm.storageMemoryUsed === 900L) // 100 bytes were evicted
178+
assertEvictBlocksToFreeSpaceCalled(ms, 100L)
179+
assert(mm.storageMemoryUsed === 900L)
175180
// Release beyond what was acquired
176181
mm.releaseUnrollMemory(maxStorageMem)
177182
assert(mm.storageMemoryUsed === 0L)

0 commit comments

Comments
 (0)