New needle_map.CompactMap() implementation for reduced memory usage (#6842)
* Rework `needle_map.CompactMap()` to maximize memory efficiency. * Use a memory-efficient structure for `CompactMap` needle value entries. This slightly complicates the code, but makes a **massive** difference in memory efficiency - preliminary results show a ~30% reduction in heap usage, with no measurable performance impact otherwise. * Clean up type for `CompactMap` chunk IDs. * Add a small comment description for `CompactMap()`. * Add the old version of `CompactMap()` for comparison purposes.
This commit is contained in:
@@ -43,6 +43,7 @@ func TestMemoryUsage(t *testing.T) {
|
||||
|
||||
PrintMemUsage(totalRowCount)
|
||||
now := time.Now()
|
||||
fmt.Printf("\tCompactMap = %s", m.String())
|
||||
fmt.Printf("\tTaken = %v\n", now.Sub(startTime))
|
||||
startTime = now
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user