Is there a way to truncate the beginning of numpy memmap files?

39 Views Asked by At

I want to keep the memmap file size unchanged when adding new rows. For example:

I create a sample memmap dataset first

x = np.range(24).reshape(4,6)
data = np.memmap('/home/test', mode='w+', dtype=np.float32, shape=(4, 6))
data[1:, :] = x[:]

So the data matrix without offset is

[[ 0.  1.  2.  3.  4.  5.]
 [ 6.  7.  8.  9. 10. 11.]
 [12. 13. 14. 15. 16. 17.]
 [18. 19. 20. 21. 22. 23.]]

Then I add a new row by offset the first row

data = np.memmap('/home/test', mode='r+', dtype=np.float32, shape=(4, 6), offset=1*6*np.dtype(np.float32).itemsize)
data[-1] = -np.arange(1, 7)

So the new data matrix is

[[ 0.  1.  2.  3.  4.  5.]
 [ 6.  7.  8.  9. 10. 11.]
 [12. 13. 14. 15. 16. 17.]
 [18. 19. 20. 21. 22. 23.]
 [-1, -2, -3, -4, -5, -6]]

Now I want to drop the first row and keep the size as (4, 6), that is, I want to roll forward to update the data matrix in numpy.memmap(). I searched and found that typical os.trucate() always delete values in the end. Is there a way that I can use to truncate numpy memmap values in the beginning, so that I can read the file without offset and get

[[ 6.  7.  8.  9. 10. 11.]
 [12. 13. 14. 15. 16. 17.]
 [18. 19. 20. 21. 22. 23.]
 [-1, -2, -3, -4, -5, -6]]

(NOT create a new memmap and copy values, which is time consuming)

0

There are 0 best solutions below