python - How can I create a numpy .npy file in place on disk? -

is possible create .npy file without allocating corresponding array in memory first?

i need create , work large numpy array, big create in memory. numpy supports memory mapping, far can see options either:

1. create memmapped file using numpy.memmap. creates file directly on disk without allocating memory, doesn't store metadata, when re-map file later need know dtype, shape, etc. in following, notice not specifying shape results in memmap being interpreted flat array:

   in [77]: x=memmap('/tmp/x', int, 'w+', shape=(3,3))   in [78]: x out[78]:  memmap([[0, 0, 0],        [0, 0, 0],        [0, 0, 0]])   in [79]: y=memmap('/tmp/x', int, 'r')   in [80]: y out[80]: memmap([0, 0, 0, 0, 0, 0, 0, 0, 0]) 

2. create array in memory, save using numpy.save, after can loaded in memmapped mode. records metadata array data on disk, requires memory allocated entire array @ least once.

i had same question , disappointed when read sven's reply. seems though numpy missing out on key functionality if couldn't have huge array on file , work on little pieces of @ time. case seems close 1 of use cases in origional rational making .npy format (see: http://svn.scipy.org/svn/numpy/trunk/doc/neps/npy-format.txt).

i ran numpy.lib.format, seems full useful goodies. have no idea why functionality not available numpy root package. key advantage on hdf5 ships numpy.

>>> print numpy.lib.format.open_memmap.__doc__  """ open .npy file memory-mapped array.  may used read existing file or create new one.  parameters ---------- filename : str     name of file on disk. may not filelike object. mode : str, optional     mode open file with. in addition standard file modes,     'c' accepted mean "copy on write". see `numpy.memmap`     available mode strings. dtype : dtype, optional     data type of array if creating new file in "write"     mode. shape : tuple of int, optional     shape of array if creating new file in "write"     mode. fortran_order : bool, optional     whether array should fortran-contiguous (true) or     c-contiguous (false) if creating new file in "write" mode. version : tuple of int (major, minor)     if mode "write" mode, version of file     format used create file.  returns ------- marray : numpy.memmap     memory-mapped array.  raises ------ valueerror     if data or mode invalid. ioerror     if file not found or cannot opened correctly.  see -------- numpy.memmap """