python - How can I create a numpy .npy file in place on disk? -


is possible create .npy file without allocating corresponding array in memory first?

i need create , work large numpy array, big create in memory. numpy supports memory mapping, far can see options either:

  1. create memmapped file using numpy.memmap. creates file directly on disk without allocating memory, doesn't store metadata, when re-map file later need know dtype, shape, etc. in following, notice not specifying shape results in memmap being interpreted flat array:

    in [77]: x=memmap('/tmp/x', int, 'w+', shape=(3,3))   in [78]: x out[78]:  memmap([[0, 0, 0],        [0, 0, 0],        [0, 0, 0]])   in [79]: y=memmap('/tmp/x', int, 'r')   in [80]: y out[80]: memmap([0, 0, 0, 0, 0, 0, 0, 0, 0])

  2. create array in memory, save using numpy.save, after can loaded in memmapped mode. records metadata array data on disk, requires memory allocated entire array @ least once.

i had same question , disappointed when read sven's reply. seems though numpy missing out on key functionality if couldn't have huge array on file , work on little pieces of @ time. case seems close 1 of use cases in origional rational making .npy format (see: http://svn.scipy.org/svn/numpy/trunk/doc/neps/npy-format.txt).

i ran numpy.lib.format, seems full useful goodies. have no idea why functionality not available numpy root package. key advantage on hdf5 ships numpy.

>>> print numpy.lib.format.open_memmap.__doc__  """ open .npy file memory-mapped array.  may used read existing file or create new one.  parameters ---------- filename : str     name of file on disk. may not filelike object. mode : str, optional     mode open file with. in addition standard file modes,     'c' accepted mean "copy on write". see `numpy.memmap`     available mode strings. dtype : dtype, optional     data type of array if creating new file in "write"     mode. shape : tuple of int, optional     shape of array if creating new file in "write"     mode. fortran_order : bool, optional     whether array should fortran-contiguous (true) or     c-contiguous (false) if creating new file in "write" mode. version : tuple of int (major, minor)     if mode "write" mode, version of file     format used create file.  returns ------- marray : numpy.memmap     memory-mapped array.  raises ------ valueerror     if data or mode invalid. ioerror     if file not found or cannot opened correctly.  see -------- numpy.memmap """

Comments

Post a Comment

Popular posts from this blog

asp.net - repeatedly call AddImageUrl(url) to assemble pdf document -

i'm using abcpdf , i'm curious if can recursively call addimageurl() function assemble pdf document compile multiple urls? something like: int pagecount = 0; int theid = thedoc.addimageurl("http://stackoverflow.com/search?q=abcpdf+footer+page+x+out+of+", true, 0, true); //assemble document while (thedoc.chainable(theid)) { thedoc.page = thedoc.addpage(); theid = thedoc.addimagetochain(theid); } pagecount = thedoc.pagecount; console.writeline("1 document page count:" + pagecount); //flatten document (int = 1; <= pagecount; i++) { thedoc.pagenumber = i; thedoc.flatten(); } //now try again theid = thedoc.addimageurl("http://stackoverflow.com/questions/1980890/pdf-report-generation", true, 0, true); //assemble document while (thedoc.chainable(theid)) { thedoc.page = thedoc.addpage(); theid = thedoc.addimagetoc...
Read more

java - Android recognize cell phone with keyboard or not? -

in android app, want detect user's device if device has keyboard(like motorola milestone) , show corresponding user interface. guys know how that? ! regards justicepenny http://developer.android.com/reference/android/content/res/configuration.html public int keyboard the kind of keyboard attached device. 1 of: keyboard_nokeys, keyboard_qwerty, keyboard_12key. that's it!
Read more

iphone - How would you achieve a LED Scrolling effect? -

how achieve led scrolling effect example below? led sign - led ticker emulator iphone , ipad http://img.skitch.com/20101201-rsfh4p1bajb1wp94k466pdpiuj.preview.jpg sometimes remunerates comment in other posts :) have put code you, should implement font '8pin matrix' realistic feeling. uilabel *label; - (void)viewdidload { [super viewdidload]; label = [[uilabel alloc]initwithframe:cgrectmake(10, 50, 300, 20)]; label.font = [uifont fontwithname:@"courier-bold" size:16.0]; label.backgroundcolor = [uicolor blackcolor]; label.textcolor = [uicolor greencolor]; label.linebreakmode = uilinebreakmodecharacterwrap; // otherwise "…" label.text = @"this led text scrolled "; // blanks space between [self.view addsubview:label]; [nstimer scheduledtimerwithtimeinterval:0.12f target:self selector:@selector(scrolllabel) use...
Read more