database - SQL server scalability question -


we trying build application have store billions of records. 1 trillion+

a single record contain text data , meta data text document.

pl me understand storage limitations. can databse sql or oracle support data or have other filesystem based solution ? options ?

since central server has handle incoming load many clients, how parallel insertions , search scale ? how distribute data on multiple databases or tables ? little green database specifics such scaled environment.

initally fill database insert load high, later database grows, search load increase , inserts reduce.

the total size of data cross 1000 tb.

thanks.

1 trillion+

a single record contain text data , meta data text document.

pl me understand storage limitations

i hope have big budget hardware. big in "millions".

a trillion documents, @ 1024 bytes total storage per document (very unlikely realistic when text) size of 950 terabyte of data. storage limitations means talk high end san here. using non-redundant setup of 2tb discs 450 discs. make maths. adding redundancy / raid , talk major hardware invesment. assumes 1kb per document. if have on average 16kg data usage, is... 7200 2tb discs.

that hardware problem start with. sql server not scale high, , can not in single system anyway. normal approach docuemnt store clustered storage system (clustered or somehow distributed file system) plus central database keywords / tagging. depending on load / inserts possibly replciations of hte database distributed search.

whatever going be, storage / backup requiments terrific. lagre project here, large budget.

io load gong issue - hardware wise. need large machine , ton of io bandwidth it. have seen 8gb links overloaded on sql server (fed hp eva 190 discs) , can imagine run similar. want hardware ram technically possible, regardless of price - unless store blobs outside.

sql row compression may come in handy. full text search problem.

the total size of data cross 1000 tb.

no. seriously. bigger, think. 1000tb assume documents small - xml form of travel ticket.


Comments

Post a Comment

Popular posts from this blog

asp.net - repeatedly call AddImageUrl(url) to assemble pdf document -

i'm using abcpdf , i'm curious if can recursively call addimageurl() function assemble pdf document compile multiple urls? something like: int pagecount = 0; int theid = thedoc.addimageurl("http://stackoverflow.com/search?q=abcpdf+footer+page+x+out+of+", true, 0, true); //assemble document while (thedoc.chainable(theid)) { thedoc.page = thedoc.addpage(); theid = thedoc.addimagetochain(theid); } pagecount = thedoc.pagecount; console.writeline("1 document page count:" + pagecount); //flatten document (int = 1; <= pagecount; i++) { thedoc.pagenumber = i; thedoc.flatten(); } //now try again theid = thedoc.addimageurl("http://stackoverflow.com/questions/1980890/pdf-report-generation", true, 0, true); //assemble document while (thedoc.chainable(theid)) { thedoc.page = thedoc.addpage(); theid = thedoc.addimagetoc
Read more

java - Android recognize cell phone with keyboard or not? -

in android app, want detect user's device if device has keyboard(like motorola milestone) , show corresponding user interface. guys know how that? ! regards justicepenny http://developer.android.com/reference/android/content/res/configuration.html public int keyboard the kind of keyboard attached device. 1 of: keyboard_nokeys, keyboard_qwerty, keyboard_12key. that's it!
Read more

iphone - How would you achieve a LED Scrolling effect? -

how achieve led scrolling effect example below? led sign - led ticker emulator iphone , ipad http://img.skitch.com/20101201-rsfh4p1bajb1wp94k466pdpiuj.preview.jpg sometimes remunerates comment in other posts :) have put code you, should implement font '8pin matrix' realistic feeling. uilabel *label; - (void)viewdidload { [super viewdidload]; label = [[uilabel alloc]initwithframe:cgrectmake(10, 50, 300, 20)]; label.font = [uifont fontwithname:@"courier-bold" size:16.0]; label.backgroundcolor = [uicolor blackcolor]; label.textcolor = [uicolor greencolor]; label.linebreakmode = uilinebreakmodecharacterwrap; // otherwise "…" label.text = @"this led text scrolled "; // blanks space between [self.view addsubview:label]; [nstimer scheduledtimerwithtimeinterval:0.12f target:self selector:@selector(scrolllabel) use
Read more