Tiny datablock in saving Hadoop distributed file system wasted memory

<p>Hadoop distributed file system (HDFS) is the file system whereby Hadoop is use it to store all the upcoming data inside it. Since it been declared, HDFS is consuming a huge memory amount in order to serve a normal dataset. Nonetheless, the current file saving mechanism in HDFS save only one file in one datablock. Thus, a file with just 5 Mb in size will take up the whole datablock capacity causing the rest of the memory unavailable for other upcoming files, and this is considered a huge waste of memory in serving a normal size dataset. This paper proposed a method called tiny datablock-HDFS (TD-HDFS) to increase the usability of HDFS memory and increase the file hosting capabilities by reducing the datablock size to the minimum capacity, and then merging all the related datablocks into one master datablock. This master datablock consists of tiny virtual datablocks that contain the related small files together; will exploit the full memory of the master datablock. The result of this study is a running HDFS with a minimum amount of wasted memory with the same read/write data performance. The results were examined through a comparison between the standard HDFS file hosting and the proposed solution of this study.</p><textarea id="BFI_DATA" style="width: 1px; height: 1px; display: none;"></textarea><textarea id="BFI_DATA" style="width: 1px; height: 1px; display: none;"></textarea><textarea id="BFI_DATA" style="width: 1px; height: 1px; display: none;"></textarea><textarea id="BFI_DATA" style="width: 1px; height: 1px; display: none;"></textarea><div id="WidgetFloaterPanels" class="LTRStyle" style="display: none; text-align: left; direction: ltr; visibility: hidden;"><div id="WidgetFloater" style="display: none;" onmouseover="Microsoft.Translator.OnMouseOverFloater()" onmouseout="Microsoft.Translator.OnMouseOutFloater()"><div id="WidgetLogoPanel"><span id="WidgetTranslateWithSpan"><span>TRANSLATE with </span><img id="FloaterLogo" alt="" /></span> <span id="WidgetCloseButton" title="Exit Translation" onclick="Microsoft.Translator.FloaterOnClose()">x</span></div><div id="LanguageMenuPanel"><div class="DDStyle_outer"><input id="LanguageMenu_svid" style="display: none;" onclick="this.select()" type="text" name="LanguageMenu_svid" value="en" /> <input id="LanguageMenu_textid" style="display: none;" onclick="this.select()" type="text" name="LanguageMenu_textid" /> <span id="__LanguageMenu_header" class="DDStyle" onclick="return LanguageMenu &amp;&amp; !LanguageMenu.Show('__LanguageMenu_popup', event);" onkeydown="return LanguageMenu &amp;&amp; !LanguageMenu.Show('__LanguageMenu_popup', event);">English</span><div style="position: relative; text-align: left; left: 0;"><div style="position: absolute; ;left: 0px;"><div id="__LanguageMenu_popup" class="DDStyle" style="display: none;"><table id="LanguageMenu" border="0"><tbody><tr><td><a onclick="return LanguageMenu.onclick('ar');" tabindex="-1" href="#ar">Arabic</a></td><td><a onclick="return LanguageMenu.onclick('he');" tabindex="-1" href="#he">Hebrew</a></td><td><a onclick="return LanguageMenu.onclick('pl');" tabindex="-1" href="#pl">Polish</a></td></tr><tr><td><a onclick="return LanguageMenu.onclick('bg');" tabindex="-1" href="#bg">Bulgarian</a></td><td><a onclick="return LanguageMenu.onclick('hi');" tabindex="-1" href="#hi">Hindi</a></td><td><a onclick="return LanguageMenu.onclick('pt');" tabindex="-1" href="#pt">Portuguese</a></td></tr><tr><td><a onclick="return LanguageMenu.onclick('ca');" tabindex="-1" href="#ca">Catalan</a></td><td><a onclick="return LanguageMenu.onclick('mww');" tabindex="-1" href="#mww">Hmong Daw</a></td><td><a onclick="return LanguageMenu.onclick('ro');" tabindex="-1" href="#ro">Romanian</a></td></tr><tr><td><a onclick="return LanguageMenu.onclick('zh-CHS');" tabindex="-1" href="#zh-CHS">Chinese Simplified</a></td><td><a onclick="return LanguageMenu.onclick('hu');" tabindex="-1" href="#hu">Hungarian</a></td><td><a onclick="return LanguageMenu.onclick('ru');" tabindex="-1" href="#ru">Russian</a></td></tr><tr><td><a onclick="return LanguageMenu.onclick('zh-CHT');" tabindex="-1" href="#zh-CHT">Chinese Traditional</a></td><td><a onclick="return LanguageMenu.onclick('id');" tabindex="-1" href="#id">Indonesian</a></td><td><a onclick="return LanguageMenu.onclick('sk');" tabindex="-1" href="#sk">Slovak</a></td></tr><tr><td><a onclick="return LanguageMenu.onclick('cs');" tabindex="-1" href="#cs">Czech</a></td><td><a onclick="return LanguageMenu.onclick('it');" tabindex="-1" href="#it">Italian</a></td><td><a onclick="return LanguageMenu.onclick('sl');" tabindex="-1" href="#sl">Slovenian</a></td></tr><tr><td><a onclick="return LanguageMenu.onclick('da');" tabindex="-1" href="#da">Danish</a></td><td><a onclick="return LanguageMenu.onclick('ja');" tabindex="-1" href="#ja">Japanese</a></td><td><a onclick="return LanguageMenu.onclick('es');" tabindex="-1" href="#es">Spanish</a></td></tr><tr><td><a onclick="return LanguageMenu.onclick('nl');" tabindex="-1" href="#nl">Dutch</a></td><td><a onclick="return LanguageMenu.onclick('tlh');" tabindex="-1" href="#tlh">Klingon</a></td><td><a onclick="return LanguageMenu.onclick('sv');" tabindex="-1" href="#sv">Swedish</a></td></tr><tr><td><a onclick="return LanguageMenu.onclick('en');" tabindex="-1" href="#en">English</a></td><td><a onclick="return LanguageMenu.onclick('ko');" tabindex="-1" href="#ko">Korean</a></td><td><a onclick="return LanguageMenu.onclick('th');" tabindex="-1" href="#th">Thai</a></td></tr><tr><td><a onclick="return LanguageMenu.onclick('et');" tabindex="-1" href="#et">Estonian</a></td><td><a onclick="return LanguageMenu.onclick('lv');" tabindex="-1" href="#lv">Latvian</a></td><td><a onclick="return LanguageMenu.onclick('tr');" tabindex="-1" href="#tr">Turkish</a></td></tr><tr><td><a onclick="return LanguageMenu.onclick('fi');" tabindex="-1" href="#fi">Finnish</a></td><td><a onclick="return LanguageMenu.onclick('lt');" tabindex="-1" href="#lt">Lithuanian</a></td><td><a onclick="return LanguageMenu.onclick('uk');" tabindex="-1" href="#uk">Ukrainian</a></td></tr><tr><td><a onclick="return LanguageMenu.onclick('fr');" tabindex="-1" href="#fr">French</a></td><td><a onclick="return LanguageMenu.onclick('ms');" tabindex="-1" href="#ms">Malay</a></td><td><a onclick="return LanguageMenu.onclick('ur');" tabindex="-1" href="#ur">Urdu</a></td></tr><tr><td><a onclick="return LanguageMenu.onclick('de');" tabindex="-1" href="#de">German</a></td><td><a onclick="return LanguageMenu.onclick('mt');" tabindex="-1" href="#mt">Maltese</a></td><td><a onclick="return LanguageMenu.onclick('vi');" tabindex="-1" href="#vi">Vietnamese</a></td></tr><tr><td><a onclick="return LanguageMenu.onclick('el');" tabindex="-1" href="#el">Greek</a></td><td><a onclick="return LanguageMenu.onclick('no');" tabindex="-1" href="#no">Norwegian</a></td><td><a onclick="return LanguageMenu.onclick('cy');" tabindex="-1" href="#cy">Welsh</a></td></tr><tr><td><a onclick="return LanguageMenu.onclick('ht');" tabindex="-1" href="#ht">Haitian Creole</a></td><td><a onclick="return LanguageMenu.onclick('fa');" tabindex="-1" href="#fa">Persian</a></td><td> </td></tr></tbody></table><img style="height: 7px; width: 17px; border-width: 0px; left: 20px;" alt="" /></div></div></div></div><script type="text/javascript">// <![CDATA[ var LanguageMenu; var LanguageMenu_keys=["ar","bg","ca","zh-CHS","zh-CHT","cs","da","nl","en","et","fi","fr","de","el","ht","he","hi","mww","hu","id","it","ja","tlh","ko","lv","lt","ms","mt","no","fa","pl","pt","ro","ru","sk","sl","es","sv","th","tr","uk","ur","vi","cy"]; var LanguageMenu_values=["Arabic","Bulgarian","Catalan","Chinese Simplified","Chinese Traditional","Czech","Danish","Dutch","English","Estonian","Finnish","French","German","Greek","Haitian Creole","Hebrew","Hindi","Hmong Daw","Hungarian","Indonesian","Italian","Japanese","Klingon","Korean","Latvian","Lithuanian","Malay","Maltese","Norwegian","Persian","Polish","Portuguese","Romanian","Russian","Slovak","Slovenian","Spanish","Swedish","Thai","Turkish","Ukrainian","Urdu","Vietnamese","Welsh"]; var LanguageMenu_callback=function(){ }; var LanguageMenu_popupid='__LanguageMenu_popup'; // ]]></script></div><div id="CTFLinksPanel"><span id="ExternalLinksPanel"><a id="HelpLink" title="Help" href="https://go.microsoft.com/?linkid=9722454" target="_blank"> <img id="HelpImg" alt="" /></a> <a id="EmbedLink" title="Get this widget for your own site" href="javascript:Microsoft.Translator.FloaterShowEmbed()"> <img id="EmbedImg" alt="" /></a> <a id="ShareLink" title="Share translated page with friends" href="javascript:Microsoft.Translator.FloaterShowSharePanel()"> <img id="ShareImg" alt="" /></a> </span></div><div id="FloaterProgressBar"> </div></div><div id="WidgetFloaterCollapsed" style="display: none;" onmouseover="Microsoft.Translator.OnMouseOverFloater()"><span>TRANSLATE with </span><img id="CollapsedLogoImg" alt="" /></div><div id="FloaterSharePanel" style="display: none;"><div id="ShareTextDiv"><span id="ShareTextSpan"> COPY THE URL BELOW </span></div><div id="ShareTextboxDiv"><input id="ShareTextbox" onclick="this.select()" type="text" name="ShareTextbox" readonly="readonly" /> <!--a id="TwitterLink" title="Share on Twitter"> <img id="TwitterImg" /></a> <a-- id="FacebookLink" title="Share on Facebook"> <img id="FacebookImg" /></a--> <a id="EmailLink" title="Email this translation"> <img id="EmailImg" alt="" /></a></div><div id="ShareFooter"><span id="ShareHelpSpan"><a id="ShareHelpLink"> <img id="ShareHelpImg" alt="" /></a></span> <span id="ShareBackSpan"><a id="ShareBack" title="Back To Translation" href="javascript:Microsoft.Translator.FloaterOnShareBackClick()"> Back</a></span></div><input id="EmailSubject" type="hidden" name="EmailSubject" value="Check out this page in {0} translated from {1}" /> <input id="EmailBody" type="hidden" name="EmailBody" value="Translated: {0}%0d%0aOriginal: {1}%0d%0a%0d%0aAutomatic translation powered by Microsoft® Translator%0d%0ahttp://www.bing.com/translator?ref=MSTWidget" /> <input id="ShareHelpText" type="hidden" value="This link allows visitors to launch this page and automatically translate it to {0}." /></div><div id="FloaterEmbed" style="display: none;"><div id="EmbedTextDiv"><span id="EmbedTextSpan">EMBED THE SNIPPET BELOW IN YOUR SITE</span> <a id="EmbedHelpLink" title="Copy this code and place it into your HTML."> <img id="EmbedHelpImg" alt="" /></a></div><div id="EmbedTextboxDiv"><input id="EmbedSnippetTextBox" onclick="this.select()" type="text" name="EmbedSnippetTextBox" value="&lt;div id='MicrosoftTranslatorWidget' class='Dark' style='color:white;background-color:#555555'&gt;&lt;/div&gt;&lt;script type='text/javascript'&gt;setTimeout(function(){var s=document.createElement('script');s.type='text/javascript';s.charset='UTF-8';s.src=((location &amp;&amp; location.href &amp;&amp; location.href.indexOf('https') == 0)?'https://ssl.microsofttranslator.com':'http://www.microsofttranslator.com')+'/ajax/v3/WidgetV3.ashx?siteData=ueOIGRSKkd965FeEGM5JtQ**&amp;ctf=true&amp;ui=true&amp;settings=manual&amp;from=en';var p=document.getElementsByTagName('head')[0]||document.documentElement;p.insertBefore(s,p.firstChild); },0);&lt;/script&gt;" readonly="readonly" /></div><div id="EmbedNoticeDiv"><span id="EmbedNoticeSpan">Enable collaborative features and customize widget: <a href="http://www.bing.com/widget/translator" target="_blank">Bing Webmaster Portal</a></span></div><div id="EmbedFooterDiv"><span id="EmbedBackSpan"><a title="Back To Translation" href="javascript:Microsoft.Translator.FloaterOnEmbedBackClick()">Back</a></span></div></div><script type="text/javascript">// <![CDATA[ var intervalId = setInterval(function () { if (MtPopUpList) { LanguageMenu = new MtPopUpList(); var langMenu = document.getElementById(LanguageMenu_popupid); var origLangDiv = document.createElement("div"); origLangDiv.id = "OriginalLanguageDiv"; origLangDiv.innerHTML = "<span id='OriginalTextSpan'>ORIGINAL: </span><span id='OriginalLanguageSpan'></span>"; langMenu.appendChild(origLangDiv); LanguageMenu.Init('LanguageMenu', LanguageMenu_keys, LanguageMenu_values, LanguageMenu_callback, LanguageMenu_popupid); window["LanguageMenu"] = LanguageMenu; clearInterval(intervalId); } }, 1); // ]]></script></div>