 




<?xml version="1.0" encoding="UTF-8"?><rss version="2.0"
	xmlns:content="http://purl.org/rss/1.0/modules/content/"
	xmlns:dc="http://purl.org/dc/elements/1.1/"
	xmlns:atom="http://www.w3.org/2005/Atom"
	xmlns:sy="http://purl.org/rss/1.0/modules/syndication/"
		>
<channel>
	<title>Comments on: Running a large number of &#8220;counting QL&#8221; on a noSQL data store. Algorithm or best practice needed!</title>
	<atom:link href="http://itknowledgeexchange.techtarget.com/itanswers/running-a-large-number-of-counting-ql-on-a-nosql-data-store-algorithm-or-a-best-practice-needed/feed/" rel="self" type="application/rss+xml" />
	<link>http://itknowledgeexchange.techtarget.com/itanswers/running-a-large-number-of-counting-ql-on-a-nosql-data-store-algorithm-or-a-best-practice-needed/</link>
	<description></description>
	<lastBuildDate>Fri, 24 May 2013 02:38:15 +0000</lastBuildDate>
	<sy:updatePeriod>hourly</sy:updatePeriod>
	<sy:updateFrequency>1</sy:updateFrequency>
	
	<item>
		<title>By: mshafie</title>
		<link>http://itknowledgeexchange.techtarget.com/itanswers/running-a-large-number-of-counting-ql-on-a-nosql-data-store-algorithm-or-a-best-practice-needed/#comment-87415</link>
		<dc:creator>mshafie</dc:creator>
		<pubDate>Wed, 02 Feb 2011 05:42:41 +0000</pubDate>
		<guid isPermaLink="false">#comment-87415</guid>
		<description><![CDATA[thanks Carlosdl

the application is on the cloud &quot;GAE&quot; , so i think it will not be an issue increasing memory usage few more KBs to hold that Data structure.
but i agree with you code will be more complex and background jobs will be harder to maintain.

maybe this will make us either take solution (1), and maybe make some sort of storing for that Data Structure in GAE data store. but still this is going to make its code highly complex.

again thanks Carlosdl

regards
mohammad]]></description>
		<content:encoded><![CDATA[<p>thanks Carlosdl</p>
<p>the application is on the cloud &#8220;GAE&#8221; , so i think it will not be an issue increasing memory usage few more KBs to hold that Data structure.<br />
but i agree with you code will be more complex and background jobs will be harder to maintain.</p>
<p>maybe this will make us either take solution (1), and maybe make some sort of storing for that Data Structure in GAE data store. but still this is going to make its code highly complex.</p>
<p>again thanks Carlosdl</p>
<p>regards<br />
mohammad</p>
]]></content:encoded>
	</item>
	<item>
		<title>By: carlosdl</title>
		<link>http://itknowledgeexchange.techtarget.com/itanswers/running-a-large-number-of-counting-ql-on-a-nosql-data-store-algorithm-or-a-best-practice-needed/#comment-87370</link>
		<dc:creator>carlosdl</dc:creator>
		<pubDate>Tue, 01 Feb 2011 15:22:28 +0000</pubDate>
		<guid isPermaLink="false">#comment-87370</guid>
		<description><![CDATA[Well, I didn&#039;t know how the Google App Engine worked, but I have read a little and now I have a better understanding of the situation.

This is just a thought, as I have never developed for GAE and thus don&#039;t know of any good practices specific to it, nor do I know the inner details of how the GAE&#039;s data store works.

For a better response time I think your 2nd option (a memory tree structure) would be the best one, unless the GAE offered some kind of caching capability, which could make queries faster when data changes are not frequent, in which case your best option would probably be the first one (the statistics table).

Using a memory structure to store the data will probably be faster (depending on the type of structure and the search algorithms used), but would consume more memory and would add some complexity to the application code.  Also, a background job would probably be needed to handle updates for expiring items.

I would probably tend to prefer this option, but I really think that both approaches should be tested enough, to make a final decision.]]></description>
		<content:encoded><![CDATA[<p>Well, I didn&#8217;t know how the Google App Engine worked, but I have read a little and now I have a better understanding of the situation.</p>
<p>This is just a thought, as I have never developed for GAE and thus don&#8217;t know of any good practices specific to it, nor do I know the inner details of how the GAE&#8217;s data store works.</p>
<p>For a better response time I think your 2nd option (a memory tree structure) would be the best one, unless the GAE offered some kind of caching capability, which could make queries faster when data changes are not frequent, in which case your best option would probably be the first one (the statistics table).</p>
<p>Using a memory structure to store the data will probably be faster (depending on the type of structure and the search algorithms used), but would consume more memory and would add some complexity to the application code.  Also, a background job would probably be needed to handle updates for expiring items.</p>
<p>I would probably tend to prefer this option, but I really think that both approaches should be tested enough, to make a final decision.</p>
]]></content:encoded>
	</item>
	<item>
		<title>By: mshafie</title>
		<link>http://itknowledgeexchange.techtarget.com/itanswers/running-a-large-number-of-counting-ql-on-a-nosql-data-store-algorithm-or-a-best-practice-needed/#comment-87337</link>
		<dc:creator>mshafie</dc:creator>
		<pubDate>Tue, 01 Feb 2011 07:00:24 +0000</pubDate>
		<guid isPermaLink="false">#comment-87337</guid>
		<description><![CDATA[thanks Carlosdl

I am afraid it is not a database. it is Google App Engine Data store.

counting is done to find numbers between brackets in searching filter &quot;narrow search&quot;.
these numbers are real time data , changed by changing data itself.
kindly visit careerbuilder.com and check its narrow filter for more details.
or please tell me if i should explain further more about usage here.



i did not test it yet, but as a cloud computing platform it will cost more if it needs more CPU/... resources.
i want to get the optimum design or best practice here.

regards
mohammad]]></description>
		<content:encoded><![CDATA[<p>thanks Carlosdl</p>
<p>I am afraid it is not a database. it is Google App Engine Data store.</p>
<p>counting is done to find numbers between brackets in searching filter &#8220;narrow search&#8221;.<br />
these numbers are real time data , changed by changing data itself.<br />
kindly visit careerbuilder.com and check its narrow filter for more details.<br />
or please tell me if i should explain further more about usage here.</p>
<p>i did not test it yet, but as a cloud computing platform it will cost more if it needs more CPU/&#8230; resources.<br />
i want to get the optimum design or best practice here.</p>
<p>regards<br />
mohammad</p>
]]></content:encoded>
	</item>
	<item>
		<title>By: carlosdl</title>
		<link>http://itknowledgeexchange.techtarget.com/itanswers/running-a-large-number-of-counting-ql-on-a-nosql-data-store-algorithm-or-a-best-practice-needed/#comment-87285</link>
		<dc:creator>carlosdl</dc:creator>
		<pubDate>Mon, 31 Jan 2011 14:52:12 +0000</pubDate>
		<guid isPermaLink="false">#comment-87285</guid>
		<description><![CDATA[This is part of the question title &quot;&lt;i&gt;Running a large number of &quot;counting QL&quot; &lt;b&gt;on a noSQL data store&lt;/b&gt;&lt;/i&gt;&quot;

But, from the problem description I understand that you are in fact using a database, but you think that executing count queries every time, it will be too slow.  Is my understanding correct ?  If so, what database are you using ?

This is important, as depending on that different methods could be considered.  For example, if Oracle, maybe a materialized view could be a good option (which would be the first option you mentioned), if MySql, maybe a memory table (heap) could be considered.

&quot;&lt;i&gt;...even though i still think this is not enough&lt;/i&gt; &quot;

Have you tested this approaches.  Is there a reason to think this won&#039;t be enough ?]]></description>
		<content:encoded><![CDATA[<p>This is part of the question title &#8220;<i>Running a large number of &#8220;counting QL&#8221; <b>on a noSQL data store</b></i>&#8221;</p>
<p>But, from the problem description I understand that you are in fact using a database, but you think that executing count queries every time, it will be too slow.  Is my understanding correct ?  If so, what database are you using ?</p>
<p>This is important, as depending on that different methods could be considered.  For example, if Oracle, maybe a materialized view could be a good option (which would be the first option you mentioned), if MySql, maybe a memory table (heap) could be considered.</p>
<p>&#8220;<i>&#8230;even though i still think this is not enough</i> &#8221;</p>
<p>Have you tested this approaches.  Is there a reason to think this won&#8217;t be enough ?</p>
]]></content:encoded>
	</item>
	<item>
		<title>By: mshafie</title>
		<link>http://itknowledgeexchange.techtarget.com/itanswers/running-a-large-number-of-counting-ql-on-a-nosql-data-store-algorithm-or-a-best-practice-needed/#comment-87258</link>
		<dc:creator>mshafie</dc:creator>
		<pubDate>Sun, 30 Jan 2011 06:58:07 +0000</pubDate>
		<guid isPermaLink="false">#comment-87258</guid>
		<description><![CDATA[this question is mainly aimed to people interested in:
- high performance computing
- cloud based computing
- algorithms 
- best practice 
- data structure 

hope anyone can help....
any questions or suggestions are welcomed !!!

regards
mohammad s.]]></description>
		<content:encoded><![CDATA[<p>this question is mainly aimed to people interested in:<br />
- high performance computing<br />
- cloud based computing<br />
- algorithms<br />
- best practice<br />
- data structure </p>
<p>hope anyone can help&#8230;.<br />
any questions or suggestions are welcomed !!!</p>
<p>regards<br />
mohammad s.</p>
]]></content:encoded>
	</item>
</channel>
</rss>

<!-- Performance optimized by W3 Total Cache. Learn more: http://www.w3-edge.com/wordpress-plugins/

Page Caching using memcached
Database Caching 3/8 queries in 0.032 seconds using memcached
Object Caching 325/326 objects using memcached

Served from: itknowledgeexchange.techtarget.com @ 2013-05-24 02:53:01 -->