[ZODB-Dev] grab zodb dict keys by search term

[Tamer Higazi]

Hi Darryl,
Thank you very much for your quick response.

Perhaps it's my fault, the way I stored the data.
The point is, that I am looking the fastest and performant way to grab 
the data from a big pool.

I don't want to iterate the whole thing over and over again to get the 
matched entries.
If i would have 100.000 - 500.000 entries, where 150 would match, I 
don't want to go over those completely.

So, if there is a pythonic way to get it solved on a performant way, or 
a ZODB way as well that would be wondefull.


Thanks a lot



Tamer


On 11.12.2013 03:56, Darryl Dixon - Winterhouse Consulting wrote:
> Hi Tamer,
>
> It depends on what you're trying to achieve. BTrees should not be mutated
> whilst iterating over them, so depending on how you have structured your
> data, you might do something like:
>
> for key in [x for x in BTree.keys() if x.startswith('aa')]:
>      # Do stuff
>
> Or perhaps something else like:
>
> for key in list(BTree.keys()):
>      if key['subkey'] == 5:
>          yield BTree[key]
>
> ...Or some other pattern depending on your use case.
>
> The key here is that the ZODB really allows you to "cut out the middle
> man" where previously you might store data in an RDBMS and then do
> processing/transformation of that data in Python after extracting it with
> SQL (and then re-storing it again afterwards), instead now you can just
> directly process the data as if it was in Python objects waiting the
> entire time.
>
> Hope this helps.
>
> regards,
> Darryl Dixon
> Winterhouse Consulting Ltd
>
>
>> Hi people!
>>
>> I am working a lot with IOB and OOB Trees as well with PersistentDict
>> and PersistentList to store my Data in ZODB.
>> I want to ask if there is a way to grab data from a dataset by search
>> time.
>>
>> I know only the SQL way:
>>
>> select lower(COL1) from table1 where COL1 LIKE 'aa%'
>>
>> so usually I want to get the entryies only that starts with 'aa'
>>
>> do I have to iterate the entire dict with an interator or are there
>> builtin functions that could accomplish those tasks for me ?!
>>
>>
>> I would kindly thank you
>>
>>
>>
>> Tamer
>> _______________________________________________
>> For more information about ZODB, seehttp://zodb.org/
>>
>> ZODB-Dev mailing list  -ZODB-Dev at zope.org
>> https://mail.zope.org/mailman/listinfo/zodb-dev
>>