[Zope-dev] Request for comments: Directory storage

[Jimmie Houchin]

From my naive understanding, would this help with the problem ZODB has
with regard to folders with many objects? Would a person who is using
DirectoryStorage not necessarily be required to partition their objects
into an artificially derived hierarchical directory structure?

In other words can it be a possible solution to
http://www.zope.org/Wikis/zope-dev/ReallyBigFolders ?

Ooo, if so any idea on ETA?

Thanks, 

Jimmie Houchin


Petru Paler wrote:
> 
>           Hello all,
> 
>    You probably saw my yesterday post with the first alpha of
> ReiserStorage. One of the questions that people tend to ask about it is
> wheter they can use it without reiserfs.
>    There are two problems with not using reiserfs:
> 
> 1. ReiserStorage (now renamed to DirectoryStorage) stores each object in a
> separate file and *all* the files in a single directory. This was done in
> order to let the filesystem what it was meant to do: store and retrieve
> files quickly. While reiserfs is *extremely* good at this (it uses a btree
> to store directory entries), most other filesystems do linear searches
> when finding a file so performance is very bad when you have many files in
> a single directory.
>    This problem can be solved by splitting files into multiple directories
> when not using reiserfs. This would add a little overhead but it is
> tolerable.
> 
> 2. Waste of space. Typical block-allocation filesystems like ext2 and FAT
> will waste alot of space in the usage pattern of DirectoryStorage.
> ReiserFS packs small files together in the btree, so it solves the
> problem, but I have no ideea how this could be fixed easyly on the other
> fs's.
> 
> Comments ? Suggestions ?
> 
> PS: a new DirectoryStorage release will be done today, with bugfixes and
> new features.
> 
> -Petru
> 
> _______________________________________________
> Zope-Dev maillist  -  Zope-Dev@zope.org
> http://lists.zope.org/mailman/listinfo/zope-dev
> **  No cross posts or HTML encoding!  **
> (Related lists -
>  http://lists.zope.org/mailman/listinfo/zope-announce
>  http://lists.zope.org/mailman/listinfo/zope )