ZODB vs. fs Re: [Zope] Announce - Audio Product

[Peter Sabaini]

Thats just a matter of trust to me. Ext2 has proven to be stable even
if I did nasty things to it. Ext2 and the tools it comes with has had
a lot more exposure, and a lot more time to mature (and besides
Data.fs sits on top of Ext2, so you have the vulnerabilities of
both, in a single point of failure).

The added features of ZODB sure would be handy sometimes, and yes, the
added bookkeeping is painful, but I simply wouldn't want big amounts
of data in ZODB. Or in a RDBMS, for that matter. Use RDBMS or ZODB for
metadata, keep the bulk in lowtech storage...

ru,
peter.


On Thu, 3 May 2001, marc lindahl wrote:

>
>
> > From: Bill Anderson <bill@libc.org>
> >
> > Which is worse: A corrupted Data.fs, or a corrupted .mp3 file? If you
> > get a corruption in the backup process (Yes, it ODES happen), or the
> > restore, which would you rather have?
>
> Well, if it was something like the MP3.COM site, there wouldn't be another
> centralized source for replacements for the .mp3 files, so I don't see the
> issue.  If the backup is corrupted, then you'd go to the next-oldest
> backup...
>
>
>
> >> What do you mean, it refuses to pack?  It gives an error or something?
> >
> >
> > No error, and no pack. I haven't had  a lot of time to go into it ...
>
>
> Anyone else out there have this problem???  Hope it isn't a serious bug...!
>
>
>
>
> >> Also, importantly, version control!
> >
> > you can version control the metadata.
>
> That's another ball of worms - syncronizing with the external data (e.g.
> what if the file moves/renames/changes?  Do you automatically scan?  Or have
> a maintainance procedure of some kind to check?  And have error handling,
> etc.?)  I haven't looked at the code of ExtFile or LocalFS, don't know how
> they deal with that.
>
>
> > pack every day, and you may as well forget about an undo from yesterday.
>
> Right, which is bad for all the other stuff you might want to undo...
>
> >> I'm aiming at uploading of audio as something with a little more control
> >> (like a review process), so having 1000 versions of the same file unpacked
> >> isn't an issue, and the versioning is important.
> >
> > Like I said, you can make it so the metadata is version-controllable.
> > 1000 copies at 2MB=2GB ... for ONE lousy MP3? While _you_ may be fine
> > with that, most people ar enot. Still, it is your choice. Multiply that
> > by, say a hundred mp3s ...
>
>
> Well, I was overstating the case.  But I find it typical that, for example,
> one might mistakenly put up the wrong mix of a song, and have to pull it
> quickly... or that a mistake with some meta data, like publisher name or
> song title, is made once or twice before it's gotten right.
>
> I wonder if, to get super-tweeky, it would be possible for a 'partial pack',
> where you could follow the 'thread' of a particular object or set of
> objects, and just pack those?
>
>
> > Except that you are then you are using quota on the _entire_ Data.fs.
> > With an ExtFile you can have a seperate mountpoint that has a separate
> > quota control. say I only want up to X Gb of mp3 files, but would also
> > like up toX Gb of other objects, such as ExtImage objects. by limiting
> > the data.fs, you can't do that very well. Additionally, maybe I don't
> > want 1.5-2.5 Gb of a single song stored. ;^)~
>
> A related thing I was thinking about was how to limit file size of
> individual files, and individual user quotas.  My answer to that one was:
> 1. filesize would be limited by the 30 minute built-in timeout of zope...
> you can do the math... as long as you have enough storage for one of those
> (for each zope thread) you won't die...  then you could, after upload, check
> the filesize of the file (either internal or external), and if it's too big,
> delete it and warn the user.
> 2. again, you can have methods to scan the user's area and add up his file
> storage, and perhaps disable uploading if it's over.
>
> To me, it seems you could integrate these types of controls better with the
> site functionality if everything is inside zope.
>
>
>
>
> > Anyway, I was just kicking out some real-world reasons whyan ExtMP# type
> > file would be (more) useful. Feel free to disagree, just understand that
> > there are many of us out here here who have very valid reasons for not
> > storing file objects that average over 1MB in the ZODb, especially when
> > we are talking about potentially thousands, as you indicated. I've gone
> > through all these issues in developing a Zope Advertising Product.
>
> I'm looking at this issue keenly, and trying to dig down to the roots on
> these issues... I don't think there are easy answers, but overall I think
> the ZODB is underappreciated, and I'm trying to get a better grip on it's
> limitations.  I'm still not clear on how the size of a python object is a
> factor (as opposed to the number of objects in data.fs), though it's pretty
> clear to me that anything that passes thru zope (e.g. via ExtFile) is at
> some point inside zope and therefore the same, at that point, as something
> stored in data.fs, as far as speed and performance goes.
>
>
>
> _______________________________________________
> Zope maillist  -  Zope@zope.org
> http://lists.zope.org/mailman/listinfo/zope
> **   No cross posts or HTML encoding!  **
> (Related lists -
>  http://lists.zope.org/mailman/listinfo/zope-announce
>  http://lists.zope.org/mailman/listinfo/zope-dev )
>

-- 

_________________________________________________
peter sabaini, mailto: sabaini@niil.at
-------------------------------------------------