[ZODB-Dev] Priorities?

[Jim Fulton]

On Sat, Nov 21, 2009 at 3:59 AM, Christian Theune <ct at gocept.com> wrote:
> when jotting down a thought about bug 240381 I got the feedback that
> caring about the issue isn't among the (current) priorities of ZODB
> development.
>
> Unfortunately, I don't know what those priorities are.

Fair enough. Understand that the highest priorities fall into 2 categories:

1. Documentation

2. Technical priorities

I *want* to write documentation, but I need to get out of the way. I think
that's the best area for people to contribute in the short term.

As far as technical priorities, these are mostly things that I need to work on
and frankly, having other people in the code makes my job harder.
The best way to deal with this is to have any work done by people
other than me done on branches and merged by me.

As far as my priorities:

- Improve the performance of ZEO servers by making them
  multi-threaded. I have a good start on that on the jim-thready-zeo2
  branch.

- Improve the ZEO implementation to make it more robust,
  testable, and pluggable.  This will involve reimplementing it
  using zc.ngi. Among the benefits:

  - Simpler and more robust connection logic

  - Easier to write tests

  - The ability to use other low-level protocols including SSL, and
     my personal favorite favorite, SSH.

- A more scalable alternative to FileStorage.  FileStorage has gone
  a lot further than I ever thought it would, thanks in part to dropping
  RAM prices. :)

  My current project is another stab at a storage build on Berkeley Database.
  Hopefully, I'll have something to report on this in a couple of
weeks. I actually
  have a decent benchmark now to do meaningful analysis.

Longer term, some projects include:

- Allowing multiple transactions to commit at once if they touch
  different objects.

- Adding the ability to have higher consistency for apps that need it
  by optionally raising conflict errors if a transaction reads a value
  that is written by another.

- Adding authentication and authorization to the storage server.
  This would open up a number of new applications.

Another current recent project of note is zc.zodbdgc, which
provides distributes garbage collection. Note that this project
includes a database verification tool that checks all references
from the roots across multiple databases, is blob aware, and
can optionally provide a reference database that let's you track
object references backward.

One important project that can be worked on independently is
a back up tool, either for FileStorage or more generally. One
way to do this would be to write better tests for repozo
and work out if/how it can be used with blobs.

> In addition, when going through the bug tracker this morning I found
> that the triage of bugs isn't executed very well currently and I'm a bit
> wary of trying to do something that will be shot down with a "not our
> priorities" quickly.

Note that when preparing for the 3.9 release, I did quite a bit of triage.
This mostly categorized bugs into those that needed to be fixed in 3.9
and those that didn't. I also asked for input, setting the stage for a lot of
the clean ups you did today.

> I'm trying to make up time to help where I can and I'm happy to do some
> leg work. For example I find it extremely important to have a well
> organised bug database. So one thing I'm trying to do is keep the list
> of untriaged bugs down so that when someone does have time to work on a
> fix for a bug he can easily find those that are important/urgent and
> real issues not duplicates or unreproducable things.

Cool.

> This involves moving bugs out of the "new" status as quickly as
> possible. For example #183203 has been prioritized to medium but is
> still sitting there as new. The content doesn't hint to it as
> "confirmed" so I'd suggest moving this over to "incomplete".

The fact that I set a priority for this indicates that I believe there
is an issue
there.  The thready zeo work will likely overtake this as the affected code
will change quite a bit.

> I find those thoughts to be relatively obvious, but I'd rather not keep
> "contributing" in my spare time with things that aren't of "priority" to
> the project.
>
> And again: asides from the (to me) obvious cleaning of the bug database.
> What are the priorities?

The things that people other than me can do include:

- Documentation
- Documentation
- Documentation
- Get repozo or some other backup solution under control
- Maybe triage/evaluate the script zoo, cleaning out scripts that
  don't have tests and have generally been crapped into the
  scripts directory.
- Work on bug fixes.

Please though, do code work on branches, or separate projects.

Note that I worked n zc.zodbdgc in a separate project even though I
plan to include it with ZODB in the future. For tools, I think this is
a good model (as Jeff argued).

Jim

-- 
Jim Fulton