[Grok-dev] Re: zc.sourcefactory

[Philipp von Weitershausen]

On 4 Sep 2007, at 02:02 , Tres Seaver wrote:
>> Consider Jan Ulrich's example in this very thread:
>>
>> * Jan Ulrich's app -- let's call it 'zoo' for now -- depends on
>> 'grok'. Grok depends on ZODB3==3.8.0b2. So it actually pins down the
>> ZODB, fulfilling the job of the meta/framework egg.
>>
>> * Now Jan Ulrich wants to make use of zc.sourcefactory. 'zoo' now
>> depends on 'grok' and 'zc.sourcefactory'.
>>
>> * 'zc.sourcefactory' OTOH depends on ZODB3 as well. Because it's a
>> *library* (not a meta/framework/application egg) it doesn't and
>> shouldn't pin down a version. zc.buildout isn't smart enough to
>> understand that grok's requirement is more specific than this one.
>
> The bug is in setuptools;  it downloades and *installs* eggs before
> doing the full workingset computation.  The problem is that the
> dependencie metadata is not available until after the egg becomes
> importable (e.g., installed at least to the extent that  
> 'pkg_resources'
> can find it).
>
> If PyPI exposed the dependency information as metadata, which  
> setuptools
> then checked *before* downloading the eggs, this issue would be  
> solvable.

It would be, yes. We'd have to significantly extend PyPI *and*  
setuptools though. I wonder if there's something simpler that we can  
do with less effort.

>> It will actually try and download the latest ZODB3 for  
>> zc.sourcefactory
>> and then bail out with an error because it conflicts with the version
>> pinned down for grok.
>>
>>
>> That, and the fact that it's currently impossible to override any
>> version dependencies that are made with the '==' operator, I consider
>> such pinnings in setup.py both ineffective (because they don't remove
>> degrees of freedom for grok) and hazardous (because they constrain
>> *me* instead).
>
> I'm afraid they are goint to be *necessary*, whether we like them or
> not:  nobody can expect to run against a "wild West" collection of
> distributions without building a packaging culture rivaling Debian's
> (even then, 'unstable' *does* break people's systems).

Oh, I absolutely agree that the "wild west" scenario isnt' going to  
go work for much longer, hence the working set discussion. It's just  
that I don't think pinning things down in setup.py is a solution, at  
least not for libraries. Applications may do so, perhpas, though I  
still think it's a deployment thing so you should at least be able to  
override it (which you currently can't).