I recently came across a bug report regarding configuring zeo clients storage to list multiple zeo servers here
<a href="https://bugs.launchpad.net/zope2/+bug/143843">https://bugs.launchpad.net/zope2/+bug/143843</a> . I had not realized that was possible so I tried it by creating a second zeo server instance with a copy of the Data.fs from the first instance. I then added a second storage server like so:<div>
...</div><div><span style="color:rgb(51,51,51);font-family:'Ubuntu Mono',monospace;font-size:12px;line-height:18px;text-align:left;background-color:rgb(255,255,255)"><zeoclient></span><br style="color:rgb(51,51,51);font-family:'Ubuntu Mono',monospace;font-size:12px;line-height:18px;text-align:left;background-color:rgb(255,255,255)">
<span style="color:rgb(51,51,51);font-family:'Ubuntu Mono',monospace;font-size:12px;line-height:18px;text-align:left;background-color:rgb(255,255,255)"> server localhost:9997</span><br style="color:rgb(51,51,51);font-family:'Ubuntu Mono',monospace;font-size:12px;line-height:18px;text-align:left;background-color:rgb(255,255,255)">
<span style="color:rgb(51,51,51);font-family:'Ubuntu Mono',monospace;font-size:12px;line-height:18px;text-align:left;background-color:rgb(255,255,255)"> server localhost:9998</span>
</div><div><span style="color:rgb(51,51,51);font-family:'Ubuntu Mono',monospace;font-size:12px;line-height:18px;text-align:left;background-color:rgb(255,255,255)">...</span></div><div><span style="color:rgb(51,51,51);font-family:'Ubuntu Mono',monospace;font-size:12px;line-height:18px;text-align:left;background-color:rgb(255,255,255)"><br>
</span></div><div style="text-align:left"><font color="#333333" face="Ubuntu Mono, monospace"><span style="font-size:12px;line-height:18px">To my amazement, the client initially connected to localhost:9997 and when I shutdown that server, the client almost instantly connected to localhost:9998. I could continue switching them off and on and the client switches back and forth. I immediately realized that hot failover might be alot easier than I expected. However, with more testing I run into an issue in zodb.ZEO.ClientStorage.ClientStorage.verify_cache if a there are transactions recorded in the client cache that were not synced up in the secondary zeo server:</span></font></div>
<div style="text-align:left"><font color="#333333" face="Ubuntu Mono, monospace"><span style="font-size:12px;line-height:18px"><br></span></font></div><div style="text-align:left"><font color="#333333"><div style="font-family:'Ubuntu Mono',monospace;font-size:12px;line-height:18px">
elif server_tid < cache_tid:</div><div style="font-family:'Ubuntu Mono',monospace;font-size:12px;line-height:18px"> message = ("%s Client has seen newer transactions than server!"</div>
<div style="font-family:'Ubuntu Mono',monospace;font-size:12px;line-height:18px"> % self.__name__)</div><div style="font-family:'Ubuntu Mono',monospace;font-size:12px;line-height:18px">
logger.critical(message)</div><div style="font-family:'Ubuntu Mono',monospace;font-size:12px;line-height:18px"> raise ClientStorageError(message)</div><div style="font-family:'Ubuntu Mono',monospace;font-size:12px;line-height:18px">
<br></div><div style="font-family:'Ubuntu Mono',monospace;font-size:12px;line-height:18px">would it be so bad to do something like the following?:</div><div style="font-family:'Ubuntu Mono',monospace;font-size:12px;line-height:18px">
<div>elif server_tid < cache_tid:</div><div> message = ("%s Client has seen newer transactions than server!"</div><div> % self.__name__)</div><div> logger.critical(message)</div>
<div> self._cache.clear()</div><div> raise ClientStorageError(message)</div></div><div style="font-family:'Ubuntu Mono',monospace;font-size:12px;line-height:18px"><br></div><div style="font-family:'Ubuntu Mono',monospace">
<span style="font-size:12px;line-height:18px">So an error is raised and logged, but with the cache being cleared so that on the second try it reconnects? My rational for this change is that If your doing a hot failover that means that a) something bad has happened to main server and the recovery of those transactions probably won't happen any way or b) it happened during a maintenance window/the failover is happening for convenience and any difference between the two servers are probably minor, such as session data. With b. the server admin would be in the position to restore the main server anyways.</span></div>
<div style="font-family:'Ubuntu Mono',monospace"><span style="font-size:12px;line-height:18px"><br></span></div><div style="text-align:-webkit-auto"><span style="font-family:'Ubuntu Mono',monospace;font-size:12px;line-height:18px">Here is where the change took place </span><a href="http://svn.zope.org/ZODB/trunk/src/ZEO/ClientStorage.py?rev=93195&view=rev" style="font-family:'Ubuntu Mono',monospace">http://svn.zope.org/ZODB/trunk/src/ZEO/ClientStorage.py?rev=93195&view=rev</a><font face="Ubuntu Mono, monospace"> and I noticed that the rational was to handle 'an odd edge case' and in the tests the comment is that '</font><font face="sans-serif">It is bad if a client has newer data than the server'. If the edge case makes this proposed change a bad idea, would it be reasonable to have the self._cache.clear call as a optional, configurable feature?</font></div>
</font></div>