[Zope] Vhosting
Adam Getchell
AdamG@hrrm.ucdavis.edu
Mon, 21 Oct 2002 21:13:05 -0700
Sean,
I went through your notes:
I'm running on OpenBSD, and I'm ignoring SSL for now (I'd like that to work
later ...) The latest OpenBSD Squid package was PRE5 ... is PRE10 necessary?
Grabbed pyredir and put:
pyredir in /usr/local/libexec
pyredir.conf in /etc/squid
Went through and commented out pyredir.conf since I want to redirect local
stuff, don't care about CGIs, Ad banners, or Debian stuff.
Since I'm only doing normal http: and I want to redirect
http://ucdra.ucdavis.edu to http://ucdra.ucdavis.edu/ucdra I added this to
pyredir.conf (one line, cut and pasted from server):
^http://ucdra.ucdavis.edu[/]?(.*)=http://ucdra.ucdavis.edu:8080/ucdra/Virtua
lHos
tBase/http/ucdra.ucdavis.edu/VirtualHostRoot/\1
I then went into /etc/squid/squid.conf and changed:
redirect_program /usr/local/libexec/pyredir
redirect_children 10
I grabbed squidctl and put it in /root, made it executable, and changed
paths for the binary to /usr/local/bin/squid, but it doesn't detect the pid
of my current squid, so I left that alone. I can still configure using just
/usr/local/bin/squid
I rebooted my server, and if I understand VHM and the rewrites above, that
should be it. However, no rewriting is occuring as far as I can tell, so
where did I go wrong?
Thanks,
-----Original Message-----
From: sean.upton@uniontrib.com [mailto:sean.upton@uniontrib.com]
Sent: Friday, August 23, 2002 5:28 PM
To: AdamG@hrrm.ucdavis.edu; creiman@kefta.com; marc@bowery.com;
quentins@comclub.dyndns.org
Cc: zope@zope.org
Subject: RE: [Zope] Vhosting
How 'bout a slightly different approach:
Setup Squid on port 80; use a redirector. Search freshmeat for pyredir;
it's a good one, written in python, and very simple to use. The redirector
is your friend, and makes virtual hosting work quite nicely; it acts like
Apache's mod_rewrite, and plays well witha Virtual Host Monster.
Instructions assume you use Zope with a VHM, and perhaps (might) want SSL
and load-balancing for ZEO. Setup requirements: Squid, Zope, use of a VHM,
use of a redirector, and optionally OpenSSL if you want SSL accel.
Details:
1 - Prerequisites: OpenSSL; realpath and stat commands; Chris McDonough's
squidctl script (find on the squid-users mailing list); pyredir
2 - Go download Squid 2.5pre10 source; untar in a directory of your choice.
3 - ./configure --prefix=/usr/local/squid --enable-dlmalloc
--enable-pthreads --enable-storio=aufs
--with-openssl=/usr/lib/include/openssl --disable-internal-dns
--disable-identd-lookups --enable-ssl
--> This is for Linux; you may want to adjust
--enable-pthreads and use of aufs, since I think
you want to use diskd instead on BSD
--> Change path to openssl include files
4 - make && make install
--> Squid will install in /usr/local/squid ($PREVIX)
--> Squid.conf will be in /usr/local/squid/etc
--> Sometime between Squid 2.5pre5 and Squid2.5pre10, the dir layout
switched so that the squid binary is in $PREFIX/sbin
5 - Put the pyredir script in $PREFIX/libexec
6 - Put the pyredir.conf file in $PREFIX/etc
7 - Edit/hack pyredir:
--> Disable redirector logging in the code if you have a
big server (perhaps later, after this works?)
--> Change path to log and pyredir.conf file
--> Make sure all needed commands are supported (GET,POST,HEAD,etc)
8 - Put a VHM in the folder(s) you want to act as your host.
9 - Edit pyredir.conf:
#this rule is for the public url mysite1.foo.com, which should
#be accessed via SSL on port 443 on Squid...
#it says ^http:// becuase squid passes the URL to the redirector
#with the https stripped off so it is more like a normal URL.
#note: there is a Zope VHM in the MySite1 folder
^http://mysite1.foo.com[/]?(.*)
=http://zopeserver:8080/MySite1/VirtualHostBase/https/mysite1.foo.com:443/My
Site1/VirtualHostRoot/\1
#Note that this one is http (not https) in the rewrite rule passed
eventually to the virtual host monster
^http://mysite2.foo.com[/]?(.*)
=http://zopeserver:8080/MySite2/VirtualHostBase/http/mysite2.foo.com:443/MyS
ite2/VirtualHostRoot/\1
10 - Add "zopeserver" to /etc/hosts
--> if you have a ZEO cluster, add the name zopeserver for every IP
in your cluster; it will be round-robined
--> Squid uses dnsserver (a resolver helper child program) for
/etc/hosts support since its internal resolver only works with DNS
--> this is why we compiled with --disable-internal-dns
11 - Set up any ssl keys/certs you need with openssl.
12 - Setup squid:
http_port 80
#You will need to setup these keys with openssl first:
https_port 443 cert=/usr/local/squid/var/mysite1.foo.com_cert.pem
key=/usr/local/squid/var/mysite1.foo.com_key.pem
cache_dns_program /usr/local/squid/libexec/dnsserver
dns_children 5
redirect_program /usr/local/squid/libexec/pyredir
redirect_children 12
redirect_rewrites_host_header off
httpd_accel_host virtual
httpd_accel_port 80
httpd_accel_single_host off
httpd_accel_uses_host_header on
#obviously there are other things to set up, these
#are the interesting/specific ones
13 - Download and setup the squidctl script in /usr/local/squid
--> you will have to change the path to reflect that the squid
binary is in $PREFIX/sbin
--> hack/adjust as necessary
14 - Chreate the cache dir; make sure permissions are okay for
nobody/nogroup.
--> Start squid; pay attention to any warnings about misconfig, and
fix
15 - Adjust:
--> ACLs in squid.conf as needed.
--> Pyredir rules in pyredir.conf
'killall -s HUP pyredir' will reload rules without a squid
restart
--> SSL config, if needed
16 - Once squid is going, make sure DNS or hosts on your client points to
your accelerator box, and try the URL...
17 - Once you have everything working, if you have a ZEO cluster, give
consideration to trying the ICP patches for Zope to allow for true
load-balancing by adding Zope servers to Squid's notion of its caching
fabric...
18 - Email me with questions if anything doesn't work.
I plan on trying to turn these really rough notes into a howto at some point
on Zope.org, but I'm too busy at the moment. Perhaps discussion here on the
list will help me with that process.
Questions, thoughts?
Sean
-----Original Message-----
From: Adam Getchell [mailto:AdamG@hrrm.ucdavis.edu]
Sent: Friday, August 23, 2002 3:51 PM
To: 'Charlie Reiman'; Marc Lindahl; Quentin Smith; Adam Getchell
Cc: zope@zope.org
Subject: RE: [Zope] Vhosting
Hello all,
Okay, so I read the links you gave me.
http://www.zope.org/Documentation/Books/ZopeBook/2_6Edition/VirtualHosting.s
tx
http://www.zope.org/Members/Jace/apache-vhm
http://www.zope.org/Members/bowerymarc/squid-zserver-virtual
Starting from scratch, I created a zope user on my OpenBSD box, with a home
directory of /home/zope. I then ran
# /usr/local/bin/zope-instance /home/zope
as root, to create an instance of Zope.
# chmod -R zope:nobody /home/zope
# su zope
# /home/zope/start &
To start Zope as user zope. I then logged into the site, created my /site1
and /site2 folders, and created a Virtual Host monster with mappings of:
site1.ucdavis.edu/site1
site2.ucdavis.edu/site2
At this point, going to site1.ucdavis.edu goes to the main website, while
going to site1.ucdavis.edu:8080 goes to the subfolder. So I think my Squid
configuration is off.
I went through /etc/squid/squid.conf and have the following options enabled
(not commented out):
http_port 80
acl QUERY urlpath_regex cgi-bin \?
no_cache deny QUERY
cache_dir diskd /var/squid/cache 100 16 256
emulate_httpd_log on
auth_param basic children 5
auth_param basic real Squid proxy-caching web server
auth_param basic credentialsttl 2 hours
httpd_accel_host my.ip.
httpd_accel_port 8080
httpd_accel_single_host on
I'm confused about the ACL's in step 4 of
http://www.zope.org/Members/bowerymarc/squid-zserver-virtual
I don't see an acl of type webserver in squid.conf ... So what should that
line be?
acl ext-ip-addr site1.ucdavis.edu www.xxx.yyy.zzz/255.255.255.255
And then the ACL is ...?
Http_access allow MATCH ext-ip-addr
Clearly, the zope side of things are working so far. What is the
SiteAccessEnhanced used for? Do I still need a site root in each virtual
folder?
Many thanks,
--Adam
-----Original Message-----
From: Charlie Reiman [mailto:creiman@kefta.com]
Sent: Friday, August 23, 2002 9:06 AM
To: Marc Lindahl; Quentin Smith; Adam Getchell
Cc: zope@zope.org
Subject: RE: [Zope] Vhosting
You should also read the 2.6 docs on VHM. They're downright tasty.
http://www.zope.org/Documentation/Books/ZopeBook/2_6Edition/VirtualHosting.s
tx
They apply to 2.5.1 as far as I noticed.
> -----Original Message-----
> From: zope-admin@zope.org [mailto:zope-admin@zope.org]On Behalf Of
> Marc Lindahl
> Sent: Thursday, August 22, 2002 11:42 PM
> To: Quentin Smith; Adam Getchell
> Cc: 'zope@zope.org'
> Subject: Re: [Zope] Vhosting
>
>
> Try: http://www.zope.org/Members/bowerymarc/squid-zserver-virtual
>
>
> on 8/22/02 10:15 PM, Quentin Smith at quentins@comclub.dyndns.org
> scrivened:
>
> > Hi-
> > Use a Virtual Host Monster, one in the root of the site. Ideally, you
> > should put Apache or Pound in front of Zope instead of squid, but you
> > can use the Mappings tab of the Virtual Host Monster to define the
> > virtual hosts. I used to use SiteRoots for my site, and none of the
> > directions I found actually worked.
> > HTH,
> > --Quentin
> > On Thursday, August 22, 2002, at 08:27 PM, Adam Getchell wrote:
> >
> >> Hello all,
> >>
> >> I've read and re-read
> >> http://www.zope.org/Members/4am/SiteAccess2/vhosting a
> >> number of times.
> >>
> >> I have a webserver that needs to host a bunch of different sites,
> >> site1.ucdavis.edu, site2.ucdavis.edu, etc.
> >>
> >> I planned to use name based redirection, so I have c-names to the IP
> >> address
> >> of the server, i.e. site1.ucdavis.edu --> IP, site2.ucdavis.edu --> IP,
> >> etc.
> >>
> >> I installed on OpenBSD, so Zope runs as an instance in /home/zoperoot
> >> under
> >> port 8080, and Squid sits in front of it and redirects from 80 to port
> >> 8080.
> >> This part is working fine.
> >>
> >> Under my main zope site, I created two folders, site1 and site2. I then
> >> created, in each folder, an empty SiteRoot. That is, Title, Base, and
> >> Path
> >> are all blank. At least, that's how I interpreted the directions above.
> >>
> >> I then created a DTML method called host_redirector, with the following
> >> content:
> >>
> >> Extract the part of HTTP_HOST we care about, and declare our rewrite
> >> dictionary.
> >> <dtml-let hostname="_.string.upper(_.string.split(HTTP_HOST, '.')[0])"
> >> sitemap="{'SITE1': 'site1',
> >> 'SITE2': 'site2'}">
> >> Do we have a match?
> >> <dtml-if expr="sitemap.has_key(hostname)">
> >> Set the logical root: <dtml-call "REQUEST.set('SiteRootPATH', '/')">
> >> Add physical root: <dtml-call
> >> "REQUEST['TraversalRequestNameStack'].append(sitemap[hostname])">
> >> </dtml-if>
> >> </dtml-let>
> >>
> >> Since I only care about the first part of the name, I rewrote
> the script
> >> from the example. It may be incorrect! I tried the equivalent at a
> >> python
> >> interpreter, but I may certainly have goofed up.
> >>
> >> In the root folder, I Set an Access rule to host_redirector.
> >>
> >> I changed the index pages in each of Site1 and Site2 to differentiate
> >> them.
> >> However, going to site1.ucdavis.edu gives the top level index_html,
> >> while
> >> going to site1.ucdavis.edu/site1 gives index_html in the Site1 folder,
> >> so
> >> clearly I did something wrong.
> >>
> >> Any pointers?
> >>
> >> ***************************
> >> * Adam Getchell
> >> AdamG@hrrm.ucdavis.edu
> >> * System Architect/Programmer (530) 752-1584
> >> * Human Resources Information Systems
> >> http://www.hr.ucdavis.edu/
> >> ***************************
> >> "Invincibility is in oneself, vulnerability in the opponent."
> -- Sun Tzu
> >>
> >>
> >>
> >>
> >> _______________________________________________
> >> Zope maillist - Zope@zope.org
> >> http://lists.zope.org/mailman/listinfo/zope
> >> ** No cross posts or HTML encoding! **
> >> (Related lists -
> >> http://lists.zope.org/mailman/listinfo/zope-announce
> >> http://lists.zope.org/mailman/listinfo/zope-dev )
> >>
> >
> >
> > _______________________________________________
> > Zope maillist - Zope@zope.org
> > http://lists.zope.org/mailman/listinfo/zope
> > ** No cross posts or HTML encoding! **
> > (Related lists -
> > http://lists.zope.org/mailman/listinfo/zope-announce
> > http://lists.zope.org/mailman/listinfo/zope-dev )
> >
>
>
> _______________________________________________
> Zope maillist - Zope@zope.org
> http://lists.zope.org/mailman/listinfo/zope
> ** No cross posts or HTML encoding! **
> (Related lists -
> http://lists.zope.org/mailman/listinfo/zope-announce
> http://lists.zope.org/mailman/listinfo/zope-dev )
>
_______________________________________________
Zope maillist - Zope@zope.org
http://lists.zope.org/mailman/listinfo/zope
** No cross posts or HTML encoding! **
(Related lists -
http://lists.zope.org/mailman/listinfo/zope-announce
http://lists.zope.org/mailman/listinfo/zope-dev )