Nowadays, you can see too much of internet resources are consumed by useless garbage like spams, virus traffics, circular forwarded useless e-mails and many other stuff….
Its administrator’s job to decrease internet traffic with using power of accelerators, caches, antispams and software’s like that….
Today I installed new web-cache server, the most famous one SQUID. About a year ago I installed one on OpenBSD with a Compaq proliat, but the server was not so powerful to handle all demands as fast as you can feel it. The company bought Two Compaq G3 servers, I installed one as corporate mail server using Qmail on FreeBSD, the last one that I was working on today, I used it as Network Gateway and Firewall, I’m sure it can handle all demands, so I’m not worry about that.
The reason I’m writing this is to tell you some directions to make your own cache server, you can easily find more resources on the net which can help you to install SQUID and FreeBSD and I don’t want to duplicate all of them. I just give you some useful directions.
FreeBSD or OpenBSD?
I prefer to use OpenBSD because its very fast and minimized, and I believe its very secure, though FreeBSD is good and secure, both are very stable, I am using both of them when I work with OpenBSD I feel better, that’s the feeling….I don’t want to argument 😉 …
OpenBSD has its own pf firewall but on FreeBSD you have to use pf as a port and you may face with some difficulties but it works.
The most important advantage of FreeBSD is that it can be installed on Smart Array, if you want to install you OS on a new generation servers which use Smart Array, Mirroring/RAID systems you have to forget about OpenBSD at least for these available versions.
Try to get the latest STABLE version of your OS and check the MD5 checksum then install it once to and see if everything is normal then continue the procedure, the OS I recently used to install my SQUID was FreeBSD 5.2.1 Release version
http://www.freebsd.org/ is official FreeBSD website and http://www.openbsd.org/ is for OpenBSD.
What else you need?
You need to get the pf port from http://pf4freebsd.love2party.net/ but its easier to install it through ports, you can follow my steps if you have your OS installed and connected to Internet:
1. cd /usr/ports/security/pf
2. make
Then it will start downloading pf from http://pf4freebsd.love2party.net/
3. make install
4. cp /usr/local/etc/pf.conf.default /etc/pf.conf
Finished, now have pf downloaded and installed, but your kernel is not capable of using pf, you need to rebuild and install a new kernel capable of using pf firewall, see here to find more information about compiling FreeBSD kernel. If you are not going to install a transparent webcache/proxy server you can easily jump over pf installation.
You need to add these items in you kernel configuration file:
device bpf
options PFIL_HOOKS
options RANDOM_IP_ID
you don’t need to memorize these items because as soon as you install pf it will show you these items which must be enabled in your OS kernel, then it will ask you if you need to do all changes in you rc.conf files, I suggest you to have a copy of your rc.conf but I usually trust it, it doesn’t add too many lines to the rc.conf file, these items will be added to rc.conf:
pf_enable=”YES”
pf_logd=”YES”
pf_conf=”/etc/pf.conf”
Installer might set pf_conf to different location, check rc.conf, and set the correct one, if you are following this document you should set it to “/etc/pf.conf”
Then save the followinf shell script as /etc/rc.d/pf and make it executable and read-only:
#!/bin/sh
# PROVIDE: pf
# REQUIRE: DAEMON
# KEYWORD: FreeBSD
/usr/local/etc/rc.d/pf start
IF YOU HAVE BETTER SUGGESTION YOU WILL BE WELCOMED, SEND IT TO ME AND I WILL REPLACE IT WITH THIS SIMPLE SCRIPT.
Installing SQUID
Before installing the SQUID you just need to download its latest STABLE version from http://www.squid-cache.org , you can find lots of useful documents at that website specially, Squid put its latest documentation in a well formatted RTF file you can get it from http://squid-docs.sourceforge.net/latest/zip-files/book-rtf.zip the version I used was squid-2.5.STABLE6 .
When you get the tarball zipped version of it open it under /usr/local/src directory, follow these steps:
1. mv squid-2.5.STABLE6.tar.gz /usr/local/src
2. cd /usr/local/src
3. tar xvfz squid-2.5.STABLE6.tar.gz
4. cd squid-2.5.STABLE6
5. Run ./configure –help to see configuration options, select options which suit your needs then use ./configure to make your squid source ready for compile and installation
6. make
7. make install
Now you have your squid installed in /usr/local/squid, but if you use another installation directory using –PREFIX=[…Directory…] option , you should find it in that directory.
OK! Installation is over, you just need some configuration to make you SQUID running.
Decide! if you need to install transparent cache server or normal one, its easy to ask you users to use proxy by setting their browser but the beauty of your job is to make your users life easier installing SQUID as a transparent proxy needs more steps but your users get about 10% better internet speed.
Then select a directory to lay out your webcache file system in that, it must be big enough to fit your cache needs, I usually use a separate partition mounted as /cache, then give its ownership to nobody:nobody.
Same access must be gained by /usr/local/squid/var/ directory and its subdirectories.
Configuring SQUID
Five magic words are used to setup your SQUID as a transparent webcache
1. http_port=3128
2. http_accel_host=virtual
3. http_accel_port=80
4. http_accel_with_proxy=on
5. http_accel_uses_host_header=on
You should do these modifications in /usr/local/squid/etc/squid.conf.
NOTE: Remember that your new installed SQUID will never give any access to HTTP requests, you need to find line access deny all and add these lines before that:
# replace you network range below
acl myNetwork src 10.0.0.0/255.0.0.0
access allow myNetwork
IMPORTANT: If you allow all ranges to have access to your proxy after a very short time you will see too much traffic on your network, there are lots of people who are looking for free open proxies to gain access to porn sites or even websites which they normally cannot access.
Find cache_dir tag and set it like this:
cache_dir ufs [cache dir you prepared] [cache size] 16 256
You can change 16 and 256, you can find more useful information about these setting in squid.conf itself, these numbers are depended to you needs and server configuration.
SQUID Configuration is finished, unless if you want to do some additional modifications on that.
Now you need to create you cache file system structure use this:
/usr/local/squid/sbin/squid -z
Wait until it finishes the job, it may take several minutes keep you eyes on your hard drive if you don’t see any special activity on your HDD maybe you system is hung!
If your installation is clean and good you shouldn’t face with a strange problem, your problem in installation might be because of three things
1. Trying to install squid before making cache filesystem, squid -z
2. Mistyping a setting in squid.conf
3. Problem with cache or log directories, missing ownership/access assignment of these directories to user nobody:nobody
If you are sure about what you did, you might need to do some googleing to find more information.
To do final check run squid from command line
/usr/local/squid/sbin/squid
Then goto one of your clients on the network set its browser to use your cache server as proxy on port 3128, on the server you can use tail command to monitor access.log
tail -f /usr/local/squid/var/logs/access.log
If you followed steps from the begining, your server should work properlyIs everything OK?
If yes, now we are about to run it during system startup, save the following shell script as /etc/rc.d/squid :
#!/bin/sh
# PROVIDE: squid
# REQUIRE: DAEMON
# KEYWORD: FreeBSD
. /etc/rc.subr
name=squid
rcvar=`set_rcvar`
command=”/usr/local/squid/sbin/${name}”
pidfile=”/usr/local/squid/var/logs/${name}.pid”
load_rc_config $name
run_rc_command “$1”
Make it executable and read only!
Then add this line in rc.conf:
squid_enable=”YES”
Okay! Have fun, restart your server and see what is happening!
Without login you client, the one you set it before, should be able to browse the net.
Make it transparent
Step one is to setup your server as a gateway, use sysinstall bring up Gateway feature during startup; see FreeBSD e-Book to find more.
Step two is to add this line in /etc/pf.conf :
rdr on bge0 proto tcp from any to any port 80 -> 127.0.0.1 port 3128
NOTE: You can substitute your functional LAN adaptor name instead of bge0 in this example
NOTE: If you have other settings in pf.conf it’s important to insert the line in its correct place.
Now, restart your server. Then go to your client and set its gateways address to your new server and remove those proxy settings from its browser. Use traceroute to make sure if it can route correctly, then browse the net. If you monitor /usr/local/squid/var/logs/access.log using tail command you can see logs rolling when you browse the net. Be sure that your browser is not using previous proxy setting.
If everything looks normal on your client, try to use all internet services to be sure about it, and then you can use your server as gateway.
This document does not cover all issues but at least you can experience installing your own webcache server. I hope this could be help for people who are looking for shortcuts.