Sebagai akselerator akses website, maka squid berada antara klien dan satu atau lebih website.

Keuntungan yang diperoleh yaitu :

• Caching

• Fleksibilitas

Kapan melakukan Cache
Berikut beberapa analisa untuk melihat kesesuaian konfigurasi squid.

1. Prosentasi Permintaan Ganda

- Prosentasi ini menunjukkan seberapa sering URL diminta dalam jangka waktu tertentu.
- ( 1 – unik URL / total URL) * 100
- 100% berarti hanya 1 URL yang diminta
- 0 % berarti tidak ada URL ganda
- Prosentasi kecil berarti tidak memberi untung pada caching squid

Untuk prosentasi yang dihitung otomatis oleh squid dapat dilihat dengan cara :

$ squidclient -p 80 mgr:info | grep Ratios

Cache information for squid:
 Request Hit Ratios: 5min: 71.7%, 60min: 49.1%
 Byte Hit Ratios: 5min: 85.9%, 60min: 54.0%
 Request Memory Hit Ratios: 5min: 2.5%, 60min: 4.9%
 Request Disk Hit Ratios: 5min: 49.1%, 60min: 51.2%
 Storage Swap size: 48621658 KB
 Storage Mem size: 65528 KB
 Mean Object Size: 12.70 KB
 Requests given to unlinkd: 0

2. Besar kebutuhan storage untuk cache

3. Rata-rata besar obyek

Studi Kasus
Proxy Hotspot ditempat saya:

$ squidclient -p 80 mgr:info
HTTP/1.0 200 OK
Server: squid/2.6.STABLE21
Date: Wed, 24 Feb 2010 10:40:41 GMT
Content-Type: text/plain
Expires: Wed, 24 Feb 2010 10:40:41 GMT
Last-Modified: Wed, 24 Feb 2010 10:40:41 GMT
X-Cache: MISS from ServerDarutTaqwa
Proxy-Connection: close

Squid Object Cache: Version 2.6.STABLE21
Start Time: Sat, 20 Feb 2010 10:26:07 GMT
Current Time: Wed, 24 Feb 2010 10:40:41 GMT
Connection information for squid:
 Number of clients accessing cache: 0
 Number of HTTP requests received: 1145760
 Number of ICP messages received: 581593
 Number of ICP messages sent: 581593
 Number of queued ICP replies: 0
 Number of HTCP messages received: 0
 Number of HTCP messages sent: 0
 Request failure ratio:  0.00
 Average HTTP requests per minute since start: 198.4
 Average ICP messages per minute since start: 201.4
 Select loop called: 17121470 times, 20.236 ms avg
Cache information for squid:
 Request Hit Ratios: 5min: 60.2%, 60min: 50.9%
 Byte Hit Ratios: 5min: 64.4%, 60min: 47.1%
 Request Memory Hit Ratios: 5min: 3.5%, 60min: 5.0%
 Request Disk Hit Ratios: 5min: 52.4%, 60min: 53.0%
 Storage Swap size: 48622284 KB
 Storage Mem size: 65532 KB
 Mean Object Size: 12.70 KB
 Requests given to unlinkd: 0
Median Service Times (seconds)  5 min    60 min:
 HTTP Requests (All):   0.01035  0.02069
 Cache Misses:          0.89858  0.85130
 Cache Hits:            0.00865  0.00767
 Near Hits:             0.00000  0.64968
 Not-Modified Replies:  0.00286  0.00379
 DNS Lookups:           0.00464  0.00464
 ICP Queries:           0.00510  0.00453
Resource usage for squid:
 UP Time: 346473.578 seconds
 CPU Time: 4897.484 seconds
 CPU Usage: 1.41%
 CPU Usage, 5 minute avg: 0.81%
 CPU Usage, 60 minute avg: 1.24%
 Process Data Segment Size via sbrk(): 0 KB
 Maximum Resident Size: 491440 KB
 Page faults with physical i/o: 1
Memory accounted for:
 Total accounted:       321602 KB
 memPoolAlloc calls: 171018880
 memPoolFree calls: 162918841
File descriptor usage for squid:
 Maximum number of file descriptors:   11072
 Largest file desc currently in use:     81
 Number of file desc currently in use:   67
 Files queued for open:                   0
 Available number of file descriptors: 11005
 Reserved number of file descriptors:   100
 Store Disk files open:                   3
 IO loop method:                     kqueue
Internal Data Structures:
 3829225 StoreEntries
  15127 StoreEntries with MemObjects
  15045 Hot Object Cache Items
 3829160 on-disk objects

Kalau dilihat Rationya maka :

Cache information for squid:
 Request Hit Ratios: 5min: 60.2%, 60min: 50.9%
 Byte Hit Ratios: 5min: 64.4%, 60min: 47.1%
 Request Memory Hit Ratios: 5min: 3.5%, 60min: 5.0%
 Request Disk Hit Ratios: 5min: 52.4%, 60min: 53.0%
 Storage Swap size: 48622284 KB
 Storage Mem size: 65532 KB
 Mean Object Size: 12.70 KB
 Requests given to unlinkd: 0

Mengapa prosentase rationya rendah ?

1. Kalau dilihat contoh kasus diatas, maka dapat dilakukan evaluasi sebagai berikut :
Website mana yang sering diakses, coba lihat di access_log? Ternyata setelah diamati banyak yang akses mail.google.com, facebook.com, meebo.com, dimana website tersebut setiap URLnya bersifat spesifik untuk setiap user yang artinya tidak akan bermanfaat banyak untuk caching squid.

2. perl -lane ‘$t += $F[4]; $u += $F[4] unless $seen{$F[6]}++;’ -e ‘END { print “unique=$u total=$t” }’ /var/log/squid/access.log
unique=31224384 total=69307469
Ratio = (1 – 31224384/69307469) * 100% = 54,95%
Artinya memang URL yang diminta hanya 54,95% saja yang ganda

3. Perhitungan dilakukan jam berapa? Yaitu jam 10 pagi. Ini berbeda dengan statistik Ratio jika dipanggil jam 8 Pagi dan jam 3 Sore, sebab aktifitas pemakai Squid juga berbeda, kalau pagi baca berita dimana ini akan baik baik caching squid.

Kesimpulan ?

1. Squid di kantor kalau hanya dimanfaatkan untuk caching saja maka dapat dikatakan belum/tidak optimal
2. Proxy internet dan cache internet dapat dipisahkan.

0 komentar

Berikut tahapan instalasi Squid di windows:

• Donwload binary file squid

File download dapat diperoleh di http://squid.acmeconsulting.it/download/dl-squid.html
File yang dipilih untuk di  download ada 3 pilihan, yaitu
- Standard ==> Untuk fungsi standar dari squid
- Delay Pools ==> Menyertakan fungsi Delay Pools
- SSL Support ==> Menyertakan fungsi SSL

Binaries
Version	Release date	Build type
2.3.STABLE5-NT - OBSOLETE	11/04/2002	Standard
		External DNS
		Delay Pools
2.5.STABLE14-NT - OBSOLETE	28/05/2006	Standard
		Delay Pools
		SSL support for reverse proxy only - EXPERIMENTAL
2.6.STABLE22	20/10/2008	Standard
		Delay Pools
		SSL support for reverse proxy only - EXPERIMENTAL
2.7.STABLE2	01/06/2008	Standard
		Delay Pools
		SSL support for reverse proxy only - EXPERIMENTAL
2.7.STABLE3	29/06/2008	Standard
		Delay Pools
		SSL support for reverse proxy only - EXPERIMENTAL
2.7.STABLE4	10/08/2008	Standard
		Delay Pools
		SSL support for reverse proxy only - EXPERIMENTAL
2.7.STABLE5	20/10/2008	Standard
		Delay Pools
		SSL support for reverse proxy only - EXPERIMENTAL

• Extrack file yang sudah di download di C:\squid
• Sesuaikan konfigurasi file di c:\squid\etc\squid.conf

File ini disalin saja dari c:\squid\etc\squid.conf.default
Untuk parameter apa saja yang disesuaikan bisa lihat disini:
- http://kusprayitna.staff.uii.ac.id/2008/08/29/setup-proxy-squid-di-centos/ (point 1 & 2)
- http://kusprayitna.staff.uii.ac.id/2008/12/11/membatasi-download-pada-squid/
- http://kusprayitna.staff.uii.ac.id/2009/01/05/filter-content-sederhana-di-squid/
- http://kusprayitna.staff.uii.ac.id/2008/12/10/squid-cache-server-sangat-lambat/

Prinsip dasar yang disetup yaitu :
(dengan cara mencari kata kunci di file squid.conf dan dihapus tanda # dan disesuaikan nilainya)

# alamat portnya
- http_port 800
# Jika dibelakang proxy, tentukan proxy induknya
- cache_peer IP_PROXY_UII       parent    800  0 no-query no-netdb-exchange no-digest
# tentukan alamat komputer client, misal 192.168.0.2 mask 255.255.255.0 & localhost
- acl mylan src 192.168.0.0/24
- http_access allow mylan
- http_access allow localhost
- http_access deny all

# Jika dibelakang proxy, tambahkan baris ini
- never_direct allow all

• Buat cache direktorynya

c:\squid\sbin\squid.exe -D -z

• Jalankan squid

c:\squid\sbin\squid.exe
apabila dibelakang proxy atau tidak ada DNS server, maka bisa dengan
c:\squid\sbin\squid.exe -D
apabila ada kesalahan atau gagal start, pesan kesalahan dituliskan di c:\squid\var\logs

• Ujicoba di browser, apakah sudah bisa

Jangan lupa seting proxy server ke alamat squid diinstall dan portnya disesuaikan juga (800)

• Tuning agar squid kerja sesuai keinginan
• Install squid agar jalan otomatis sebagai service di windows c:\squid\sbin\squid.exe -i

Usage: squid [-hirvzCDFRYX] [-d level] [-s | -l facility] [-f config-file] [-u port] [-k signal] [-n name] [-O command-line]
-d level  Write debugging to stderr also.
-f file   Use given config-file instead of c:/squid/etc/squid.conf
-h  Print help message.
-i  Installs as a Windows Service (see -n option).
-k reconfigure|rotate|shutdown|interrupt|kill|debug|check|parse . Parse configuration file, then send signal to running copy (except -k parse) and exit.
-n name   Specify Windows Service name to use for service operations default is: Squid.
-r  Removes a Windows Service (see -n option).
-s | -l facility Enable logging to syslog.
-u port   Specify ICP port number (default: 3130), disable with 0.
-v        Print version.
-z        Create swap directories
-C        Do not catch fatal signals.
-D        Disable initial DNS tests.
-F        Don’t serve any requests until store is rebuilt.
-N        No daemon mode.
-O options
Set Windows Service Command line options in Registry.
-R        Do not set REUSEADDR on port.
-S        Double-check swap during rebuild.
-X        Force full debugging.
-Y        Only return UDP_HIT or UDP_MISS_NOFETCH during fast reload.

Selamat mencoba dan sukses selalu

0 komentar

Y'day I got a chance to play with Squid and iptables. My job was simple : Setup Squid proxy as a transparent server.
Main benefit of setting transparent proxy is you do not have to setup up individual browsers to work with proxies.
My Setup:

i) System: HP dual Xeon CPU system with 8 GB RAM (good for squid).
ii) Eth0: IP:192.168.1.1
iii) Eth1: IP: 192.168.2.1 (192.168.2.0/24 network (around 150 windows XP systems))
iv) OS: Red Hat Enterprise Linux 4.0 (Following instruction should work with Debian and all other Linux distros)
Eth0 connected to internet and eth1 connected to local lan i.e. system act as router.
Server Configuration

    * Step #1 : Squid configuration so that it will act as a transparent proxy
    * Step #2 : Iptables configuration
          o a) Configure system as router
          o b) Forward all http requests to 3128 (DNAT)
    * Step #3: Run scripts and start squid service

First, Squid server installed (use up2date squid) and configured by adding following directives to file:

# vi /etc/squid/squid.conf
Modify or add following squid directives:
httpd_accel_host virtual
httpd_accel_port 80
httpd_accel_with_proxy on
httpd_accel_uses_host_header on
acl lan src 192.168.1.1 192.168.2.0/24
http_access allow localhost
http_access allow lan

Where,

    * httpd_accel_host virtual: Squid as an httpd accelerator
    * httpd_accel_port 80: 80 is port you want to act as a proxy
    * httpd_accel_with_proxy on: Squid act as both a local httpd accelerator and as a proxy.
    * httpd_accel_uses_host_header on: Header is turned on which is the hostname from the URL.
    * acl lan src 192.168.1.1 192.168.2.0/24: Access control list, only allow LAN computers to use squid
    * http_access allow localhost: Squid access to LAN and localhost ACL only
    * http_access allow lan: -- same as above --

Here is the complete listing of squid.conf for your reference (grep will remove all comments and sed will remove all empty lines, thanks to David Klein for quick hint ):

# grep -v "^#" /etc/squid/squid.conf | sed -e '/^$/d'

OR, try out sed (thanks to kotnik for small sed trick)

# cat /etc/squid/squid.conf | sed '/ *#/d; /^ *$/d'

Output:
hierarchy_stoplist cgi-bin ?
acl QUERY urlpath_regex cgi-bin \?
no_cache deny QUERY
hosts_file /etc/hosts
refresh_pattern ^ftp: 1440 20% 10080
refresh_pattern ^gopher: 1440 0% 1440
refresh_pattern . 0 20% 4320
acl all src 0.0.0.0/0.0.0.0
acl manager proto cache_object
acl localhost src 127.0.0.1/255.255.255.255
acl to_localhost dst 127.0.0.0/8
acl purge method PURGE
acl CONNECT method CONNECT
cache_mem 1024 MB
http_access allow manager localhost
http_access deny manager
http_access allow purge localhost
http_access deny purge
http_access deny !Safe_ports
http_access deny CONNECT !SSL_ports
acl lan src 192.168.1.1 192.168.2.0/24
http_access allow localhost
http_access allow lan
http_access deny all
http_reply_access allow all
icp_access allow all
visible_hostname myclient.hostname.com
httpd_accel_host virtual
httpd_accel_port 80
httpd_accel_with_proxy on
httpd_accel_uses_host_header on
coredump_dir /var/spool/squid

Iptables configuration

Next, I had added following rules to forward all http requests (coming to port 80) to the Squid server port 3128 :

iptables -t nat -A PREROUTING -i eth1 -p tcp --dport 80 -j DNAT --to 192.168.1.1:3128
iptables -t nat -A PREROUTING -i eth0 -p tcp --dport 80 -j REDIRECT --to-port 3128

Here is complete shell script. Script first configure Linux system as router and forwards all http request to port 3128 (Download the fw.proxy shell script):

#!/bin/sh

# squid server IP

SQUID_SERVER="192.168.1.1"

# Interface connected to Internet

INTERNET="eth0"

# Interface connected to LAN

LAN_IN="eth1"

# Squid port

SQUID_PORT="3128"

# DO NOT MODIFY BELOW

# Clean old firewall

iptables -F
iptables -X
iptables -t nat -F
iptables -t nat -X
iptables -t mangle -F
iptables -t mangle -X

# Load IPTABLES modules for NAT and IP conntrack support

modprobe ip_conntrack
modprobe ip_conntrack_ftp

# For win xp ftp client

#modprobe ip_nat_ftp

echo 1 > /proc/sys/net/ipv4/ip_forward

# Setting default filter policy

iptables -P INPUT DROP
iptables -P OUTPUT ACCEPT

# Unlimited access to loop back

iptables -A INPUT -i lo -j ACCEPT
iptables -A OUTPUT -o lo -j ACCEPT

# Allow UDP, DNS and Passive FTP

iptables -A INPUT -i $INTERNET -m state --state ESTABLISHED,RELATED -j ACCEPT

# set this system as a router for Rest of LAN

iptables --table nat --append POSTROUTING --out-interface $INTERNET -j MASQUERADE
iptables --append FORWARD --in-interface $LAN_IN -j ACCEPT

# unlimited access to LAN

iptables -A INPUT -i $LAN_IN -j ACCEPT
iptables -A OUTPUT -o $LAN_IN -j ACCEPT

# DNAT port 80 request comming from LAN systems to squid 3128 ($SQUID_PORT) aka transparent proxy

iptables -t nat -A PREROUTING -i $LAN_IN -p tcp --dport 80 -j DNAT --to $SQUID_SERVER:$SQUID_PORT

# if it is same system

iptables -t nat -A PREROUTING -i $INTERNET -p tcp --dport 80 -j REDIRECT --to-port $SQUID_PORT

# DROP everything and Log it

iptables -A INPUT -j LOG
iptables -A INPUT -j DROP

Save shell script. Execute script so that system will act as a router and forward the ports:

# chmod +x /etc/fw.proxy
# /etc/fw.proxy
# service iptables save
# chkconfig iptables on

Start or Restart the squid:

# /etc/init.d/squid restart
# chkconfig squid on

Desktop / Client computer configuration

Point all desktop clients to your eth1 IP address (192.168.2.1) as Router/Gateway (use DHCP to distribute this information). You do not have to setup up individual browsers to work with proxies.

How do I test my squid proxy is working correctly?

See access log file /var/log/squid/access.log:

# tail -f /var/log/squid/access.log

Above command will monitor all incoming request and log them to /var/log/squid/access_log file. Now if somebody accessing a website through browser, squid will log information.

Problems and solutions
(a) Windows XP FTP Client

All Desktop client FTP session request ended with an error:
Illegal PORT command.

I had loaded the ip_nat_ftp kernel module. Just type the following command press Enter and voila!

# modprobe ip_nat_ftp

Please note that modprobe command is already added to a shell script (above).

(b) Port 443 redirection

I had block out all connection request from our router settings except for our proxy (192.168.1.1) server. So all ports including 443 (https/ssl) request denied. You cannot redirect port 443, from debian mailing list, "Long answer: SSL is specifically designed to prevent "man in the middle" attacks, and setting up squid in such a way would be the same as such a "man in the middle" attack. You might be able to successfully achive this, but not without breaking the encryption and certification that is the point behind SSL".

Therefore, I had quickly reopen port 443 (router firewall) for all my LAN computers and problem was solved.
(c) Squid Proxy authentication in a transparent mode

You cannot use Squid authentication with a transparently intercepting proxy.

0 komentar