La Vita è Bella

We configured some Squid 2.6 servers that use external_acl_type to use some headers sent by client for access control. And the authentication isn't username/password routine, but use some tag to calculate hash. When running, the memory used by Squid just keep increasing from time to time, just like it have a memory leak. We tried to disable the acl on some server, and these servers runs just fine.

As external acl run in individual process, even if the acl program have memory leak, the memory used by the squid process shouldn't be growing.

We tried many ways to figure out the problem, but all fails. Finally someone noticed that in the external_acl_type documentation, there's a parameter named "cache", with this description:

result cache size, 0 is unbounded (default)

"unbounded"! So this is the problem. For username/password routine, cache is useful. the next time some user with the same username/password comes, Squid can get the result from cache without communicate with acl program. But for our authentication method, as the headers used to calculate hash is differ from every request, cache is totally useless.

I really hope that "0" means no cache and "-1" means unbounded. But anyway, set "cache" to 1 can do the trick. Now the Squids don't have memory problems anymore, although cache replace will slow them down a bit.

In my last blog, I use ndiswrapper for wireless driver and it can only use 802.11g, but not 802.11n. But today I've got the solution!

According to this article on EeeUser forum, the rt2860 chipset released the source code for Linux driver! Download them from the official website, and build it. You will need kernel header package to build the driver.

After successfully build, use "modprobe rt2860sta" to install this module, and you may also add the line "rt2860sta" to your "/etc/mmodules" file to load it automatically every time (but seems that modprobe have done this, you may don't need this step).

Now here's a problem: seems that wpa_supplicant didn't support this driver. So you need to set wireless parameters by iwpriv manually. And the biggest problem is WPAPSK, you can't just input your passphrase to do it. Luckily there's a webpage that can calculate the WPAPSK for us. I'm using WPA2 and it works. I'm not sure about WPA. But WPA is broken! Why don't you move to WPA2?

Save the below script to "/etc/restart_wireless.sh" and give it execute privilege:

iface=ra0
w="iwpriv $iface"

total_start=$(date +%s)

init_start=$(date +%s)
echo -en "iwpriv config..."
$w set NetworkType=Infra
$w set AuthMode=WPA2PSK
$w set EncrypType=AES
$w set SSID=Your SSID
# get WPAPSK from http://www.wireshark.org/tools/wpa-psk.html
$w set WPAPSK=Your WPAPSK
echo "done"
init_end=$(date +%s)
init_time=$(($init_end-$init_start))

assoc_start=$(date +%s)
assoc=0
assoc_report=10
assoc_loop=310
echo -n "Associating..."
for ((i=0; $i < $assoc_loop; i++)); do
        if [ "$(iwconfig ra0 2>/dev/null | head -1 | cut -f2 -d: | cut -f1 -d" ")" == "\"\"" ]; then
                if [[ $(( ($i+1) % $assoc_report )) == 0 ]]; then
                        echo -n .
                fi
        else
                echo done
                assoc=1
                break
        fi
        sleep 1
done

if [[ $assoc != 1 ]]; then
        echo failed
        exit 1
fi
assoc_end=$(date +%s)
assoc_time=$(($assoc_end-$assoc_start))

total_end=$(date +%s)
total_time=$(($total_end-$total_start))

echo -e "Time spent (sesconds)\n\tinit: $init_time\n\tassociation: $assoc_time\n\tTotal: $total_time"

Credit to a1l0a2k9, the above script is also from EeeUser forum, but I removed the DHCP part and modprobe part. If you are using DHCP, then you may need the DHCP part and modprobe part from the original script.

Now the "/etc/network/interfaces" part, add the following lines for the ra0 interface:

iface ra0 inet static
up /etc/restart_wireless.sh
address 10.0.14.14
netmask 255.255.255.0
gateway 10.0.14.1
auto ra0

(for DHCP users: change "static" to "manual" and remove the "address", "netmask" and "gateway" lines.)

And now, "ifup ra0", then you're done!

UPDATE: now we have 802.11n!

My old home server is dying these days, so I bought a new Asus Eee Box B202 to replace it. It uses Intel Atom N270 CPU, 1G memory, 80G harddisk, 10/100/1000 Ethernet and 802.11n wireless.

The first thing I do on it is install Debian Lenny, my favourite system for server.

Preparing USB flash for net install

As it didn't come with a cd-rom, I choose USB flash. I use the SD card from my camera and a USB card reader to combine a USB flash, and it can be used to boot successfully.

I prepared the USB flash according to the Debian Lenny documentation, but meet some problems:

1. Google for hd-media returned the hd-media link for Sarge as the first result, I used the boot.img.gz from Sarge and lenny-businesscard iso but the iso can't be found by the installer, so the installer (boot.img.gz) and the iso must match.
2. The hd-media from Lenny and the Lenny beta2 iso didn't match, neither. The installer can recognize the iso, but complain about mismatched kernel version, and prompt you that it need network update. But the installer from boot.img.gz didn't come with ethernet driver, so it will fail and can't continue.

So finally I have to use "the flexible way" and net install. I use the initrd.gz that have ethernet driver, and the vmlinuz from Lenny hd-media. None of the iso is needed (and you can't use them), all packages will be downloaded from one of the Debian mirrors.

After prepared the USB flash, DON'T FORGET to lock the write protection lock before boot. It will save your life later.

Install Debian

Boot from the prepared USB flash, and it contains only GRUB CLI, so you need to boot the installer manually:

root (hd0,0)
kernel /vmlinuz
initrd /initrd.gz
boot

Now you have a Debian Installer that can drive your ethernet card, so you're ready to install.

Install steps are normal, nothing more to say until the grub-install step.

grub-install failure

On the grub-install step, it will complain that grub-install (hd0) failed. Why? cause (hd0) is your USB flash and (hd1) is your harddisk! That's why lock the write-protection lock is important, or otherwise it may succeed without write your harddisk mbr. That's really stupid. Manually install grub on (hd1) and it will continue.

And the installation completes.

The X problem

After installation, you may find that your X didn't work. This is because it uses a Intel Graphic chipset for lap-top, but it's not lap-top. Ubuntu wiki have the solution, and it works.

Wireless driver

UPDATE: follow my next blog article for official driver and 802.11n!

After installation, the first important thing to do is to drive the wireless card. It uses AzureWare card which uses a rt2860 chipset. And luckily, ndiswrapper can do it.

Follow the instructions on Debian wiki to install ndiswrapper. The Windows driver is on the CD (you have another computer to read the CD and copy the driver to the Eee Box, do you?), I used the WIN2KXP one. After install ndiswrapper, it works.

But there are some problem in the /etc/network/interfaces, if you use WPA (I didn't get thie problem when using WEP):

auto wlan0
iface wlan0 inet static
address 10.0.14.14
netmask 255.255.255.0
gateway 10.0.14.1
wpa-conf /etc/wpa_supplicant.conf

The configure above can be used to connect router (that I can see it from router admin), but the IP 10.0.14.14 can't be reached by another computer in the subnet. But if you execute an extra:

# ifconfig wlan0 10.0.14.14

Then it will be OK. I don't know why but if I move the "auto wlan0" line after the "wpa-conf" line, it works fine. Maybe it must wait wpa-supplicant to do something first?

Another problem is that it can only use 802.11g wireless, if anyone knows how to drive it to 802.11n, please tell me

The end and photos

Finally, I have a new home server now.

I have a Linux file server in my home running Debian Lenny, and I always use SMB for file sharing, it have a very very bad performance. Today I suddenly remember Apple have an AFP protocol, so gave it a try.

I use "apt-cache search afp" to find out that there's a package named "netatalk" can provide AFP file sharing, so install it. But I can only login use guest account, not my system user, from Leopard.

I googled it and found that the problem is: on the Debian side, as a license issue, the Debian package didn't come with SSL support; on the Leopard side, it didn't allow you exchange your password with AFP server without SSL. So the solution is build netatalk yourself, with SSL.

The building steps are described on this blog, and I also disabled atalkd as the author suggested, it caused netatalk to start-up much faster than before.

So finally I got a AFP server for my Mac (compare this icon to the famous BSOD icon for SMB servers in Leopard ):

And as expected, AFP is much much faster than SMB, here's the write test:

For AFP:

fishy@McManaman:~$ dd if=/dev/zero of=/Volumes/Home\ Directory/foo
^C57345+0 records in
57345+0 records out
29360640 bytes (29 MB) copied, 11.0833 s, 2.6 MB/s

And for SMB:

fishy@McManaman:~$ dd if=/dev/zero of=/Volumes/fishy/bar
^C4235+0 records in
4235+0 records out
2168320 bytes (2.2 MB) copied, 10.6889 s, 203 kB/s

I'm impressed!

We use diff to find out the different lines in 2 files, but sometimes we also need to find out the same lines in 2 files. So we need the "reversed diff"

And this command can be used as the reversed diff:

cat file1 file2 | sort | uniq -d

If you toke some photos by your camera, and want to post them to somewhere (for example, I want to post the photo of my Treo 650 because I'm going to sell it), you may need to batch resize your photos.

This bash script shows how to uses ImageMagick to batch resize your photos:

1 #!/bin/sh
2
3 for file in *.JPG; do
4         convert -resize 1024x768 $file ${file%.JPG}_resize.jpg
5 done

wget -Y -e "http_proxy=host:port" url

"How to set proxy for wget?" I've been asked this question for many times, but it seems didn't appears in the "-h" output nor man page, so I always forgot it.

That's why I'm making a note here

Under my Debian Linux, when I use screen, I always get some keymap/TERM setting problems, for example, the backspace key never work.

But if I set TERM to "vt100" before launch screen, it's OK, so this alias can resolve the problem:

alias screen='env TERM=vt100 screen'

For the poor network, I always need a multi-thread http downloader. Firefox has a extension DownThemAll!, but after I finish surfing websites (but didn't finish downloading yet), I used to press Command+Q to quit Firefox, and the download was interrupted. lftp has a built-in downloader: pget, so use pget is a good idea:

alias pget="lftp -c pget"

Put the codes into your bashrc file, and restart your terminal, it's done.

I have used my old lap-top as a home server. Cause I'm using ADSL at home, I have to use a dynamic DNS.

I choosed a Chinese dynamic DNS provider "Peanut Shell", they've provided a Linux client to commit your IP.

But the Linux client they provided is close-source and often crashes. After it crashed, my domain name can't be kept up-to-date.

So I have to write a script to make sure it's still running:

1 #!/bin/sh
2 pid=`pidof phlinux`
3 if [ -z "$pid" ]; then
4         echo "phlinux not running, start it as daemon now..." >&2
5         /usr/local/phlinux/phlinux -d
6 else
7         echo "phlinux is running"
8 fi

Save this script in some place, and make a link to your "/etc/cron.hourly", so it will be checked every hour.

UPDATED: or maybe you don't want to get mail report on every check, I've updated the script, and you should write a script in your "/etc/cron.hourly" instead of link the script, and writes "/path/to/your/script > /dev/null"

I wrote this script to convert mp3 files into ogg files. ogg is a open source audio format that is smaller and seems better than mp3.

Another reason I wrote it is that the audio player for PalmOS, AeroPlayer, is free with ogg plug-in. But the mp3 plugin is not free.

You'll need the following softwares to use this script: mplayer, oggenc, id3v2, iconv. If you are using Debian or Ubuntu, you can apt-get them. iconv is used to conver id3 tags that in GB18030 into UTF-8, if you don't need such feature, you can ignore it, and comment line 9.

It will convert all files under current directory into ogg, and try to keep the tag info. It will try id3v2 first, then id3v1.

It first use mplayer to dump mp3 file into wav file, then use oggenc to convert wav file into ogg file.

It can be ran with a parameter, to specify the ogg quality. If omitted, it's 2.

It's based on the script by stoneboy @ newsmth forum. My work is try to keep tag info.

Enjoy it

more...