It’s all in a day’s work

In the past few days I’ve been helping a new colleague at my current job with getting his development area setup in Ubuntu. Most of us work with Windows, including myself since I’m not working on my laptop but on a provided desktop, so helping someone with Ubuntu is nice for a change.

But  we ran into a problem that I couldn’t explain right away, if we tried to “ssh” to our svn repository it didn’t resolve the hostname, while “normal” host lookup programs had no trouble with resolving it at all. So I posted the problem on the Ubuntu forums. Unfortunately the first response was a bit disappointing: “does it work when you try to ssh via ip?”… So I thought lets call a friend who helps me out more often in cases like these. And indeed he knew the answer. It seems that avahi is causing this problem. I’ve not yet found out why avahi is doing this, but I guess it’s a bad idea to use avahi while you need to work with a “.local” domain.

I wanted to know why avahi did this, so I thought lets join the Ubuntu IRC channel. Unfortunately I didn’t get an explanation. But I got to see something about iptables that I didn’t know about before, prerouting. Someone asked the following question:

2008-11-11 21:53:48 < deus> It is possible to use the iptables to redirect http traffic to a proxy server, for example by using iptables -t nat -A PREROUTING -p tcp -i br0 –dport 80 -j DNAT
–to-destination <ip proxy>:3128 But how does the proxy server know which site he has to lookup since the destination is changed to its own address?

I didn’t know the answer to that, guess it works via the HTTP protocol itself, but well, perhaps I’ll find this out some other time. But while I was looking into this prerouting thingy I found a nice blog post about using prerouting for flash remoting. I’ve no use for this information yet, but you never know.

Update: The maintainer of the wordpress package released “my” bugfix. It’s available now for those using unstable (sid). I guess it will be available in testing (lenny) soon, since the bug is listed in the release critical bugs list. To bad he credited me with the name Del Gurt, but well… The only thing I did was finding the missing methods in Wordpress 2.6. But it’s my first fix in Debian, yeh.

I just ran an apt-get update + apt-get upgrade and after that my Wordpress homepage was broken when I’m logged in, it gives a nice "Fatal error: Call to undefined function admin_url() in /usr/share/wordpress/wp-includes/link-template.php on line 470". And so is it for all people using the debian unstable package for Wordpress, for example this guy.

I quickly searched what functions where missing and created a patch and submitted it to the related debian bug. I hope they will accept it soon, but well. My Wordpress works fine again now.

Creating a patch took me a while, since I never created a patch for a debian package before. Lucky me, there are tutorials on how to create patches for debian packages, so I just had to read up on that. I read about how to submit the patch in the tutorial on how to help squashing Release-Critial bugs.

And I had to read up on dpatch, because the other patches on Wordpress where created using that. Lucky me, someone has created a nice short tutorial on how to use dpatch already. Dpatch seems like a good way to keep your patches separated from the original code, so when the original code is updated, you can determine easily if that conflicts with your patches or not.

And then I had to create a GPG key to sign my patch. Since dpkg-buildpackage complains about it otherwise.

So in the end, what I had to do to patch my Wordpress and submit the patch (besides generating the GPG key):

Install the required tools

sudo apt-get install build-essential devscripts lintian linda diff patch patchutils dpatch

Get the Wordpress package source

apt-get source wordpress

Enter the root directory of the extracted source

cd wordpress-2.5.1

Let the package know I’m making the changes

dch -i

Start the dpatch process

dpatch-edit-patch 009CVE2008-3747.addendum

Make the changes

Exit dpatch process

exit

The wordpress patches are all called .patch so rename it

mv debian/patches/009CVE2008-3747.addendum.dpatch debian/patches/009CVE2008-3747.addendum.patch

Change the e-mail address in the patch

EDIT debian/patches/009CVE2008-3747.addendum.patch

Add the patch to the list of patches

echo "009CVE2008-3747.addendum.patch" >> debian/patches/00list

Test the patch

fakeroot debian/rules binary && sudo dpkg -i ../wordpress_2.5.1-6.1_all.deb

Build the source package

dpkg-buildpackage -kMYKEY -rfakeroot

Create the patch

interdiff -z ../wordpress_2.5.1-6.diff.gz ../wordpress_2.5.1-6.1.diff.gz > ../bug_497524.patch

Email the bugs list with the patch

Email bug control to inform them that a patch has been made for the bug

That was it. Not that hard, if you know the commands + procedure. Now wait and see what the package maintainer is going to do with the patch I created.

Update: seems I forgot to check the edit link itself after the fatal error was gone… It didn’t work anymore. Fixed that as well now.

Update2: for those who find my post and do not want to wait on the maintainer to release the patch, I’ve “released” it. The MD5 of wordpress_2.5.1-6.1_all.deb is: dc46d35743122d933def7db431c8f49e.

Seems my previous blog post about the Belkin is found quite often when people search for how to get it to work on Ubuntu in general, also with ndiswrapper. Please note, my post is not about getting it to work with ndiswrapper (that worked fine in my case, but I’m on 32-bit Ubuntu, and didn’t try it on 64-bit). There are quite some posts about getting it to work with ndiswrapper (also posts about it not working, will drop him a comment). This one also explains how to unpack the windows driver if you don’t have a windows installation.

Anyway, so far about using ndiswrapper. Recently ralink released a new version of it’s native linux driver, 1.3.1.0 which can be found on their linux support page, and with this one I got the Belkin F5D8053 working. It doesn’t see the 5Ghz network I’ve over here, but well I don’t want to do normal Wireless over that network anyways, since that will decrease the speed of the Wireless to Wireless bridge I setup. I’m still blaming incompatibility between the Belkin USB stick and the Apple Airport routers for the Belkin not being able to find the 5Ghz network.

What I had to do to get it working.

1. make sure you can build kernel modules: sudo apt-get install build-essential linux-headers-generic
2. check if your Belkin USB id is listed in the driver: lsusb | grep Belkin | awk '{print $6}';, which returns 050d:815c in my case. Then use the id found in grep -i '0x050d,0x815c' include/rt2870.h if that returns a result, then skip step 3 since your device is already supported by the driver (please note the '0x' added before the 2 parts and the ',' in stead of the ':').
3. If it’s not there add it: edit include/rt2870.h Search for Belkin; Copy/Paste that line and change the usb id of your device to that you found in step 2.
4. Edit os/linux/config.mk and change the HAS_WPA_SUPPLICANT=n HAS_NATIVE_WPA_SUPPLICANT_SUPPORT=n to =y.
5. Edit Makefile and remove any line containing /tftpboot (still don’t know why they are in there, and running make as normal user fails if they are in there)
6. make && sudo make install
7. Edit (as root) /etc/modules and add rt2870sta to it (remove ndiswrapper if you used that before to get it working).
8. Either reboot or do modprobe rt2870sta to get it working (if you used ndiswrapper before you first need to run modprobe -r ndiswrapper). You will also need a /etc/init.d/networking restart if you won’t reboot.

That was all I needed to do.

Update: I got it working now, see my new post about it.

If I look at the Windows driver for my new Belkin F5D8053 it is based on the RT2870 chipset. There is a native driver available for this chipset. Unfortunately so far I’m not able to get it to work. This version is already linux 2.6.24 ready so I don’t need to apply these patches. But even though I’m able to compile it, I’m not able to use it, yet. Side note: the warning in the Ubuntu forums about backing up any existing configurations in /etc/Wireless/RT2870STA is useful, since you need to put your configuration in there and it’s removed by the make install.

One strange thing in the make process is that it wants to copy the result to /tftpboot. I’m not sure why it wants to do that, and I’m almost certain it shouldn’t be a part of the make process…

The first problem I ran into was the fact that the USB Device id for my Belkin, 050D:815C, is missing in the driver (that could be a big hint about it not being able to work for it …) so I added that to the include/rt2870.h (I copied the line for the other Belkin device and changed the device id into mine).

With that added, my Belkin is detected by the driver as ra0. But even when I compile it with:

HAS_WPA_SUPPLICANT=y
HAS_NATIVE_WPA_SUPPLICANT_SUPPORT=y


I’m not getting my device visible in the gnome network manager, so I tried the command line configuration options.

First I tried only configuring the /etc/Wireless/RT2870STA/RT2870STA.dat. But if I had only that file configured, the activity light was turned off on an ifup ra0. So I guess that’s not right.

Then I tried to configure it in the /etc/network/interfaces as described on the Ubuntu forums. The light staid on when I did ifup ra0 but no DHCP lease. So finally I tried the “iwpriv” way described here. But still no DHCP lease.

So I guess I’ll have to contact ralink, to see if they are able to help me out here.

With high traffic sites, performance issue’s rise. Now one of the solutions is to use an opcode caching mechanism for PHP. There are two mainstream open source solutions for that, APC and eAccelerator. And there is Zend platform and Zend core.

Installing eAccelerator
I found a very straight forward howto about installing eAccelerator (and others) on Debian.

But, as always, there are some problems with that tutorial. First the -- is replaced by a special character on that tutorial (and in my blog if I don’t use the <code> tag) in the configure command, so that doesn’t work if you copy paste it. The working configure command is:


./configure --enable-eaccelerator=shared --with-php-config=/usr/bin/php-config


And you don’t need to do all the work as root, only the apt-get, the make install and the php.ini editing should be done as root (best practice, only use root privileges when needed).

Also the double quotes are changed into a special character in that tutorial, so if you get an error in your logs about unable to load \xe2\x80\x9deaccelerator.so\xe2\x80\x9d you copied them just like I did

This is what I put in the php.ini, without the special double quotes, and with one change


extension="eaccelerator.so"
eaccelerator.shm_size="16"
eaccelerator.cache_dir="/var/cache/eaccelerator"
eaccelerator.enable="1"
eaccelerator.optimizer="1"
eaccelerator.check_mtime="1"
eaccelerator.debug="0"
eaccelerator.filter=""
eaccelerator.shm_max="0"
eaccelerator.shm_ttl="0"
eaccelerator.shm_prune_period="0"
eaccelerator.shm_only="0"
eaccelerator.compress="1"
eaccelerator.compress_level="9"


The tutorial writer creates the cache directory in /tmp. But he fails to make the remark that /tmp is cleared on server restart. So that’s not really a good option. So I’ve done the following in stead


sudo mkdir /var/cache/eaccelerator
sudo chown www-data /var/cache/eaccelerator
sudo chmod 0755 /var/cache/eaccelerator


Yes, the 0644 from the eAccelerator wiki is what you want for the files being created in there, but not for the directory itself.

The big disadvantage of this way of installing it, is that you probably need to recompile it (after a make clean) and reinstall it on every update of PHP. But reading up on APC vs eAccelerator made me choose for the latter.

But does it really help?
Now, the results. What better to test then my own PHP blog (and yes it’s not the fastest one, it’s running on a VPS).

Before installing eAccelerator:

Server Software:        Apache/2.2.8
Server Hostname:        blog.delgurth.com
Server Port:            80

Document Path:          /
Document Length:        44436 bytes

Concurrency Level:      10
Time taken for tests:   397.121299 seconds
Complete requests:      1000
Failed requests:        0
Write errors:           0
Total transferred:      44796000 bytes
HTML transferred:       44436000 bytes
Requests per second:    2.52 [#/sec] (mean)
Time per request:       3971.213 [ms] (mean)
Time per request:       397.121 [ms] (mean, across all concurrent requests)
Transfer rate:          110.16 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   0.3      0       6
Processing:   712 3961 1674.7   3696   10696
Waiting:      368 2177 1013.3   2033    6943
Total:        712 3961 1674.7   3696   10696

Percentage of the requests served within a certain time (ms)
  50%   3696
  66%   4454
  75%   4977
  80%   5270
  90%   6225
  95%   7039
  98%   8188
  99%   8632
 100%  10696 (longest request)

After installing eAccelerator

Server Software:        Apache/2.2.8
Server Hostname:        blog.delgurth.com
Server Port:            80

Document Path:          /
Document Length:        44436 bytes

Concurrency Level:      10
Time taken for tests:   242.584585 seconds
Complete requests:      1000
Failed requests:        0
Write errors:           0
Total transferred:      44796000 bytes
HTML transferred:       44436000 bytes
Requests per second:    4.12 [#/sec] (mean)
Time per request:       2425.846 [ms] (mean)
Time per request:       242.585 [ms] (mean, across all concurrent requests)
Transfer rate:          180.33 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   0.2      0       4
Processing:   430 2419 983.2   2330    6409
Waiting:      100  648 448.6    578    3025
Total:        430 2419 983.2   2330    6409

Percentage of the requests served within a certain time (ms)
  50%   2330
  66%   2798
  75%   3051
  80%   3246
  90%   3712
  95%   4182
  98%   4614
  99%   4901
 100%   6409 (longest request)

So it helps also quite a bit on my wordpress install, it’s about a 33% performance increase!

Some material I read

• Some benchmarks of opcode caches (in german)
• Some performance tips with some more reading material
• A benchmark on APC vs eAccelerator on Drupal
• A benchmark on APC vs eAccelerator vs XCache on Drupal
• Another benchmark and an explanation about the differences between APC and eAccelerator

P.s. yet another wordpress rant, if you have empty lines between <code> tags, wordpress inserts <p> tags there for you. Not very nice to my liking, since it breaks the formatting of the <code> tag, which I used to format the apache bench results…

One disadvantage of developing websites/web applications on Unix (in my case Ubuntu) is that it’s hard to test things on Internet Explorer. Yes, you have IEs4Linux but that’s just not the same thing.

For real testing with the different versions of Internet Explorer Microsoft makes images available so you can test your websites/web applications in Microsoft Virtual PC. Unfortunately it’s not easy to use them when you develop on Unix.

So I wanted to see if I could convert these images into VMware images, since VMware player can run on Ubuntu. And it seems this is possible. Even though I’m not entirely sure I don’t break any EULA’s with this… To get it all working you need a Windows machine (with administrator rights) on which you need to unpack and convert the images. This conversion is not needed when you use VMware workstation. This conversion needs to be done every 6 months or so, since the windows version on the VPD has an expiry date.

Installing VMware player
Download VMware player tar.gz unpack it and run the installer as root.

When you are installing it you might get the following error (if you have a newer kernel):

include/asm/bitops_32.h:9:2: error: #error only <linux/bitops.h> can be included directly


But long live people who blog about their experiences, here is the solution for this problem. The writer forgot to put sudo before some of the commands and he assumes you came on his site after it failed and you didn’t try to remove the partially installed VMware player. Here are the steps with all the sudo’s:

1. cd /usr/lib/vmware/modules/source
2. sudo cp vmmon.tar vmmon.tar.orig (if you want to be able to undo your changes)
3. sudo tar xvf vmmon.tar
4. sudo vi vmmon-only/include/vcpuset.h
5. change line 74 from: #include "asm/bitops.h" to: #include "linux/bitops.h"
6. sudo rm vmmon.tar
7. sudo tar cvf vmmon.tar vmmon-only/
8. sudo rm -rf vmmon-only/
9. sudo vmware-config.pl

Then you have a working VMware player.

Getting the Virtual PC Images
Microsoft makes VPC images available for IE6 and IE7, and now also the IE8 beta 1. Just download them and unpack them on a windows machine.

Converting the Virtual PC images
Get VMware converter to convert the free Virtual PC images to VMware images. Before you can convert them you first need to make a VPC image for the VHD. This can be done by running Virtual PC 2007. After that you can put the image in VMware converter and convert it into a VMware virtual machine. This is the part where I’m not sure if it’s legal or not. When converting the image, don’t forget to set the checkbox for the VMware tools.

Using the Virtual PC images
Because we use VMware player, we don’t have the VMware tools package. Lucky for us you can download the VMware workstation evaluation copy and unpack it. Then you need to copy the iso images in there to make sure that the VMware tools can get installed (you might get a message about inserting the windows xp sp2 cdrom, but if you just cancel those, the virtual machine still works good enough to test IE). You might wonder if this is legal, but according to this thread it should be ok. The following command I used to copy them, but you need to be in the directory below the one where you unpacked the workstation evaluation copy.

sudo cp -r vmware-distrib/lib/isoimages /usr/lib/vmware/

After that you are ready to use the Virtual PC images with VMware tools.

Other nice things to do

Using your existing windows partition
Other thing you can do is use your dual boot windows partition. For now no blog post about that, since I didn’t try that out myself. But here are some links about it:

• Using windows apps in ubuntu
• Ubuntu forums post about it
• VMWare blog post about it
• Another post about it
• And yet another post about it

When you use the same windows install from disk and from VMware, you will run into problems with the Windows hardware based activation. But also for that problem a a solution exists and is blogged about. It’s pretty simple, if you know it. It seems that windows activation creates two files that contain the activation information. If you just keep both versions of the activation information available and copy them into the right place at the right time, all should work well

Running things outside the VMware scope
If you want the Internet Explorer in a window outside your virtual machine, you can do that using rdesktop. Some information about that can be found on the following sites.

• Mohammad Azimi blogs about it
• Mohammad Azimi blogs about some more
• and yet another blog

A side note
Unfortunately, while writing this post I encountered one problem with referring to other articles which contain the HOWTO for your problem. When you google for Run Existing Windows Installation on Ubuntu with Vmware Player you will find a lot of referrals to one specific article, which is no longer available. This is the main reason why I copy/pasted the steps to install VMWare player, in stead of only linking to them.

For a while I was not able to print from Windows XP via Samba with my Brother HL 2030 laser printer which is connected to one of my servers. I’m running debian Etch there and had upgraded some packages, including samba. The printer was listed in my SWAT samba status, but on windows I couldn’t find it.

In the end it seems that I was missing one statement in my smb.conf:

[printers]
comment = All Printers
path = /var/spool/samba
create mask = 0700
guest ok = Yes
printable = Yes
read only = No


The odd thing is, that when I look at the documentation, setting printable = Yes should have been enough. At least, that’s how I read it. But I guess I’m wrong since the read only = No helps (although you can’t set this parameter for a printer share using SWAT).

I wanted Skype on my laptop, stupid me thought it was nice to have 64 bit linux. Many applications don’t seem to be 64 bit ready, one of them is Skype. Good thing there is a –force-architecture flag in dpkg, but still not all applications work that way. Skype is one of them. But I found a good script called getlibs that helps installing the required libraries, once more on the ubuntu forums.

But even though these type of scripts can be found, I’m still going to switch my Desktop back to 32 bits. The support for 64 bits is not good enough, yet. And well, since I’m not using > 4 Gb’s of memory, I don’t really need 64 bits.

Some time ago I had some problems sending mail to one of our clients, something with their primary mailserver not accepting connections and a non-existent secondary mailserver. So I wanted to enable authenticated mail sending on my own mailserver.

It was quite easy to get it working, after finding some articles which I could use. The biggest problem was the jail Postfix was in so it couldn’t talk with saslauthd direcly.

I added the following in my main.cf

#SMTP Auth
smtpd_sasl_auth_enable = yes
smtpd_sasl_security_options = noanonymous
smtpd_sasl_authenticated_header = yes
smtpd_sasl_path = smtpd
smtpd_sasl_local_domain =
broken_sasl_auth_clients = yes


smtpd_recipient_restrictions = permit_mynetworks, permit_sasl_authenticated, reject_unauth_destination, reject_unknown_recipient_domain

And I created directory sasl and placed a smtpd.conf in there containing:

pwcheck_method: saslauthd
mech_list: plain login


And I changed /etc/default/saslauthd to use the suggested option (suggested in comment in the file itself):

OPTIONS="-c -m /var/spool/postfix/var/run/saslauthd"

Used articles:

• The first article I found on the subject
• A mailinglist post describing a problem I encountered (I didn’t use the suggested solution)
• A good article on howtoforge describing the final solution I used

Update 2008-05-15:
Today I ran into a problem after I did a package upgrade. I got the warning:

SASL authentication failure: cannot connect to saslauthd server: Permission denied

Seems some permissions where incorrect. I did a:

sudo chgrp sasl /var/spool/postfix/var/run/saslauthd
sudo passwd postfix sasl
sudo /etc/init.d/postfix restart


and that solved it. Why I didn’t get this before I don’t know, but well, this fixed it. Thanks to Jimmy who blogged about Debian and SASL as well.

A colleague of mine uses the same laptop as I’ve. But he didn’t like the fact that the build-in display is not used when it’s docked in the docking station. He wants to use his laptop LCD as secondary display when he’s at work.

Unfortunately I’m still on Feisty, but once more Prevu came to the rescue. I found a post about the order in which to build the packages to get xrandr 1.2 (the required version to get the thing to work properly).

And then I used this ubuntu forum post to configure my xorg.conf.

I added:

Viewport 1680 0
Virtual 3360 1050


to the “Screen” Section of my xorg.conf (SubSection “Display”), restarted X and could use the following command to get my display to be a second display and not just a clone:

xrandr --output TMDS-1 --right-of LVDS --mode 1680x1050

To switch back to the clone I use:

xrandr --output TMDS-1 --same-as LVDS --mode 1680x1050

Or you can just restart X again.

This was all.

P.s. If anyone knows how you can dock the D830 on Ubuntu while running, please drop a comment. It kinda hangs when I try to do that and the only thing left for me to do is a hard reset.