Category Archives: Development

Lightweight & fast webmail client

Leave a reply

My main home server is a measly pentium II class machine running (now) on Debian Etch. Not the fastest machine but it works, uses little power and is cheap while setup from left-overs. Among other things it serves me my email when not at home using RoundCube Webmail. With the low-end hardware I am using it is key to keep it as fast and lightweight as possible. The server is behind a DSL connection at home so compression comes in handy as well.
Continue reading →

Ubuntu .local domain resolving issue’s and iptables prerouting

3 Replies

In the past few days I’ve been helping a new colleague at my current job with getting his development area setup in Ubuntu. Most of us work with Windows, including myself since I’m not working on my laptop but on a provided desktop, so helping someone with Ubuntu is nice for a change.

But  we ran into a problem that I couldn’t explain right away, if we tried to “ssh” to our svn repository it didn’t resolve the hostname, while “normal” host lookup programs had no trouble with resolving it at all. So I posted the problem on the Ubuntu forums. Unfortunately the first response was a bit disappointing: “does it work when you try to ssh via ip?”… So I thought lets call a friend who helps me out more often in cases like these. And indeed he knew the answer. It seems that avahi is causing this problem. I’ve not yet found out why avahi is doing this, but I guess it’s a bad idea to use avahi while you need to work with a “.local” domain.

I wanted to know why avahi did this, so I thought lets join the Ubuntu IRC channel. Unfortunately I didn’t get an explanation. But I got to see something about iptables that I didn’t know about before, prerouting. Someone asked the following question:

2008-11-11 21:53:48 < deus> It is possible to use the iptables to redirect http traffic to a proxy server, for example by using iptables -t nat -A PREROUTING -p tcp -i br0 –dport 80 -j DNAT
–to-destination <ip proxy>:3128 But how does the proxy server know which site he has to lookup since the destination is changed to its own address?

I didn’t know the answer to that, guess it works via the HTTP protocol itself, but well, perhaps I’ll find this out some other time. But while I was looking into this prerouting thingy I found a nice blog post about using prerouting for flash remoting. I’ve no use for this information yet, but you never know.

Eclipse Subversive SVN+SSH and Putty Agent (pageant)

9 Replies

I’m using Subversive within Eclipse to provide SVN support. But since I’m now using SVN+SSH as connection mechanism to SVN, I wanted to use my SSH Key to do the authentication for me. It took me some time to find out (find with google ;) how I could get this to work. I ended up on another blog which contained a working explanation on how to get it to work.

You need to create an environment variable called “SVN_SSH” that points to an executable file that accepts the same command line arguments as ssh on unix. I did this by doing the following:-

  1. Set up ssh keys. Not going to cover that here as you can easily Google for that. You need to end up with your public key on the SVN server and your private key loaded into Paegent locally.
  2. Download and installed the excellent TortoiseSVN client for Windows.
  3. Set the following environment variable (by right-clicking on My Computer, Properties, Advanced, Environment Variables, New):-

    Variable name:
    SVN_SSH
    Variable value:
    C:\\Program Files\\TortoiseSVN\\bin\\TortoisePlink.exe

    (The “\\” is very important, otherwise it won’t work. Equally, you cannot use the plink.exe that comes with putty as that fires up a command shell window which is really annoying. The TortoisePlink.exe is a windows implementation of plink that doesn’t bring up any UI)
  4. Configure the Subclipse plugin to use JavaHL (JNI)
  5. Restart Eclipse
  6. Do a little victory jig (optional)

Yes the \\ is really needed, don’t (yet) understand why, but it works.

If you dislike Tortoise, then you can set the SVN_SSH variable to c:\\putty\\plink.exe -ssh -2 -A -l username

But as Martin said, plink has one disadvantage. It creates a msdos popup window each time it’s used…

JavaHL is also needed, SVNKit doesn’t use the SVN_SSH variable.

Talking about Tortoise… A while ago I encountered some annoying lock problems (maven clean install didn’t work because maven was not allowed to delete the target directory). This was caused by the Tortoise cache. Now I disabled it in total, which helped against my locking issue, and the performance of my explorer increased as well. So it’s a win/win situation. And since I like to develop using Ubuntu, I’m not used to see the svn status in my explorer window right away anyways (although I’ve to say, it can be useful, if it didn’t cause problems I would have kept it).

Perhaps this locking problem is related to having my workspace on an ext2 fs partition on my usb drive… Not sure who to blame, yet. On my new workstation I’ve not had such locking problems yet, with this tip the cache is performing good enough.

Debugging mails being send from applications

Leave a reply

Ever needed to test the e-mail functionality of your application, on windows, without having access to a SMTP server? Try SMTP server for developers, a brilliant piece of software, so far.

You need .Net for it. It’s really easy to setup and use. Another advantage of this SMTP server is that when you test your application, you don’t have to worry about test e-mails being send to customers. This because the SMTP server doesn’t really send the messages, but just places all of them on disk.

Update: one disadvantage. You can’t get it to run on any other port then 25, which is inconvenient if McAfee is blocking port 25. I contacted the author a while ago, but no response so far. Guess I’ll have to add this functionality myself, but then I need to get up to speed on .Net first. Ah well, I’ll wait a little longer.

Talking about McAfee… It’s kinda weird, it blocks e-mails on port 25 if I use for example Java to send the e-mail. But if I use telnet to connect to port 25, all is fine?

Keyboard shortcuts

Leave a reply

Perhaps you have noted that I added a new site to the list of sites I like: Windows Keyboard Shortcut of the Day. Using keyboard shortcuts just improves your work speed a lot.

A few windows shortcuts that I didn’t know before

Windows + BREAK
Go to system properties
CTRL + SHIFT + ESC
Taskmanager
Windows + R
Run command

Eclipse

I also found a nice blog entry about eclipse shortcuts. But he lacks to explain one of the uses for CTRL + T: if you have a certain function you can find which class implements that function with CTRL + T as well. Very useful when you are working a lot with interfaces.

This is also a good site with Eclipse shortcuts. Not sure if it’s a bad thing that it’s not updated to Eclipse 3.4, yet. One helpful command is missing on that list: CTRL + SHIFT + L. This lists all bound keyboard shortcuts and pressing it again gives you the opportunity to change and export them to .csv (at least the export function is there on Eclipse 3.4).

Mac OS X

And a while ago I found a nice site with lots of Mac OS X keyboard shortcuts. The shortcut to make your mac sleep quickly was provided by me :-) There is also a shortcut to restart your mac quickly without confirm, Cmd + Ctrl + Eject. Quite annoying if you use that one when you want to put your mac to sleep.

PHP Acceleration with eAccelerator

Leave a reply

With high traffic sites, performance issue’s rise. Now one of the solutions is to use an opcode caching mechanism for PHP. There are two mainstream open source solutions for that, APC and eAccelerator. And there is Zend platform and Zend core.

Installing eAccelerator
I found a very straight forward howto about installing eAccelerator (and others) on Debian.

But, as always, there are some problems with that tutorial. First the -- is replaced by a special character on that tutorial (and in my blog if I don’t use the <code> tag) in the configure command, so that doesn’t work if you copy paste it. The working configure command is:


./configure --enable-eaccelerator=shared --with-php-config=/usr/bin/php-config

And you don’t need to do all the work as root, only the apt-get, the make install and the php.ini editing should be done as root (best practice, only use root privileges when needed).

Also the double quotes are changed into a special character in that tutorial, so if you get an error in your logs about unable to load \xe2\x80\x9deaccelerator.so\xe2\x80\x9d you copied them just like I did ;-)

This is what I put in the php.ini, without the special double quotes, and with one change


extension="eaccelerator.so"
eaccelerator.shm_size="16"
eaccelerator.cache_dir="/var/cache/eaccelerator"
eaccelerator.enable="1"
eaccelerator.optimizer="1"
eaccelerator.check_mtime="1"
eaccelerator.debug="0"
eaccelerator.filter=""
eaccelerator.shm_max="0"
eaccelerator.shm_ttl="0"
eaccelerator.shm_prune_period="0"
eaccelerator.shm_only="0"
eaccelerator.compress="1"
eaccelerator.compress_level="9"

The tutorial writer creates the cache directory in /tmp. But he fails to make the remark that /tmp is cleared on server restart. So that’s not really a good option. So I’ve done the following in stead


sudo mkdir /var/cache/eaccelerator
sudo chown www-data /var/cache/eaccelerator
sudo chmod 0755 /var/cache/eaccelerator

Yes, the 0644 from the eAccelerator wiki is what you want for the files being created in there, but not for the directory itself.

The big disadvantage of this way of installing it, is that you probably need to recompile it (after a make clean) and reinstall it on every update of PHP. But reading up on APC vs eAccelerator made me choose for the latter.

But does it really help?
Now, the results. What better to test then my own PHP blog (and yes it’s not the fastest one, it’s running on a VPS).

Before installing eAccelerator:

Server Software:        Apache/2.2.8
Server Hostname:        blog.delgurth.com
Server Port:            80


Document Path:          /
Document Length:        44436 bytes


Concurrency Level:      10
Time taken for tests:   397.121299 seconds
Complete requests:      1000
Failed requests:        0
Write errors:           0
Total transferred:      44796000 bytes
HTML transferred:       44436000 bytes
Requests per second:    2.52 [#/sec] (mean)
Time per request:       3971.213 [ms] (mean)
Time per request:       397.121 [ms] (mean, across all concurrent requests)
Transfer rate:          110.16 [Kbytes/sec] received


Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   0.3      0       6
Processing:   712 3961 1674.7   3696   10696
Waiting:      368 2177 1013.3   2033    6943
Total:        712 3961 1674.7   3696   10696


Percentage of the requests served within a certain time (ms)
  50%   3696
  66%   4454
  75%   4977
  80%   5270
  90%   6225
  95%   7039
  98%   8188
  99%   8632
 100%  10696 (longest request)

After installing eAccelerator

Server Software:        Apache/2.2.8
Server Hostname:        blog.delgurth.com
Server Port:            80


Document Path:          /
Document Length:        44436 bytes


Concurrency Level:      10
Time taken for tests:   242.584585 seconds
Complete requests:      1000
Failed requests:        0
Write errors:           0
Total transferred:      44796000 bytes
HTML transferred:       44436000 bytes
Requests per second:    4.12 [#/sec] (mean)
Time per request:       2425.846 [ms] (mean)
Time per request:       242.585 [ms] (mean, across all concurrent requests)
Transfer rate:          180.33 [Kbytes/sec] received


Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   0.2      0       4
Processing:   430 2419 983.2   2330    6409
Waiting:      100  648 448.6    578    3025
Total:        430 2419 983.2   2330    6409


Percentage of the requests served within a certain time (ms)
  50%   2330
  66%   2798
  75%   3051
  80%   3246
  90%   3712
  95%   4182
  98%   4614
  99%   4901
 100%   6409 (longest request)

So it helps also quite a bit on my wordpress install, it’s about a 33% performance increase!

Some material I read

P.s. yet another wordpress rant, if you have empty lines between <code> tags, wordpress inserts <p> tags there for you. Not very nice to my liking, since it breaks the formatting of the <code> tag, which I used to format the apache bench results…

Using VMware player for testing Internet Explorer on Ubuntu

2 Replies

One disadvantage of developing websites/web applications on Unix (in my case Ubuntu) is that it’s hard to test things on Internet Explorer. Yes, you have IEs4Linux but that’s just not the same thing.

For real testing with the different versions of Internet Explorer Microsoft makes images available so you can test your websites/web applications in Microsoft Virtual PC. Unfortunately it’s not easy to use them when you develop on Unix.

So I wanted to see if I could convert these images into VMware images, since VMware player can run on Ubuntu. And it seems this is possible. Even though I’m not entirely sure I don’t break any EULA’s with this… To get it all working you need a Windows machine (with administrator rights) on which you need to unpack and convert the images. This conversion is not needed when you use VMware workstation. This conversion needs to be done every 6 months or so, since the windows version on the VPD has an expiry date.

Installing VMware player
Download VMware player tar.gz unpack it and run the installer as root.

When you are installing it you might get the following error (if you have a newer kernel):

include/asm/bitops_32.h:9:2: error: #error only <linux/bitops.h> can be included directly

But long live people who blog about their experiences, here is the solution for this problem. The writer forgot to put sudo before some of the commands and he assumes you came on his site after it failed and you didn’t try to remove the partially installed VMware player. Here are the steps with all the sudo’s:

  1. cd /usr/lib/vmware/modules/source
  2. sudo cp vmmon.tar vmmon.tar.orig (if you want to be able to undo your changes)
  3. sudo tar xvf vmmon.tar
  4. sudo vi vmmon-only/include/vcpuset.h
  5. change line 74 from: #include "asm/bitops.h" to: #include "linux/bitops.h"
  6. sudo rm vmmon.tar
  7. sudo tar cvf vmmon.tar vmmon-only/
  8. sudo rm -rf vmmon-only/
  9. sudo vmware-config.pl

Then you have a working VMware player.

Getting the Virtual PC Images
Microsoft makes VPC images available for IE6 and IE7, and now also the IE8 beta 1. Just download them and unpack them on a windows machine.

Converting the Virtual PC images
Get VMware converter to convert the free Virtual PC images to VMware images. Before you can convert them you first need to make a VPC image for the VHD. This can be done by running Virtual PC 2007. After that you can put the image in VMware converter and convert it into a VMware virtual machine. This is the part where I’m not sure if it’s legal or not. When converting the image, don’t forget to set the checkbox for the VMware tools.

Using the Virtual PC images
Because we use VMware player, we don’t have the VMware tools package. Lucky for us you can download the VMware workstation evaluation copy and unpack it. Then you need to copy the iso images in there to make sure that the VMware tools can get installed (you might get a message about inserting the windows xp sp2 cdrom, but if you just cancel those, the virtual machine still works good enough to test IE). You might wonder if this is legal, but according to this thread it should be ok. The following command I used to copy them, but you need to be in the directory below the one where you unpacked the workstation evaluation copy.

sudo cp -r vmware-distrib/lib/isoimages /usr/lib/vmware/

After that you are ready to use the Virtual PC images with VMware tools.

Other nice things to do

Using your existing windows partition
Other thing you can do is use your dual boot windows partition. For now no blog post about that, since I didn’t try that out myself. But here are some links about it:

When you use the same windows install from disk and from VMware, you will run into problems with the Windows hardware based activation. But also for that problem a a solution exists and is blogged about. It’s pretty simple, if you know it. It seems that windows activation creates two files that contain the activation information. If you just keep both versions of the activation information available and copy them into the right place at the right time, all should work well ;-)

Running things outside the VMware scope
If you want the Internet Explorer in a window outside your virtual machine, you can do that using rdesktop. Some information about that can be found on the following sites.

A side note
Unfortunately, while writing this post I encountered one problem with referring to other articles which contain the HOWTO for your problem. When you google for Run Existing Windows Installation on Ubuntu with Vmware Player you will find a lot of referrals to one specific article, which is no longer available. This is the main reason why I copy/pasted the steps to install VMWare player, in stead of only linking to them.

SVN merge changes from a deleted branch

5 Replies

Today I had a problem with a merge. I thought I had merged all changes from a branch into the trunk, so I deleted the branch. But somehow (probably forgetting to remove a --dry-run) not all got merged. So I had to merge the changes again. I thought I could do that with just a:

svn merge -r 3173:3246 $SVN/project/branches/MY_BRANCH .

Where 3173 is the revision in which the branch was created and 3246 is one revision before I deleted the branch.

But this gave me a svn error:

svn: '/svn/project/!svn/bc/3251/branches/MY_BRANCH' path not found

That 3251 is the HEAD revision number. So even though I’m specifying the revisions I want to merge, SVN first checks the given path using the HEAD revision. So in stead of the above command I had to specify the revision in the SVN URL as well:

svn merge -r 3173:3246 $SVN/project/branches/MY_BRANCH@3246 .

This worked perfectly. So if you know you have to specify the revision just before the deletion twice, it’s not that hard to merge changes from a deleted branch. I was already thinking about a checkout of the deleted branch and lots of copy’s. But fortunately for me that wasn’t necessary.

WordPress … I really need to change my blogging software

Leave a reply

Today I accidentally found a bug in a version of a wordpress stats plugin. I searched for a word with a “‘” in it (foto’s) and as a result I got the following error on some blog:


WordPress database error: [You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 's','nl','Linux','Firefox','Google','','')' at line 1]
INSERT INTO turbostatpress (date, time, ip, urlrequested, agent, referrer, search,nation,os,browser,searchengine,spider,feed) VALUES ('20080222','16:41:43','MY-IP-HERE','','Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.8.1.12) Gecko/20061201 Firefox/2.0.0.12 (Ubuntu-feisty)', 'LONG-REFERER', 'foto's', 'nl', 'Linux', 'Firefox', 'Google', '', '')

I’m wondering… Will this bug also appear if I change my browser identify string into something with a single quote in it? Seems that this idea has already been tested + fixed in the latest version of StatPress. ‘Unfortunately’ the $_SERVER['HTTP_REFERER'] string is escaped automagicly. Which seems to be done by wordpress itself, since if I try to echo $_SERVER['HTTP_REFERER'] within the StatPress plugin the ‘ in my referrer is escaped with a \, while this is not the case in a normal PHP script. I say unfortunately since the StatPress plugin writer doesn’t escape the referrer himself.

Most striking though was that I just read a post from Tom Kyte about bad practice on SQL. The quote “… apostrophe is often mistaken for a piece of computer code, corrupting the system. …” was nice. So don’t use apostrophes please, they corrupt computer systems ;-)

P.s. True this bug is not related to wordpress itself, since it was just a plugin writer who screwed-up…

P.s.2. I think spelling checking software should add HTTP_REFERER to their word lists, since it’s a RFC approved spelling error ;) .

Creating your own debian package for non-source applications

Leave a reply

For magproductions I needed a debian package of our “own version” of Eclipse (latest Eclipse with certain plugins pre-installed).

At first I was told to look at checkinstall, but since I didn’t have a makefile, that was not the answer. In the end I used the debian-administration guide to create my own package.

I changed the install into a few mkdir -p and cp -r commando’s and the clean into a rm -r command. That’s basically it. I didn’t think it would be that simple.