Install Ubuntu Linux Cont'

Today, I go to fine tune the Ubuntu Linux in my desktop. One disadvantage of Ubuntu comparing with Debian is Ubuntu supports less packages officially. For example, the SCIM, a Input Method modules support many languages, is not included in Ubuntu.

To install the SCIM, I includes the "Universe" packages pool in my Ubuntu and install the SCIM. The "Universe" pool contains some extra packages which is not officially support by Ubuntu team. After install the SCIM, I try to run the "Setup" of SCIM. However it can't run properly -- seg. fault. As a Debian user, this problem becomes the weak points of Ubuntu. In Debian, it officially support much more packages than Ubuntu. I think Ubuntu need to put more effort to support more packages.

Finally, I decide to install a Debian on my desktop and give up the fine tuning the Ubuntu. The installation of Debian is more complicated than Ubuntu. Fortunately, I am a experienced Debian User.

After installation complete, the XWindow can start correctly. However, my sound card doesn't work. To solve this, I need to install ALSA package via the "aptitude" (Note: aptitude is a Debian package manager). Moreover, the middle button of my mouse doesn't work and this is similar to the case of Ubuntu. The solution was mentioned yesterday's BLOG entry. Then, I install the SCIM and other Chinese related packages. The SCIM works properly!

Install Ubuntu Linux

According to DistroWatch, Ubuntu Linux is the most popular Linux distribution. Today, I want to experience how good it is.

The installation is quite simple and straight forward. It can detect all my hardware and can run up XWindow without any manual settings. My sound card and CD RW also works properly.

There is one problem on my mouse. The middle key doesn't works. With hours of investigation, I find the solution. The problem is due to the "psmouse" kernel module. It detects my mouse incorrectly. To solve the problem, the parameter "proto=imps" should be used while loading the kernel module. So, I create a file "psmouse" in "/etc/modprode.d" with the following contents:

options psmouse proto=imps 

CSV File Format

CSV stands for Comma Separated Values. It is a common file format for storing tabular data, e.g. spread sheet data. The file format of CSV is simple -- a text file, values are separated by comma (,) and rows are separated by newline. However, there is still some tricky in the format.

The tuck point is how to escape the comma in the values. If a value contains commas, the value should be quoted with double-quote ("). e.g.:

value one,"value two with ',' inside",value three

If a value contains a double-quote, the value should be quoted with double-quote and the double-quote in the value should be escaped with another double-quote. e.g.:

value one,"value two with '""' inside",value three

The specification of the CSV is described in RFC4180.

Even there is an RFC standard for CSV format, there are some deviation standard exists. In some applications, the comma and double-quote is escaped by a back-slash (\). Moreover, the character encoding is not stored in the file format and causes ambiguity. So, it is not trivial while using this format.

Missing Samba after Upgrade Packages

My Old PC is a Debian Linux box. I have not upgraded the packages for a long time. Therefore, yesterday, I run Aptitude to do a mass updates. Today, I find the Samba is removed after the mass updates.

I run Aptitude to figure out the reason. I found the Samba has some broken dependency -- libdb4.1. Then, I search the libdb4.1 on the Web. In the unstable branch of Debian, it only supports AMD64 Architecture. Oh! Some packages in Debian is moving from unstable to stable branch.

So, I edit the "/etc/apt/sources.list" to add the stable repository:

deb http://ftp.hk.debian.org/debian/ stable main non-free contrib

Then run Aptitude and update the package list. The broken dependency problem solve and Samba can be installed.

Free Virtual Machine -- QEMU

Long time ago, I have tried QEMU. I run QEMU in Debian and the guest OS is Win2000. It worked but it was very slow. Recently, I visit the site of QEMU and there is a QEMU Accelerator Module. According to the site, with the accelerator, QEMU's performance is comparable to commercial products.

I want to try QEMU in Windows XP. However, the accelerator for windows is alpha version. Anyway, I still want to try to download and run it on my Win XP. The guest OS is the sample Linux which can be download the QEMU's website. It crashes in LILO. Too bad!!!! After that, I try to run QEMU without the accelerator. It works fine. I think the accelerator for windows need to be improved.

Then, I try another accelerator module -- qvm86. This is a open source accelerator module. This module is even worse. It crashes my WinXP.

Related Links:
Offical site of QEMU
QEMU porting on Windows
QVM86

Trojan Found

Two days ago, my sister reported to me that her computer had torjan. On one hand, IE has many many security bugs. On the other hand, she have not done Windows Update for a long long time. I guess the trojan was planted by visiting some bad web-site. Actually, I have told her don't use IE, use Firefox!

The trojan was found by AVG virus scanner. Actually, there are two trojans found: PSW.Generic.BWM and PSW.Generic.DAP. I search them in google, and nothing related found. The infected file are "WINDOWS\SYSTEM32\msd.dll" and "WINDOWS\SYSTEM32\roddll.dll".

For the filename "roddll.dll", I find some information on the Web. There is a trojan named "Troj/Ragrok-A". This trojan creates "WINDOWS\rundll132.exe" and the "roddll.dll" file. Moreover, it also creates a registry entry to execute the "rundll132.exe" in HKLM\SOFTWARE\Microsoft\Windows\CurrentVersion\Run. So, it is not difficult to remove.

For the "msd.dll", there is not much information find. Anyway, I inspect the HKLM\SOFTWARE\Microsoft\Windows\CurrentVersion\Run in the registry, I find a suspect entry: "WINDOWS\system\rundll32.exe". I search on Google by key words "rundll32.exe" and "trojan". I find that rundll32.exe file is located in the c:\windows\System32 folder. In other cases, rundll32.exe is a virus, spyware, trojan or worm! Then, I remove the entry in the registry, remove the "rundll32.exe" in "WINDOWS\SYSTEM", as well as the "msd.dll". Then, reboot. Everything works fine! The AGV scanner doesn't find any trojan.

Obfuscate the Spammer's Robot

Most email users should have experience of receiving junk email. For me, my HongKong.com account, it receives around 100 junk email each day. This wastes our time to filter the email and makes us feel annoying.

Do you know how a spammer get your email addresses? Actually, your email addresses may already be put on the Web which can be accessed by the public. For example, in web-sites of many companies, they put contact lists containing email addresses of its staff. spammers use a program, called robot, to automatically retrieve email addresses on the Web. The spammers' robots move on the Web and try to get the email addresses. So, don't put your email addresses publicly.

However, sometimes you can't avoid it. So, it is suggested to post your email addresses on the Web in a obfuscated form. For example:

john [at] google [dot] com

The robot is a very simple program which can only recognize the email addresses in the form of "xxx@xxxx.xxx". Once the email address is transformed, the robot can't recognize it, but human can. However, it is not user-friendly. I suggest to use JavaScript to transform the the obfuscated email addresses back when mouse is over the email addresses. Here is a real example in my homepage: Contact. Such that, someone can copy and past the email address directly.

Get the Date of Tomorrow/Yesterday in Shell Script

In shell script, we may need to get the date of tomorrow/yesterday. If you use GNU version of "date", it is a piece of cake:

date -d tomorrow +%Y-%m-%d

The above command will display the date of tomorrow in "yyyy-mm-dd" format. Similarly, to find the date of yesterday:

date -d yesterday +%Y-%m-%d

The power of the GNU date command is out of my imagination, see the following examples:

date -d "2 days ago"
date -d "3 weeks ago"
date -d "1 month ago"
date -d "3 days"
date -d "4 weeks"
date -d "2 month"
date -d "2 month 1 week"

However, if your "date" is non-GNU version (e.g. in Solaris), it doesn't support "-d" option. Then, there seems no simple command can do this. You may need to write Perl script or use other more powerful programming/scripting language to achieve same results.

Power of "find"

"find" is a command line utility. Its basic usage is to find some files. e.g.

find . -name '*.html'

The above example is to find all HTML files under current directory.

While building a homepage, it is common to change HTML and graphic files to 0644 permission. We can use the "-exec" option of find to execute the "chmod"; command to change the permission of the files, e.g:

find . \( -name '*.html' -o -name '*.png' -o -name '*.jpg' \) -exec chmod 0644 {} \;

The parameters after "-exec" is the command to be executed. In the command, "{}" is replaced by the found file. The "\;" is the terminator of the command of "-exec".

Comments Submission

I continued to my Blog development. In the comments text area of comments submission, my original implementation did NOT allow any HTML tag. It was mainly due to security reasons. However, this made the text of comments looks dull. I think it is better to allow some simple formatting tag e.g. <p>, <b>, <i>....

After did some survey on the web, I found the most popular technology is BBCode. BBCode defines a set of tags in form of "[xxx]" which will be converted to HTML tags. For example, "[b]bold text[/b]" is converted to "<b>bold text</b>". There are several free BBCode parsers available. It is convenient to deploy this technology.

One disadvantage of BBCode is not user-friendly. Users need to learn a new set of code. I thinks it is better to have a WYSIWYG editor. Unfortunately, I cannot find any web-based WYSIWYG editor for BBCode.

On the other hand, I found some open source web-based WYSIWYG HTML editors. It is great! I have tried the Cross-Browser Rich Text Editor and it works well in both Firefox and IE.

However, it is not secure to accept all HTML tags. On server side, it should filter out some dangerous tags, e.g. , .... There already exist some HTML filters. One of them is safehtml. Finally, I decided to use the WYSIWYG HTML editor plus HTML filter.

Java Generics

J2SE 5.0 have been released for a long time. However, I didn't have time to study it. Today, I spend some time to have a look of it's new features. One new feature introduced in J2SE 5.0 is generics.

In Wikipedia, the term generics is defined as:

generics is a technique that allows one value to take different datatypes (so-called polymorphism) as long as certain contracts such as subtypes and signature are kept. The programming style emphasizing use of this technique is called generic.

For example, a List object can contains different elements String, Integer, etc. In generics, we can declare a List object only contains certain types in elements. E.g, List as a List object only contains String objects.

Generics is easily mixing up with the term template. In Java, the syntax of generics is similar to Template in C++. However, generics and Template are two different concept. Template is about code generation, and generics is about type checking. You can refer to a Wikipedia's article: Comparison of generics to templates.

With generics, it can avoid type-safe problem of collection framework of Java. However, the generics in Java is very complicated. It is not elegant. After you read the tutorial of generics, you may feel very confusing, and you may find that there are many exceptional cases need to take care when using generics in Java.

Fix the RAID

Yesterday, I found that one of my harddisk in my Old PC has bad sectors. I decided not to replace the bad harddisk but to relocate the root partition to other free spaces in the harddisk. Originally, the RAID of root filesystem contains the hda8 and hdc8 partitions. The bad sectors were found in hda8. I try to search whether there exists unused partition on hda. I find hda7 is not used and the size matches the hdc8. To add hda7 to the RAID:

raidhotadd /dev/md0 /dev/hda7

From "/proc/mdstat", it shows hda7 is adding to md0. Wait for a while, mirroring of data from hdc8 to hda7 finished. I restart the PC.

After startup, it fails. The OS still try to use hda8 as the RAID. After some inspection, it is due to the configuration problem. First, it needs to correct the "/etc/mdadm/mdadm.conf" file. Second, it needs to regenerate the initrd image using the "mkinitrd" command. Lastly, it needs to correct the GURB config. Then, I restart te PC and this round, it works!

GIF vs PNG

Today, I test my homepage with IE. IE cannot display transparent PNG graphics properly, and the background becomes grey in color.

Fortunately, I find a workaround on the Web -- http://homepage.ntlworld.com/bobosola/. It has details about the problem and how to solve it. In short, it uses javascript to patch all <img> tag of PNG graphic's CSS using the "filter:progid:DXImageTransform.Microsoft.AlphaImageLoader" of IE.

Actually, PNG has some advantages over GIF.

1. Max color depth of PNG is 24-bit but that of GIF is 8-bit.
2. PNG has higher compression rate than GIF.
3. PNG support alpha channel transparency but GIF only support binary transparency.

However, GIF supports animation but PNG not. This is a weak point of PNG.

Related Links:
AlphaImageLoader Filter
PNG Offical Site

Makefile to Build Homepage

After study GNU Make yesterday, I write a generic Makefile for building my homepage today.

The the following is the Generic Makefile:

PHP=php

all: premake all_subdirs all_curdirpages postmake

premake:
 if test -x ./premake.sh; then ./premake.sh; fi

postmake:
 if test -x ./postmake.sh; then ./postmake.sh; fi

include Makefile.dep

all_subdirs: $(subdirs)
 for dir in $(subdirs); do (cd $$dir; make); done

all_curdirpages: $(pages)

$(filter %.html,$(pages)): %.html: %.tpl
 $(PHP)  $< > $@
 touch timestamp

clean:
 for dir in $(subdirs); do (cd $$dir; make clean); done
 rm -f *.html timestamp *.gen

.PHONY: all clean all_subdirs all_curdirpages premake postmake $(subdirs) $(dep_phony)

This Makefile is put (symbolic-link) in all sub-directories of the homepage. In other words, all sub-directories use the same Makefile. So, it is called generic Makefile. If it needs to add some extra logics before or after make, we can create "premake.sh" and "postmake.sh" shell scripts respectively. The scripts will be executed accordingly. In the Makefile, the marco "pages", "subdirs" and "dep_phony" are provided by the "Makefile.dep" The customization depends on the "Makefile.dep". In each directory, it should contain a "Makefile.dep". The following is a sample "Makefile.dep":

subdirs=blog secImage

pages=index.html tba.html contact.html

index.html: index.tpl header.tpl footer.tpl rightbar.tpl \
 lib/search.tpl lib/acknowledgement.tpl blog/recent/index.html

contact.html: contact.tpl header.tpl footer.tpl rightbar.tpl \
 lib/search.tpl lib/acknowledgement.tpl

tba.html: tba.tpl header.tpl footer.tpl

The "subdirs" macro defines the the sub-directories requiring to process, and the "pages" macro defines the HTML pages requiring to generated. The "Makefile.dep" file also defines the dependency of HTML files to the template files (*.tpl).

To build the homepage, I only need to issue the "make" command in the root directory of the homepage source.

Designing My Homepage

I want to host my homepage in my NLSU2-Linux box. Today, it's the time to think about it. Since the NLSU2-Linux box is not a powerful machine, it is not appropriate to use Java technology. To put less load on the CPU, it should avoid using too much dynamic pages.

In my past experience, using static HTML, there are many duplication parts among different pages, for example, the page header, page footer, menus.... This make the web-site difficult to maintain. If I need to changes the page header, I need to edit nearly all HTML pages repeatedly. In programming, we avoid code duplication by using functions. How about make web pages? I think template is a solution.

In my homepage, I decided to use PHP as the template language. (To avoid confusion with PHP scripts for dynamic pages, the extension of template files is ".tpl") The following shows a typical page contains header and footer in PHP:

<? include 'header.tpl' ?>

.... the contents here .....

<? include 'footer.tpl' ?>

The "header.tpl" and "footer.tpl" are PHP scripts which generate the header and footer HTML fragments respectively. In PHP, the "include" function embedded the contents of the specific file in current pages, similar to the "include" directive in C/C++. In the homepage, every pages contains header and footer will be like the above example. If we want to change the header, we only need to change the "header.tpl" file once.

However, we need to convert the PHP file to HTML. For example, we have an "index.tpl" file which is a PHP script for generating "index.html". We can issue the following commands:

php index.tpl > index.html

For a complete homepage, there should be many ".tpl" files. I think nobody want to type the above command line again and again. GNU Make is a good utility to do this task.

Since I only have some experience to create a simple "Makefile", it is not enough for handling a complete homepage. I spent some hours to study the GNU Make and hope I can create a generic Makefile tomorrow.

Related links:
PHP
GNU Make

Purchased Linksys NSLU2

I want to setup a 7x24 server at home. The main purpose of this server is for hosting my homepage and files sharing with my friends. After some googling, I found the Linksys NSLU2 is quite suitable for this purposes.

NSLU2 is just a SOHO-grade network attached device with simple functions. However, it can be turned into a more powerful Linux box with the help of NSLU2-Linux project. Then, I can have a lower power consumption, ultra-slim and fanless server :P

Finally, I bought it at Golden Shopping Center. The following is a photo of my running NSLU2 : )

Start Hacking the NSLU2

Today, I had annual leave. I had full day to hack my NSLU2 box. After read the wiki of NSLU2-Linux project, I decided to install the Unslung firmware. I followed the user guide and reflashed the firmware of the NSLU2 box. Everything worked fine. :)

Next, I setup the SSH, so I can remotely access the NSLU2 box. Then, PHP, Perl, and other useful utilities have been installed. I also setup a native compilation environment in the NSLU2 box, such that it can compile additional software packages.

Moreover, I tried to setup a cross compilation environment in my old PC which running Debian. This can help me to compile any additional packages faster. However, I fail to compile the cross compilation environment. Then, I read the guide in Wiki carefully, I found that I missed to install some necessary packages (flex, bison, rsync...). After installed the necessary packages, the cross compilation environment was compiled successfully.