A way out of all those Wubi problems

With Ubuntu 9.10 came a serious problem with Wubi: sometimes, it just won't load the kernel, sometimes, it won't find the initial RAM disk. It's something I've already covered here. What makes the problem worse is that it's intermittent. You don't know when it will strike.

Apparently, this is the cause:

To be able to boot Wubi, Grub2 has to access the ntfs partition which is hosting Wubi. There is a bug in Grub 2, which prevents Grub2 to read any files on an ntfs partition beyond the first 4GB. If any of the boot files is outside of the 4GB limit, booting will fail. Depending on which boot file is outside the limit, the symptoms can be quite different. Since any kernel or Grub update relocates some of the boot files, you might be hit by this bug at any time.

To fix the problem, go to the link above, download the new wubildr, and replace the one you have in C:\ of your Windows partition.

Removing the user list from Gnome 2.28 login screen

From a security standpoint, it's not wise to present a list of users at the logon window. Why give intruders an idea of the accounts on the system, after all? Why make it easier for them to guess for access?

Unfortunately, Gnome 2.28, which comes with Ubuntu 9.10, uses the dropdown list by default, and what's worse, doesn't give you any easy and intuitive way of changing it. That's just sad.

In order to modify this, you'll have to open a terminal window and execute:

sudo -u gdm gconftool-2 --set --type boolean /apps/gdm/simple-greeter/disable_user_list true

Idea from Ubuntu Tutorials: Remove the User List from Ubuntu 9.10 GDM.

Finding duplicate files

Given the ease with which we create and copy files, it's inevitable to end up with duplicates in our hard drives. Not a good situation, of course: duplicates take up space, and can sometimes lead to inconsistencies in data. Wouldn't it be nice to be able to sniff out these unwanted copies?

fdupes is a small utility that does just that. It's only 18Kb, but darned! it's fast. On my quad-core, it can sift through a thousand PDF files in under two minutes and churn out a list of the dupes.

The man page explains everything you need to know. It's probably best used with its recursive option, though:

fdupes -r mydirectory/

Update on Wubi/Grub2 problem

Several weeks after it was first reported, the Wubi/Grub2 problem still remains unresolved. On the bright side, neither I nor my students have encountered it again, but that's probably because I told my class to hold off on any updates until a fix comes out.

I'm following this issue on Launchpad (Bug #477169). So far, the most succinct diagnosis of the problem:

The problem is now identified, grub2 in some cases recognizes loop0 as an ext2 when is an ext4 (and so it is unable to read it properly)! But is strange how it is taken so much time to find a solution...

However, I feel that the workarounds presented for this issue are beyond the ken of the average user. I gave up trying to fix it using those methods because they were just too time-consuming. So I suggest two things that have worked for me so far:

1) Back up your work files.

2) For Wubi installations, do not run any upgrades that might touch Grub2 or initrd (typically these will involve kernel updates.)

Kernel panic - not synching

I'm about as big a booster of Ubuntu as you can find without being actually part of the Ubuntu team. But from yesterday till today I've found myself cursing Karmic Koala. The reason: as of yesterday afternoon, after an update, the Wubi install of Ubuntu 9.10 on my HP Mini 2140 just stopped working.

In more gory detail: after going into Grub, I would get the following message:

Kernel-panic - not syncing : VFS : Unable to mount root fs on unknown-block(0,8)

Was I upset? You betcha! I had some important files in that partition that I had not backed up yet.

Read more »

Defragging in Linux

An interesting topic in the forums: does Linux need defragmenting? The most insightful answer, from irne.barnard, below:

I'd actually strongly suggest not defraging ... the reason behind this? Even on windows most defraggers have 2 options, 1 Defragment, 2 Compact ... sometimes called something different, but the end result's the same.

The "defragment" is supposed to make all files into contiguous blocks. The "compact" is supposed to defragment and make the free space into a contiguous block. Now while this sounds nice, the reality is quite different:

Fragmentation does cause productivity loss as each file access now could become hundreds of HDD reads / writes. The file system makes all the difference:

1. FAT based file systems save each file directly following each other. So if you later edit / add to this file, the added portion needs to be saved somewhere else. This will create a fragment (or more than one). A new file is saved starting from the 1st blank spot, even if that blank spot is too small to contain the entire file.

2. NTFS is a bit better in theory, it allows some free space around each file. Then if it notices that the file will become fragmented, it "attempts" to save the entire file in a new location. The caveat to this is "efficiency": if it would not take too much time to save the entire file over again it will be done, otherwise just create a new fragment. Just how it determines "too much time" is up for grabs ... and like most M$ ideas also a secret!

3. Ext3/4 also generates blank spaces "around" each file. Where it differs significantly from NTFS is that it'll only fragment when it's impossible to keep the file as one single fragment ... that (usually) only happens when the disk becomes too full.

Darik's Boot And Nuke

A question came up in the forums about the best way to erase a hard disk. All respondents unanimously pointed to Darik's Boot and Nuke. According to the site:

Darik's Boot and Nuke ("DBAN") is a self-contained boot disk that securely wipes the hard disks of most computers. DBAN will automatically and completely delete the contents of any hard disk that it can detect, which makes it an appropriate utility for bulk or emergency data destruction.

Now it can't get any simpler than that.

Why would you want to do such a thing? Well, let's say you're selling your computer. You'll want to make sure there's no trace whatsoever of any incriminating data. It's happened many times before that a compnay would sell a bunch of their old systems and some newshounds still managed to recover some old files, much to the seller's embarrassment.

ClamAV -- fail!

Using a USB drive infected from a Windows PC from school, I attempted to scan for viruses using ClamAV. Result, though: a big, epic fail. ClamAV could not recognize the infection even when I used the latest virus definitions from the ClamAV web site. Nothing to do but shake my head and claim disappointment; of course, since I'm new to virus detection on Linux and ClamAV, the fault may be mine. Still....

I first attempted the scan using ClamTK, which I wrote about earlier. However, I've been having doubts about ClamTK because there's no way for me to check on the validity or currency of the virus database.

I then ran the scan using the command-line tool clamscan, using the virus definitions main.cvd (released ver. 51 released on 14 May 2009 10:28 :0400) and daily.cvd (ver. 9814 released on 17 Sep 2009 13:17 :0400) from the ClamAV site. The command I used was:

clamscan -d CVD -r /media/disk

where CVD was the directory where I put the virus definition files. The result:

----------- SCAN SUMMARY -----------
Known viruses: 623483
Engine version: 0.95.2
Scanned directories: 8
Scanned files: 37
Infected files: 0
Data scanned: 50.70 MB
Data read: 88.91 MB (ratio 0.57:1)
Time: 9.567 sec (0 m 9 s)

So, no detection from ClamAV. However, a listing of the files on the USB disk reveals:

-rwx------ 1 dodgie root 1426205 2009-09-17 16:14 Beej's Guides.exe
-r-x------ 1 dodgie root 1426205 2009-09-17 16:14 Recycle.exe
-rwx------ 1 dodgie root 1426205 2009-09-17 16:14 Sample Programs.exe

Running a search on the file size 1426205 led me to the submitted definition on Threat Expert. Checking the MD5SUM on my infected files gives a match on the submitted MD5 numbers on the site. To be fair, the signature was submitted on September 9, but on the other hand, several of the other AV products seem to have already detected the virus.

So, nothing else to do but to submit the file to the ClamAV folks.
Let's see what happens next.

ClamTK

Because I use Linux, I take a cavalier attitude towards viruses. Probably not a good idea, but really, when did we ever have a major virus outbreak in the wild for Linux. Okay, I'll shut up now, because I don't want to jinx the system.

On the other hand, Linux theoretically makes a good platform for checking for Windows virus infections. I say theoretically because in the end, it all depends on the engine and signature files. What I've seen recommended in many sites is ClamAV; again, there may be pros and cons in the discussion, but I'll leave that to the experts.

ClamTK is the front-end for ClamAV. Simple enough to understand.





I'll try to catch a virus one of these days and report it here.

Thin clients and centralized servers

Today in class I had the students log in to our server, galactus, via XDMCP. This basically turns their machines into thin clients, working off a central machine. Nearer to LTSP, though not quite just yet. I wanted to impress upon the students the idea of centralized management and all the upsides and downsides associated with it.

All told, the exercise went well. galactus, with a dual-core Intel processor and 2GB of RAM, managed to serve twenty or so students well enough with very little palpable load. The CPU utilization stayed steady at around 40% during idle moments. The OpenOffice.org processes barely registered a blip on top.

The only time galactus began to lag was when all the students opened up Firefox. Firefox consistently stayed as the top process. During those moments, CPU idle time went to zero, while the actual load was split between 80% user and 15% system. Fortunately, wait time was still close to zero; otherwise we would have seen thrashing.

Memory still had a few tens of megabytes left over, which fits the profile of other desktop systems that I've handled. Other parameters remained normal.

Given this performance, I would probably recommend a lighter web browser for a thin client setup.

Careful with the Janitor...

Starting with 9.04, Ubuntu ships with a program called Janitor. Janitor is meant to find unused files and packages, giving you the option to delete them and save space. Good idea in theory, not so good in practice.

I gave Janitor a spin and it ended up deleting programs that I was using, among them VirtualBox and Skype! Alright, a large part of it was my fault, because I should have read the instructions and the options carefully. But still...

...use Janitor with care. Or don't use it at all.

Apparently, I'm not the only one who feels this way.

Understanding the Boot Process

Understanding The Boot Process

A presentation I put together on the PC boot process, covering both Linux and, ugh, Windows. This is for the class in Operating Systems that I'm teaching at Ateneo de Davao. I found M. Tim Jones DeveloperWorks article Inside the Linux Boot Process very helpful.

Something I've put aside for a long while, but recently came up -- how Ubuntu differs from the old Linux distros I used in its handling of startup processes. I used to rely on /etc/inittab but I always wondered where it went with Ubuntu. Apparently, it's been replaced by Upstart. More on that next time.

Notes on automated network installation of Ubuntu

Last month, I taught an advanced seminar for Ateneo de Davao's MS in Information Technology program. Per their request, I covered automated network installation of Ubuntu. On the whole, the exercise was successful, but with a few glitches I could have done without.

Now, I've done automated network installation before, but some time way back so I needed a refresher. The Ubuntu Installation Guide provided a quick refresher, in particular, the section on TFTP net booting. Unfortunately, the instructions needed a bit of polish.

I won't go into the process in much detail here as it's already well-covered in several other places on the web. Too many, in fact, such that sifting through the sometimes-contradictory information becomes a challenge. Thus I'm writing this note to point out which sources gave me the most relevant help.

In a nutshell, you need three components on the server to set up automated network installation: the DHCP server, to give out IP addresses and instructions on where to download the boot image; the TFTP daemon, to serve the boot image; and a web server or FTP server to serve out the operating system files.

For the most part, the instructions on TFTP net booting work well. However, it has a problem on its recommendation concerning which TFTP daemon to use.

The Ubuntu repositories offer two TFTP daemons, the plain old tftpd (used in the instructions) and tftpd-hpa. Of the two, tftpd-hpa is easier to set up. tftpd-hpa runs standalone, so you don't need to install and configure additional components.

Tero Karvinen's notes on PXE Network Booting were particular helpful, as well as the Ubuntu community help notes on PXE Install Server.

I'll revisit the topic again with step-by-step instructions at a later time. I'll be taking this up with my undergraduate class later this semester.

Partimage

From the Partimage website:

Partimage is a Linux utility which saves partitions having a supported filesystem to an image file. Most Linux and Windows filesystems are supported. The image file can be compressed with the gzip / bzip2 programs to save disk space, and they can be splitted into multiple files to be copied on CDs / DVDs, ...

And, yes, it's open source and part of the Ubuntu APT repository.

Since it's so easy for me to reinstall Ubuntu, I don't think I actually need Partimage for that. But I did need it, though, for my Windows XP partition. XP was a pain to install, and since it's prone to viruses and such, I felt I needed a quick way to restore XP. Partimage fits the bill.

Following are some screenshots of Partimage. On the whole, really very intuitive to use.



Doubly interesting is the network backup / restore function. According to the web site:

Partitions can also be saved across the network since version 0.6.0 using the partimage network support, or using Samba / NFS.

On the downside, I haven't had time to test the restore function, nor do I think I want to. Till then, I'll just keep my fingers crossed.

Gasp! Dual-booting with Windows XP!

Because of my previous confession, it is with deep embarrassment that I admit that, yes, I am now dual-booting my office computer between Ubuntu and Windows XP. I feel so...tainted. All that gives me comfort now is the thought that I still use Ubuntu 95% of the time and only switch to That Other Operating System when I need a quick gaming fix.

I suppose I reached this stage because, after my last encounter, I just couldn't let the idea go. I wondered why Windows XP wouldn't install on my system. I guessed that it might have been because the disk I was installing on was SATA, and apparently that is indeed the case. Installing Windows XP on a SATA hard disk needs a driver on floppy, and without a floppy, it means additional contortions with nLite.

Who ever said installing Windows was easy?

As it turns out, I did have an old IDE hard disk already installed on my computer. I didn't need to create my own Windows XP installation disk anymore. The IDE only held backup data, and I moved that off to another hard disk. When I attempted the Windows XP installation with the 40GB IDE as a target, XP finally recognized the hard disk.

Installing Windows XP was every bit as painful as I remembered and imagined it to be. Copying files took interminably long, drivers had to be found, and it took several reboots to install all of them. That actually gave me some more appreciation for Ubuntu and its frequent release cycle -- but I'll cover that in a later post.

Along the way, I learned a couple of things about GRUB and booting that otherwise would have escaped my attention. If nothing else, this was worth the pain of the transition process.

To understand the situation, I have to explain my setup. Here goes:

I have three hard disks on my computer: a 160GB SATA hard disk (for Ubuntu), a 500GB SATA hard disk (for data), and an old 40GB IDE hard disk, as previously mentioned. My BIOS allows me to select which of the disks to boot from.

Originally, my system was set to boot from the 160GB hard disk. To install Windows XP, I had to switch to the 40GB IDE hard disk as the primary boot disk. Otherwise, XP will try (and fail) to overwrite MBR of the 160GB hard disk. Besides, I didn't want to have to muck with my already working Ubuntu system.

So, with all the attendant pains, I managed to get Windows XP on the 40-GB hard disk.

After installing XP, I switched the primary hard disk back to the 160GB SATA. With bated breath, I waited for the Ubuntu splash screen to come up...and there it was! Hooray!

Now I knew I could get GRUB to recognize and boot to the Windows XP hard disk. I added the following lines to the bottom of /boot/grub/menu.lst:

title Windows XP
rootnoverify (hd2,0)
chainloader +1

I was already reasonably sure that the disk was on the designated drive. However, after I selected Windows XP on the GRUB menu, the system would just hang. Booting to the Ubuntu partition worked perfectly fine, though. I thought it might have had something to do with the hard drive selection, so I tried the other two combinations, hd0 and hd1, but both resulted in errors.

After much head-scratching, it turned out what I actually needed to add was:

title Windows XP
rootnoverify (hd2,0)
map (hd2) (hd0)
makeactive
chainloader +1

The line map (hd2) (hd0) tells GRUB to present hd2 to Windows XP as if it were hd0. This is because when I installed Windows XP, the 40GB hard disk (now hd2) was the primary (hd0). Therefore I needed this bit of trickery to fool XP.

As per the GRUB manual:

map maps the drive from_drive to the drive to_drive. This is necessary when you chain-load some operating systems, such as DOS, if such an OS resides at a non-first drive.

makeactive performs a supplemental function. It set the active partition on the root disk to GRUB's root device. This command is limited to primary PC partitions on a hard disk.

After these changes, I finally got the system to dual boot. But it sure took a long time.

Sigh. Who ever said installing Windows was easy?

Ubuntu for Intranet Services

Ubuntu For Intranet Services

View more Keynote presentations from Dominique gerald Cimafranca.

A presentation I put together for the Advanced Seminar class of Ateneo de Davao's Masters in Information Technology program. Covers the use of Ubuntu as a server in an intranet setting.

Upgrading from 8.10 to 9.04

I'm a little embarrassed to say this but this is the first time I've upgraded from one version to another. In the past, I would always go the way of a fresh install, a nasty habit I picked up from my Windows days. Unfortunately, it's something I never managed to shake off. Until now, that is.

But with the new release of Jaunty Jackalope, I thought I'd give it a shot. I was already pretty happy with the settings on my laptop running Intrepid Ibex. An upgrade would save me the hassle of reconfiguring everything.

And you know what? The upgrade worked without any major hitches.

Alright, to be honest, I did have to tweak a couple of things (more on that later), but by and large, the upgrade went smoothly. I gotta hand it to the Ubuntu team, the design of the upgrade process is slick and easy.

There's two ways to do the upgrade: over the network through Synaptic, and through a local copy of the Alternate Installation disk. The difference only happens in the beginning. After the first few steps, they converge.

I tried the Synaptic network upgrade first, but decided to abort it on account of a slow network. Regardless, the process is almost idiot-proof. When you fire up Synaptic, you'll see a notice for the new version, alongside an Upgrade button.

What follows next is additional info.


Then, Ubuntu downloads the two files which consist the upgrade tool. The files are relatively small and the download finished quickly.



At this point, the network upgrade and the CD-ROM upgrade now converge. I did let the network upgrade run a bit until I realized it was a Bad Idea. I cancelled the operation, and Ubuntu rolled back to 8.10.

I then decided to download the Alternate Install CD. Once done, I mounted it locally via

sudo mount -o loop /home/dodgie/Desktop/ubuntu-9.04-alternate-i386.iso /media/cdrom0/

On the root of the Alternate Install CD is an executable called cdromupgrade. Invoking the program:

gksu ./cdromupgrade

And it brought me back to the point which you see now. The rest of the pictures are pretty much self-explanatory. Not much more to do except just to answer a couple of prompts.

My next post will be about the minor hiccups I mentioned earlier. Even those could not dampen my enthusiasm for the breezy process.

Ubuntu rocks!

Rename: Stripping Beginning Numbers

I have a whole slew of files that look like this:

01 - filename1.txt
02 - filename2.txt
03 - filename3.txt

Problem: how to remove the numbers and the dash, leaving just the filename?

Solution:

rename "s/^\d\d - //g" *

The expression ^\d\d - basically means, the first two digits (\d) at the start of the line (^), followed by the space and the dash.

Simple trick, really. Putting it here in case I forget.

APT segmentation fault

Segmentation fault is never a pretty sight, and moreso if it involves the APT system. It means that you can't get new software to update whatever it is that might be wrong with your system. But that's exactly what happened to me this morning: my update manager was borking every time it ran.

Running apt-get from the command line wasn't much help either:


Fortunately, the fix was easy, as detailed in this Ubuntu Forums entry. Apparently this is caused by a corrupt APT cache.

To fix this problem, just delete the *.bin files from /var/cache/apt.

sudo rm /var/cache/apt/*.bin

That did the trick!

Even more speedup tricks

My Thinkpad is really showing its age. Even with all the speedup tricks I've used, this laptop still feels slow sometimes. Using the top monitor, I found one of the biggest resource hogs was gdm.

So, I turned it off. I kept it from starting up automatically at rcconf.

But...but...isn't gdm the Gnome Desktop Manager? Indeed it is. If I turned it off, I wouldn't be able to log on to the computer, right? Well, it only affects graphical logins.

Now I start off with a command-line prompt, after which I bring up graphical mode with startx. Performance improvement? You bet!

However, turning off gdm also does have the unfortunate side-effect of disabling automount of USB devices and CD-ROM drives. I still keep gdm handy in case I need to.

One other thing: by default, startx brings up Gnome, but I prefer working within Openbox. I also had to create an .xsession file within my home directory. Only one line is needed there:

exec openbox

Man, I really need to get a new computer.