Archive for the ‘linux’ Category

Recovering your data from clicfs (aka openSUSE Live USB stick)

Wednesday, December 2nd, 2009

Today I wanted to recover some of the data I had saved on my openSUSE 11.2 live USB stick, just to find out that it was not really trivial. It actually included reading the manpage of coolo’s clicfs, so I’ll write it down here to save others the hassle ;)

I assume the normal live USB stick layout (dd’ed the ISO which was treated with isohybrid to the stick, then added a second partition with fdisk, if I plug in the stick it turns up as /dev/sdb1 and /dev/sdb2) for the following.

First, create the mountpoints and mount the “normal” ISO image:

# mkdir /mnt/iso /mnt/root /mnt/clic
# mount /dev/sdb1 /mnt/iso -oro

You’ll now find the clicfs container on the mounted ISO fs:

# hexdump -C /mnt/iso/openSUSE-kde-11.2-livecd-kde-read-only.i686-2.8.0 |head -n 1
00000000 43 4c 49 43 30 32 0b 00  00 00 66 73 64 61 74 61 |CLIC02....fsdata|

Second, mount the clicfs:

# clicfs -c /dev/sdb2 /mnt/iso/openSUSE-kde-11.2-livecd-kde-read-only.i686-2.8.0 /mnt/clic/
# ls -l /mnt/clic/
total 0
-rw-r--r-- 1 root root 3891789824 Jan 1 1970 fsdata.ext3

Third, just loop mount the fsdata.ext3 file:

# mount /mnt/clic/fsdata.ext3 /mnt/root/ -oloop

That’s it. You can now access all data that you stored on your Live stick under /mnt/root.
If you only want to have the “original” contents of the Live system, without the stuff you saved, then just replace the “-c /dev/sdb2″ with “-m 128″ which will use a ramdisk for the “write” part of the filesystem instead of the second partition on the stick. (Don’t just leave off the “-c …” without adding “-m”, this got me a hanging mount later when I was trying the loop mount. I was able to resolve it with a brave “umount -f /mnt/clic” and then “killall -9 clicfs” though).

To get rid of it, just unmount it in reverse order:

# umount /mnt/root
# umount /mnt/clicfs
# umount /mnt/iso

More ipw2200 eeprom fun

Monday, November 30th, 2009

As written in my last post, I have packaged an ipw2200 kmp that enables writing to the eeprom. Incidentally, looking closer at one of my other machines, it turned out to have an ipw2200 that had the same problem as Obi’s, namely not being able to use all 13 11b/g channels that are allowed here:

ipw2200: Detected geography ZZM (11 802.11bg channels, 0 802.11a channels)

The hexdump of the eeprom showed that the location of the geography code was the same as in Obi’s case:

0040 9c e8 00 0e 35 d0 fa 25  00 46 00 01 5a 5a 4d 00 ....5..%.F..ZZM.

So I installed the ipw2200-eepromwrite-kmp from my build service repo and fixed the card.

blub:~ # ethtool -E eth1 magic 0x2200 offset 0x4e value 0x44
Cannot set EEPROM data: Invalid argument

That’s one of the errors I wrote about in my last post, but it did no harm:

blub:~ # dmesg|tail -1
ipw2200 eeprom_write_u16(27, 0044)

shows that the eeprom write function was actually called.
Unloading and reloading the ipw2200 module then gave me:

ipw2200 0000:02:06.0: PCI INT A disabled
ipw2200: Intel(R) PRO/Wireless 2200/2915 Network Driver, 1.2.2kdmprq
ipw2200: Copyright(c) 2003-2006 Intel Corporation
ipw2200 0000:02:06.0: PCI INT A -> Link[LNKE] -> GSI 11 (level, low) -> IRQ 11
ipw2200: Detected Intel PRO/Wireless 2200BG Network Connection
ipw2200 0000:02:06.0: firmware: requesting ipw2200-bss.fw
ipw2200: device failed to start within 500ms
ipw2200: EEPROM checksum is invalid (0xe89c, should be 0xe895 instead).
ipw2200: To let the driver fix it use the module option 'repair_eeprom'.
ipw2200: Unable to load firmware: -62
ipw2200: failed to register network device
ipw2200 0000:02:06.0: PCI INT A disabled
ipw2200: probe of 0000:02:06.0 failed with error -5

So I did what the driver asked me and used “modprobe ipw2200 repair_eeprom=1″:

ipw2200: Intel(R) PRO/Wireless 2200/2915 Network Driver, 1.2.2kdmprq
ipw2200: Copyright(c) 2003-2006 Intel Corporation
ipw2200 0000:02:06.0: PCI INT A -> Link[LNKE] -> GSI 11 (level, low) -> IRQ 11
ipw2200: Detected Intel PRO/Wireless 2200BG Network Connection
ipw2200 0000:02:06.0: firmware: requesting ipw2200-bss.fw
ipw2200: device failed to start within 500ms
ipw2200: EEPROM checksum is invalid (0xe89c, should be 0xe895 instead).
ipw2200 eeprom_write_u16(20, e895)
ipw2200: Detected geography ZZD (13 802.11bg channels, 0 802.11a channels)

Now everything is fine and I can also use channels 12 and 13.

Note that you should know what you are doing and be willing to sacrifice your wifi card if you are attempting anything like this. It worked for me, but that does not at all imply that it will work for anybody else. You have been warned! ;)

HP does not want my wireless to work

Saturday, November 28th, 2009

I’m using an rather oldish, but still functioning hp compaq nc6000. It had built in atheros wireless. Whis is a pain. Because whenever NetworkManager thinks it is a good idea to do background scanning (at least every 2 minutes, I’d guess), the network quality degrades to it being unusable. Not only drops the data rate to ~100 to 200 kbyte/second, even worse, lots of packets get lost, so that working via SSH on a remote host becomes very annoying.
(With a recent kernel. Older kernels actually regularly stopped transmitting and dropped the connection completely during such events). Sometimes, reloading the ath5k module improves the situation (or at least it feels like it does), but then, every 5 reloads or so, the machine locks up hard (no SysRQ!) when reloading ath5k.

Today, that happened again. This was when I decided that I had enough and that I would swap the card for an intel ipw2200 I had lying around here.

But HP, THOSE GREEDY BASTARDS of course thought of a way to avoid that people upgrade their hardware. After putting the card in, the machine only shows an error “104 - we want you to pay lots of money for an HP approved Wireless” and does not boot.

Searching the internet, I found out that I had two possible options: patching the BIOS or patching the ipw2200 card’s eeprom.
The second option is possible because there actually are ipw2200 cards that are “certified” by HP and they have different subvendor and subdevice IDs, so that the BIOS can detect them.

I went for a mix of all the procedures I found. The patch for ipw2200 I used is from Obi’s Homepage (thanks), also a large part of the procedure is described there, even though he had a different goal.
Which bytes I had to change was described, ironically, in the HP support forum (search for “ethtool” on the page).

So that’s what I did:

  • put the card into another notebook that would boot with it
  • build a patched ipw2200 (too bad upstream apparently does not want to take this patch)
  • boot an openSUSE 11.2 live ISO (from an USB stick, of course)
  • unload the original module, load the patched module (important)
  • issue the following commands:
ethtool -E eth0 magic 0x2200 offset 0x8 value 0xf6
ethtool -E eth0 magic 0x2200 offset 0x9 value 0x12
ethtool -E eth0 magic 0x2200 offset 0xa value 0x3c
ethtool -E eth0 magic 0x2200 offset 0xb value 0x10

Note that I did get errors from ethtool for every one of those commands (invalid address and others, pretty confusing ;) ). First I thought that it did not work, but “dmesg” told me that the write function was actually called, and so I just continued.

  • I unloaded and reloaded the patched ipw2200 module. I did expect an eeprom checksum error, but I actually did not get one. So I also did not do the “repair_eeprom=1″ routine that Obi describes.
  • Put back the module in the “target” notebook.

Now I have:

02:04.0 Network controller: Intel Corporation PRO/Wireless 2200BG
        Subsystem: Hewlett-Packard Company Compaq nw8240/nx8220

and so far it has been working fine.

Attention: according to the forum post in the HP forum, if your card is from the US and not from Europe, you will need

ethtool -E eth0 magic 0x2200 offset 0x8 value 0xf5

instead of the 0xf6 for the european version.

Oh - and if you are too lazy to patch and build your own ipw2200 module - there is the “ipw2200-eepromwrite-kmp” package for 11.2 in the buildservice, project home:seife ;)

eXtreme uPdating

Saturday, November 28th, 2009

I had that machine sitting in my living room which was once a media player, but is now (after I implemented VDR recording playback on my satellite STB :-) ) only used for playing back the recordings of my i4l-vbox answering machine. It was still running 10.3.
Now, having read that 10.3 was out of service, I decided to finally kill it.
So how to do that with style? Well, update it!

I removed the 10.3 repos and added the 11.0 repos. (Does anybody still remember how slow package management actually was back then? It was horrible!) I did not trust the old zypper to already having implemented a proper “zypper dup”, so I updated with the YaST FACTORY module.
At first, YaST FACTORY updater refused to do a 10.3=>11.0 update. Well, edit the SuSE-release file and voila - it thinks it’s a 11.0=>11.0 update.
Only a few Package conflicts (I had lots of stuff installed, including self-built RPMs and Packman packages) and thus half an hour waiting for the resolver, then I found out, that the packaging had switched from bzip2 to lzma at that time, so yast could not really install anything, but updating rpm manually behind YaST’s back solved that. About four hours later (this is a lowly Toshiba Tecra with P3-1000 and 256MB RAM), I had 11.0 on the machine.

To my big disappointment, everything was still working!
Ok, so remove the 11.0 repos, add the 11.1… and zypper dup you go. Now I did not even have to patch SuSE-release…
Again, a few hours later (zypper was faster on 11.0, but not as fast as today), I had an 11.1 on the machine. And still, everything was working well, there even was still KDE3 on it.

I had intended to kill the machine, so I was not satisfied.
Remove 11.1 repos, add 11.2, zypper dup….
Yes! Finally, the X server did not produce a picture on my LCD TV (which is pretty picky about the VGA input signal, but did work before) anymore. Strike! But it was easily fixed: I just removed xorg.conf and at least had a 800×600 KDE4 Desktop. Installing sax2 (it had become lost in some conflicts, which I generally solved by “remove everything”) and calling it fixed that up also.

What was actually a pity is, that no attempt had been made to import the KDE3 config (I had a few custom mime types etc, and now will have to reconfigure everything from scratch), but apart from that, the only thing I had to do was remove the kdm from KDE3 which was still installed and use the KDE4 kdm, so that logout and shutdown etc. properly work.

All in all it was pretty impressive, and I thought by myself “that’s like the stories the debian users always tell about upgrades”.

…and even though I wanted to, I will probably not get to do a fresh installation of 11.2, since all my updates went so well ;)

But what I found out in the end was, that KDE4 is really no longer usable on such a machine. Everything was very slow and sluggish, even with eye candy turned off as much as possible. But for the projected use of this box, XFCE will do just fine.

Morons

Saturday, October 10th, 2009

Holy cow.
After seeing how those boycottnovell *peep*s interpreted my last post, I’m pretty sure that they are also fans of those “the moon landing was staged in hollywood” conspiracy theories…

Boys, if you want to get your facts right:
It’s pretty normal for someone to get a new job after almost 9 years. Especially if the old job was 400km away from home and family and the new one is 9km away.
Of course I had already written that, but obviosly you are not good at reading.

Using PAN/NAP instead of rfcomm/ppd

Monday, August 10th, 2009

I finally got a new mobile phone - one that not only supports rfcomm (”serial port” via bluetooth) but also the “PAN Network Access Point” profile - which allows a computer to connect to the internet easily, without many extra layers like PPP etc.

It’s actually quite easy to use. (Note: I’m doing all that on recent openSUSE Factory, but I am pretty sure that it would work just as good on plain 11.1)

It is very handy to have the bluez-test package installed, many of the tools I am using are in that package.
First, we need to create a connection to the phone, exchange PIN numbers etc. I use bluetooth-applet from the gnome-bluetooth package (or the one from bluez-gnome) for that, it’s just the best maintained tool for the task, even if you are a KDE guy like I am, I’d highly recommend it.

Now we need to find out the phones Bluetooth-Address (bdaddr):

seife@stoetzler:~> test-discovery
[ 00:24:EF:xx:xx:xx ]
    Name = seife C510
    Paired = 1
    LegacyPairing = 1
    Alias = seife C510
    Address = 00:24:EF:xx:xx:xx
    RSSI = -54
    Class = 0x5a0204
    Icon = phone

The hex number after “Address” is the bdaddr of your device.

Now we can create a network device:

seife@stoetzler:~> test-network 00:24:EF:xx:xx:xx nap
Connected /org/bluez/16833/hci0/dev_00_24_EF_XX_XX_XX to 00:24:EF:xx:xx:xx
Press CTRL-C to disconnect

Now you should have a bnep0 network interface and can (as root) either run dhcpcd directly on it, or create a config file “ifcfg-bnep0″ in /etc/sysconfig/network like this:

BOOTPROTO='dhcp'
NAME='pand network device'
STARTMODE='auto'
USERCONTROL='no'

and then call “ifup-dhcp bnep0″.

Two things to consider:
First, if your system is using NetworkManager, you don’t get DNS addresses into resolv.conv. See this post for details. I now have

NETCONFIG_DNS_POLICY="STATIC_FALLBACK ppp0 bnep0 NetworkManager"

in my /etc/sysconfig/network/config

Second, test-network will terminate the connection after 1000 seconds (it is only a test tool), but it is easy to fix, just copy it to ~/bin/my-test-network (or whatever name you like) and apply the following trivial diff:

--- /usr/bin/test-network
+++ ~/bin/my-test-network
@@ -35,9 +35,10 @@
 print "Press CTRL-C to disconnect"

 try:
-       time.sleep(1000)
-       print "Terminating connection"
+       while 1:
+               time.sleep(1000)
 except:
        pass

+print "Terminating connection"
 network.Disconnect()

I’m not a python wizard, so I’m pretty sure you can do better, but it works well for me.

U-Boot hacking -or- The dbox2 can boot from disk!

Monday, July 6th, 2009

Thanks to a kind donator, I now own a Nokia dbox2 with IDE interface. (For those that don’t know - the dbox2 is a settop box designed for german paytv 10 years ago in a way that its bootloader would only run signed software. A limitation that was put aside by some cool hackers early in this century. Nowadays it usually runs Linux on its 66MHz powerpc CPU, 32MB RAM and 8MB FLASH ROM. Later some cool guys designed an IDE interface that plugs into the memory expansion slot)

The main use of such an IDE interface for most people is to use the box as a video recorder. However, I am already doing this via the ethernet port, wich is more practical for me than a built in disk.

I would have deemed it much more useful if I was able to boot from the IDE drive, since that would make it much easier to try out new experimental code without needing to upgrade the FLASH image all the time (having the known working version in FLASH as a fallback is mandatory for good WAF).

So I pondered the problem and found out, that people had already implemented basic support for the IDE interface in U-Boot (which is used as a “second stage” bootloader), and it could already access and read the disk.

Unfortunately, U-Boot needs to be in FLASH, and this U-Boot would then only boot from disk.

So I have been hacking on making U-Boot actually read multiple boot command lines (and and option which one to boot as default) from a config file in the /var/ partition of the FLASH which makes it pretty easy to switch the device we boot from. It is now actually similar to the boot process of the Tripledragon.

The only thing that’s missing now is a driver for the front panel keys, so that you can change the selection on the fly without using a serial console. That’s coming next, after the summer vacation ;)

Back from Linuxtag 2009

Wednesday, July 1st, 2009

I finally managed to upload my slides to the openSUSE Wiki’s Linuxtag 2009 page (I actually had nothing to do with openSUSE this year, but somebody had already listed me on that page and it’s actually pretty handy to host the slides there). I also uploaded it to the Linuxtag site, but after experiencing really abysmal organisation this year, I doubt that it will appear anywhere there.

The talk was not that exciting in my point of view - “Netbooks” are, in the end, a pretty boring topic to talk about - but the attending crowd seemed to like it, so apparently it was at least entertaining ;)

The event itself was better than what I had expected from the organisation flaws, I did talk to some embedded people and made many interesting contacts, e.g. with beagleboard guys, peple from the openembedded project and of course the cool hackers from coreboot.

So after a disappointing start, Linuxtag 2009 finally ended pretty successfully for me.

Improving XFS unlink() performance

Thursday, May 28th, 2009

I knew for quite some time that XFS has an abysmal performance on huge deletes - e.g. deleting a Linux kernel source tree can take several minutes. But when I had to do multiple images with kiwi yesterday on my relatively new server machine at home, it started to really annoy me, so I invested some time to find out if there is a way to improve the situation.
This is what I started from (numbers are from bonnie++):

/dev/sda1 /space3 xfs rw,noatime,nodiratime,noquota 0 0

Version 1.01d       ------Sequential Output------ --Sequential Input- --Random-
                    -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
Machine        Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP  /sec %CP
server           4G 69024  31 71616  10 32732   6 75462  39 77511   6 120.9   0
                    ------Sequential Create------ --------Random Create--------
                    -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
              files  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP
                 16   341   1 +++++ +++   311   1   340   2 +++++ +++   249   1

The first hint I got from a colleague was to use “logbufs=8″ mount option. This increases the number of buffers that XFS uses for its log. Whatever that means. At the first try, it did not change anything until I realized that a “mount -o remount,…” was not enough, I had to unmount and newly mount the filesystem with the option to make it work.
This is the result, still not too impressive with regard to delete performance, the difference might be purely statistical noise:

/dev/sda1 /space3 xfs rw,noatime,nodiratime,logbufs=8,noquota 0 0

bonnie++:
Version 1.01d       ------Sequential Output------ --Sequential Input- --Random-
                    -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
Machine        Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP  /sec %CP
server           4G 66413  31 68420  10 32467   5 76001  40 77143   6 107.2   0
                    ------Sequential Create------ --------Random Create--------
                    -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
              files  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP
                 16   338   2 +++++ +++   313   1   332   2 +++++ +++   269   1

But the really good results came only after also increasing the logbuffer size with the “logbsize=262144″ option:

/dev/sda1 /space3 xfs rw,noatime,nodiratime,logbufs=8,logbsize=256k,noquota 0 0

Version 1.01d       ------Sequential Output------ --Sequential Input- --Random-
                    -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
Machine        Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP  /sec %CP
server           4G 77932  33 85196  10 32134   6 59205  32 76792   6 184.5   0
                    ------Sequential Create------ --------Random Create--------
                    -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
              files  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP
                 16  1495   5 +++++ +++   848   2  1468   7 +++++ +++   916   3

Now that’s a real improvement and I’ll keep it like that.

But beware, there are some caveats:

  • The file system has to support the larger buffer sizes, which means you need a sufficiently new kernel. As I am running openSUSE 11.1 on this machine, this is not really a problem, but I had to enable the “version 2 log format” with “xfs_admin -j /dev/sda1″ because this partition was created using SLES10 and I am not sure if I still would be able to mount it on an old machine.
  • It will use more memory. I guess that it needs at least 2 MB (256kB * 8 buffers) per filesystem, probably more. But on a box with 2GB of RAM, I’ll gladly spare that for better performance.

One note: those “measurements” were taken on a Core2 Duo machine using an abit IP35-E mainboard. The used disk drive was my old Maxtor STM3500630A 500GB IDE drive. I did not do any statistical corrections etc., so your mileage may vary.

Cross compilers for openSUSE

Thursday, April 30th, 2009

Based on Torsten Duwe’s great work (currently in the Buildservice in home:duwe:crosstools), which is building but not working 100% correctly, I fixed the powerpc version up (I have no other architecture to test on) and put them into the openSUSE Buildservice in home:seife:cross for now.
Get them from here: http://download.opensuse.org/repositories/home://seife://cross/SLE_11
(don’t get put off by that SLE_11, the cross compilers should be pretty system agnostic. For testing I ran them on a SLES10 which is probably older than anything you want to use, and they worked just fine)

Edit: I just found out that this only works because I have a patched rpm on my SLES10 which understands lzma compression. I have added SLES10SP2 to the repository so that pre-LZMA-rpm distributions still can use it.

So if you need a powerpc crosscompiler, just add that repository, install packages “cross-powerpc-embed-linux-gnueabi-binutils”, “cross-powerpc-embed-linux-gnueabi-gcc”, “cross-powerpc-embed-linux-gnueabi-glibc” and “cross-powerpc-embed-linux-gnueabi-kernel-headers” and you are ready to go.
Make sure that /opt/cross/bin is in your $PATH and configure your project for target “powerpc-embed-linux-gnueabi”.

Still TODO: get them into a proper semi-official project.

Oh - and of course I already changed my favourite embedded projects so that they are now able to build with an “external” toolchain ;)