Arrfab’s Blog

CentOS 4.x machine not rebooting and faced with a grub prompt

One of my customer phoned me to say that one CentOS 4.x machine (acting as a apache reverse proxy) didn’t reboot after a power outage. The machine had two sata disks configured in raid 1 (through md/software raid) but instead of booting, the machine was just displaying a grub> prompt.

Of course i tried the traditional `grub-install –recheck /dev/sda` and `grub-install –recheck /dev/sdb` and also the manual procedure (already described here) to install grub on both devices .. but no luck .. still booting at the grub> prompt.

But then i looked (in rescue mode) at the (/mnt/sysimage)/etc/grub.conf and i counted 22 kernel entries in the file .. The customer had configured the nightly automatic yum update but he never cleaned the old kernels (both up and smp) … so i “cleaned up” the grub.conf file, once again installed grub with grub-install and …. machine rebooted normally ..

I’ve never thought that too many entries in the grub.conf file could block the machine from booting … Maybe that will save other people time

Entering CentOS 5.2 QA mode …

Yes, it started .. the CentOS QA-Team entered the 5.2 QA era .. meaning that we have to test a bunch of existing features and newer ones included in 5.2. For example, in the upstream announce mail i saw that the newer libvirt has support remote connections. So i decided to give it a try just after i updated my CentOS 5.1 x86_64 dom0 to 5.2QA (and my domU i386 and x86_64) … but when i tried to connect, i received a ‘connection reset by peer’ (i tested with only ssh and not tls/certs) … so i decided to read a little bit on the libvirt.org website and found which parameters should have been configured in the /etc/libvirt/libvirtd.conf (full list available here) . The only ‘problem’ so far is that the /etc/libvirt/libvirtd.conf is not provided by libvirt itself and doesn’t exist ! .. Strange because it’s referenced in the /etc/sysconfig/libvirtd (that you have to modify too) file .. So it seems you have to create it yourself , and then i was able to connect remotely (i tested only with ssh .. and important : don’t forget that you need ssh key-based auth for this …)

More informations about the QA tests later (and by other people/QA testers too … )

IBM Director 5.20.2 agent setup on CentOS/EL

I’m used to deploy IBM Director server/agents on IBM hardware to monitor hardware/services .. and surely due to the fact that i work for an IBM business partner and that i give myself the IBM director course for IBM …

But there is something really anoying : each time you receive a IBM director cd/iso image (like the 5.20.2 that you can download from the IBM support website), it should normally contains the Linux level 2 agent for each of the supported Linux distributions (aka RHEL 3,4,5 , SLES 9,10 and Vmware esx). You can even integrate such agent in the director console to push it to remote machine (in fact it will do it through ssh … so be careful if you tuned sshd to accept only specific user/key-based auth …)

But last time i had to deploy it on CentOS machines (usually a simple change in the /etc/redhat-release file is enough ) i did it from the director console … Task was marked as successfully but nothing was installed .. (how the hell could director answer me that it was successfull if it was not the case ?) . Okay, let’s do it manually then … but then i saw that the level2 agent located on the CD (director/agent/linux/i386/FILES/dir5.20.2_agent_linux.sh -x) contained only the RHEL3 and SLES10 RPMS inside ! WTF ?

You can download the full Director Linux agent 2 package on the IBM website and that one will contain all the required RPMS …

Red Hat EMEA Partner summit event - part 2

Red Hat partner summit is over and i really enjoyed it for both the technical labs/presentations and the nice discussions i had with Red Hat employees (for example i really appreciated Boris Devouge’s talks). One thing that was announced is the upcoming release of Paravirt drivers for Windows DomU. (probably they will be released somewhere between 5.2 and 5.3). I’ve seen them in action during a lab organized by Olivier Reneault and it’s funny to see that Windows device manager reports them as ‘RHEL scsi driver disk’ and ‘RHEL PV nic driver’. It seems the goal (as usual with Red Hat, in opposite with what Novell is always doing regarding this …) is to release them under the GPL. In fact, my discussion with Olivier learned me that they were/are developed in collaboration with Hitachi.

Other thing that i learned is that PV drivers/modules for EL3 are on the way too (you’ll never have a xen kernel for el3 because of its 2.4 kernel …) so that you’ll have better performances too.

During some presentations and labs it was mentionned also that RHN/Satellite technology will also be released as open-source/gpl but the main stopping problem is that actually both products use Oracle as a backend, and that explains also the prices for such products. I explained to them that what i do for some customers who want to save bandwidth without having to pay for Satellite is that i use reposync (from the yum-utils package) to mirror the rhn channels on a local machine .. and i was astonished that some RH tech people didn’t know that it was included in the base EL5 …

Last but not least is the fact that the Partner portal changed a bit several weeks ago and i decided to update the profile. When you do it you’re asked several questions including ‘Which products do you actually support ?’ and in the list, below RHEL, SLES and MS windows i saw CentOS …

Red Hat EMEA Partner summit event - part 1

I have actually the chance to assist to the Red Hat Emea partner summit event in Malaga (Spain) and i had the opportunity to listen to Jim Whitehurst, the new Red Hat ceo .. he’s really pleasant to listen to.

We (Dag Wieers and myself) had the oppurtunity to talk to Scott Creenshaw, the Red Hat vice president, about CentOS .. but i’ll come back probably later on that … One thing he announced during his presentation was Ovirt.org , which is an http-based Virtual Machine management system. This was produced by the Red Hat emerging techonologies group, so basically by the same people that bring koan and cobbler to live. I’m now interested in testing it and see how it can compete against other http-based systems like openqrm .. while on the other hand openqrm is not limited to vm deployment and provisioning …

Naissance de fr.centos.org

(For non native french speakers : that will be my only announce here in another language than english )

Le projet CentOS est heureux de vous annoncer la naissance du site http://fr.centos.org .
En réponse à la demande croissante de la communauté des utilisateurs francophones de CentOS, le forum fr.centos.org a vu le jour.
Nous profitons de cette annonce pour relancer l’appel aux volontaires pour traduire le wiki existant (http://wiki.centos.org)
Pour se faire, il suffit de vous inscrire dans un premier temps à la liste de diffusion centos-docs (sur http://lists.centos.org) et de vous créer un identifiant/login sur le wiki.
Demandez ensuite l’autorisation d’éditer les pages en dessous de http://wiki.centos.org/fr …

Nous tenons tout particulièrement à remercier Guillaume Kulawoski qui est à la base de l’idée et la mise en place du forum , ainsi que Thierry Delmonte pour la conception graphique.

A bientôt sur fr.centos.org !

Vmware server guest VMs on top of ocfs2

While i was testing ocfs2 on CentOS 5.1, one colleague of mine asked me if that was possible to have VMware server on top of ocfs2 to test a move from one node to the other node. Of course my first reaction was that vmware-server can’t do live migration like esx/vmware infrastructure can .. but because the machines were ready and that it’s fast to setup , we did the test.

The first vm refused to start on top of ocfs2 , while the same vm started on local storage. Google pointed me to the correct answer in 3 seconds : you need to include a special parameter in the vmx (vmware guest config file) to have it working on top of ocfs2 . The line to be included is “mainmem.usenamedfile=”FALSE” “. You can have more informations on the Vmware forum regarding this.

We then were able to quickly move (by suspending a vm on node1 and resuming it directly on node2) a VM between the two physical machines. Of course that’s not live migration, but that’s very close to … and my colleague was happy

scsi-target-utils/iscsi tgtadm not production ready on el5.1 ?

When CentOS 5.1 was announced, the upstream release notes contained some notes about new features being integrated in 5.1, like iscsi-target functionnality. Of course they were announced in the “Technology Previews” section, meaning that it’s not fully supported and not considered production ready. But most of the time, packages ‘just work’ [TM].

Is this the case for the package scsi-target-utils-0.0-0.20070620snap.el5 ? hmmm …. On my (already too long) TODO list, I planned to test Ocfs2 on top of a shared device , and because of a lack of Fiber Channel HBAs in my lab, the only solution was to play with iScsi target/iscsi initiator on both machines (3 machines : 1 as a iscsi target and the 2 others as initiator/ocfs2 machines). I already tested the standard IET iscsi target daemon in the past and i was expecting to find almost the same behavior .. but it’s not.

In fact, there is *NO* configuration files included with tgtadm so you have to type all your tgtadm commands to create the iscsi target LUNs and share them . The tgtadm tool isn’t a big deal and it’s even good to add new target on the fly … but because of the lack of config files, you can’t save your actual config and hope to restore it at the next boot … So you’d better have to save your tgtadm commands in a script and call that bash script from within a new initscript … I now understand why the release notes consider that it’s not *production ready* yet … so let’s see what will be included/modified in 5.3 …

Fosdem 2008 review - part 2

Fosdem is over .. and it was a good edition. From a CentOS perspective, i’d say that the booth was better than in 2007 and because we were not at the same place, i think more people had a look at the booth … I was at the booth and/or in the devroom for all the CentOS talks (i’ve posted some pictures) but i had opportunity to at least follow some other talks :

• Wouter Verhelst’s talk about Belgian eid on Debian (that reminds me of course my 2007 talk …)
• Alasdair Kergon’s talk about new features in LVM2
• Jens Kühnel’s talk about SELinux (Jens is really a good guy to talk to … always a pleasure to see/talk with him)

Let’s go back to normal life … and try to prepare next year ..

Fosdem 2008 review - part 1

What can i say ? hmm … I’ve posted some pics online here so that you can already feel the Fosdem event mood …

I gave the ‘Introduction to CentOS’ yesterday and it’s scheduled today too … (30 minutes to go ….)

I can already say that i enjoy the Fosdem event .. cool organizers, nice place to be (but am i objective, being a belgian guy myself ? ) and of course it’s always good to see people you’re chatting with or sending mail to .. but in the real life …

We share the devroom with Fedora and our booth is next to them … and , to quote what they say on their wiki about us, “they are nice people” ..

More informations/pictures to come ..