Developers’ Weblog

Sponsored by
HostEurope Logo

Developers’ Weblog

All 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31

GRUB sucks! • More snapshot fallout.

01.11.2009 by tg@
Tags: bug rant snapshot

The GRUB2 「memtest86+」 bootmenu option in both 「wtf ist hallowe’en」 and Grml “Hello-Wien” does not work as-is (note that Grml uses ISOLINUX, unless you either select GRUB2 from the boot menu or dd(1) it onto a USB stick, CF/SD card, etc. for manifold-boot) because nobody told the Grml team that it must now be booted with 「linux16」 ipv 「linux」 – fix is to type ‘e’ to edit the entry, move right, type the “16” and hit ^X to boot. Just great… I’ve updated the article accordingly.

We are on Sümlink. Both of us. Sweet. Remember that this could’ve been MirOS #11-RELEASE, and should be treated by everyone except us as such ☺

On the bonus side, I’ve tested the netboot.me 17001 boot, as well as the ISO (both file – in qemu – and CD-R on real hardware) on a SPARCstation 20, Setup (i386) on a VIA C7, grml and Live (i386) on an IBM X40 and in qemu… so I’d say it works. Oh, and memtest86+ on X40.

Netboot instructions: boot, e.g. via “qemu -m 256 -fda netbootme.dsk”, do not hit Ctrl-B, but hit the Anykey when it asks, “boot a configuration directly”, type 17001 and hit Enter; wait for the “boot> ” boot(8/i386) prompt, type “b bsd.rd” and hit Return. Voilà! (Or, select MirOS bsd4me, which loads a memdisk-ISO, either via the number 2038 or via “Live OSes” → BSD → “MirOS bsd4me current” and just press Enter on the prompt. The gPXE image can be dd(1)d onto a floppy disc.

Benny finally made ports/print/ttftot42 – thanks!

Bittorrent trackers, …

31.10.2009 by tg@
Tags: rant

TPB has never been a viable tracker for us… h33t lost my login data… I hope Scarywater still holds up, we’ll be adding Openbittorrent to the mix… oh yikes, Demonoid is down. How great… NOT!

At least I finally managed to release things in time, and during the night, both ISO and CVS tarball, forming the torrent, will be available to the general public. I also already know of some interested parties, from remote areas like Romania and Mauritius.

The explanation is on the front page, of course. This link is for the benefit of RSS or aggregator (like Plänet Sümlink) readers.

The files will take some time until they are up. I do have an ADSL internet connection only and they’re huge.

The GNU Project is famous for its coding style – Linus Torvalds even suggests to print them out, but not read them; burning them is a great symbolic gesture. Legibility issues aside (Linus’ own are interestingly similar to style(9) aka KNF, the BSD coding styles), why is that so?

mksh-current (R40+) now supports pathnames in arbitrary lengths on Debian GNU/Hurd (I think; I only could test on gnubber.bddebian.org that it compiles, seeing that all existing installations set sysconf(_PC_PATH_MAX) to 1024 to be consistent with POSIX PATH_MAX) by using some glibc-only functions. This is because Hurd does not have PATH_MAX (some older systems also don’t have it, but there, we just define it to POSIX 1024 and good is). Now, why? Simple: because the GNU coding style says to have “no arbitrary limits” in your code.

I would like to call that ridiculous, but it’s actually dangerous: if you are on a 32-bit machine and have a pathname of 512 MiB, you’re in danger of freezing your system or at least crashing mksh, even if you have the full 4 GiB worth of RAM, due to the amount of copying and carrying around pathnames. This is a security relevant issue, in my eyes.
Now on to the ridicule part: This is Open Source, people! Change the limit (as it’s a – only one – constant in a header file) and recompile everything! Simple as that! The BSDs do it all the time! In GNU, it’s even simpler because you force developers, redistributors and some users to actually give up freedom and require them to put the source code alongside. Now, why doesn’t anybody see this? I can’t be the only one, can I? (I actually think that changeable limits would suffice the horrid GNU coding style, but find myself reluctant to read it again due to its sheer size – similar to their licences…)

On an unrelated note, I hope to have a bi-arch ISO format snapshot on BitTorrent by end of the month. Mika is also trying to put a new Grml release out by then, of course with an up-to-date MirOS bsd4grml

mksh’s Build.sh can now generate Makefrag.inc snippets that reflect its environment and can be used like Rebuild.sh or integrated in, for example, the Android.mk files.

mksh(1) also is a great compiler testsuite: it regularily(!) points out bugs in gcc’s -fwhole-program --combine and helped to find (and fix) bugs in huge things like SunStudio, old things made modern like pcc, as well as one-man projects such as nwcc
I wish compiler authors would just use mksh as testsuite regularily.

I released a new version of jupp for Unix® today: joe-3.1jupp12 (including a backported hex editing mode, as a late birthday present for waga (from IRC) who asked for it, a plethora of bug fixes, raw octet support in UTF-8 mode, UTF-8 support in the 8-bit modes, and more changes. If you already use it, update please. If not, give it a try! It’s included in at least FreeWRT already, and I build Debian packages (link on the jupp website) in my “play repository”, as well as packages for Univention Corporate Server at work (from these packages). I’m also using it on Solaris 8, Solaris 10 and AIX 5.3 ☻ (and many more)

There are currently quite a lot more things I would like to do but can’t seem to find the time for. I had originally planned a new MirBSD snapshot, including ISO, for mid→end of September – now, end of October seems more likely (if I can get a few days off work and some RAM for the SPARCstation 5). Several things, like the mksh prompt wlog entry and webpages for a few subprojects (BSD::arc4random; arc4random.c; the RANDEX protocol, plugins, implementations, proxies; kwalletcli; keystash) are lacking too, and the Grml project is also expecting code from me. Sorry. I am, after all, human too…

As sent to the mailing list I had a chance to compile mksh for Android today, thanks to Waldemar confirming that it works (with OpenADK's eglibc linked in statically) and CeKay's help in mastering the SDK and NDK. The posting contains all relevant information; it is virtually impossible to write an Android.mk file though.

On the other hand, I did submit a patch for Bionic (the libc) to have sys_signame[] like real BSDs, so that we could simply hard-code the appropriate CFLAGS and CPPFLAGS and let the NDK compile the mksh source *.c files (signames.inc would no longer be needed).

Maybe we'll submit mksh for inclusion somewhere, as this would be the first really usable shell. For this reason, I have uploaded a binary (gunzip(1) first) at ports/mksh-39.9.20090929-android15.tgz (MD5 64ee103453d65e947f2beb1aeb6450d2) which you can place in, for instance, /data/mksh then put a (modified, as the ls(1) and id(1) and possibly other utilities are more than reduced) dot.mkshrc from the source (CVSweb) as /data/.mkshrc and run it from adb (which, by the way, is the one responsible for some cursor keys etc. not working, as Android and mksh(1) do both fine in that regard) with # HOME=/data /data/mksh in the shell. I may post an Android specific patch for dot.mkshrc some day.

Update 10.10. – .gz → .tgz and Cygwin binary now too!

I tried to improve the Font Embedding as well as the CSS on this website again. The fonts actually used should be Gentium Regular, Gentium Italic, Gentium Basic Bold, Gentium Basic Bold Italic. If you have only the Book variants installed, they are fallback. Both CSS3 and IE/EOT embedding styles should work. (Bolding seems to be broken in my Opera-Linux 9.27 though…)

  • b bold
  • strong bold
  • i italic
  • em italic
  • bold italic

Please do report any bugs (possibly with patches) again ☺ Things still look great in Lynx, so I̲ am content in any case…

Getting closer: new snapshots soon

13.09.2009 by tg@
Tags: snapshot

I'm closer to getting a new bi-arch snapshot ISO done: i386 is in a pretty shape, and the sparc looks good, will be compiling this night, and a few subsequent ones... so both architectures are on the same level. I'll need to do clean installation builds for the snapshots of course. There will probably be a ISO like the Easter 2009 snapshot on BitTorrent again (no TPB, mind you), including the (admittedly slightly dusty) MirGRML.

Since we have over 1 GiB worth of binary packages, mostly compiled by bsiegert@, some of which depend on libgcc_s.so.1.1, while the snapshots will have libgcc_s.so.2.0, fixes10.ngz will contain a binary plug of the older one (built just before the ABI change) for convenience in using the binary packages. Be advised to install it.

mksh(1) is currently being actively developed; the snapshot may come before mksh R40 will be released, but contain a lot of changes from R39 (see the changelog).

I feel like I'm going ill again, bed day for me. This sucks. And I even don't know why.

smultron, MidnightBSD developer and project member, and graphician / art master for the MidnightBSD Korn Shell (among other things), has pointed me to a blog article: 8 Useful and Interesting Bash Prompts and asked for an mksh(1) translation. These will not be minimalistic (I can do quite some things), but focus on containing no control characters or other things not easily cut-and-past'able. Made more readable, too.

Small note: all mksh examples assume the following "setup code" in front of the PS1=... line. Yes, PS1 is set twice (makes the code more readable and perform better). Most of this is from the mksh(1) manpage or dot.mkshrc and relatively portable, which is why some things seem more complicated than needed. They do, however, use some rather recent mksh features; if there's real interest in making it work with, say, Debian etch mksh R28, eMail me. Note: that → character is a literal Tab (the "[→·]" Tab-Space occurs often).

	ca=$(print -n '\001'); ce=$(print -n '\033'); cm=$(print -n '\r')
	: ${HOSTNAME:=$(ulimit -c 0; hostname -s 2>&-)}
	[[ $HOSTNAME = @(localhost|*([→·])) ]] && \
	    HOSTNAME=$(ulimit -c 0; hostname 2>&-)
	: ${USER:=$(ulimit -c 0; id -un 2>&- || print \?)}
	if (( USER_ID )); then PS1='$'; else PS1='#'; fi

	function twd {
		typeset d=${PWD:-?} p=~

		[[ $p = ?(*/) ]] || d=${d/#$p/~}
		print -nr -- "$d"
	}

If you do not need exactly the effect of "\w" (tilde unexpansion), use $PWD ipv $(twd) below. Note that $USER and $HOSTNAME are usually evaluated only once; escape them if you need it evaluated every time.

  1. Show Happy face upon successful execution
    	bash$ PS1="\`if [ \$? = 0 ]; then
    		echo \[\e[33m\]^_^\[\e[0m\]
    	else
    		echo \[\e[31m\]O_O\[\e[0m\]
    	fi\`[\u@\h:\w]\$ "
    
    	mksh$ PS1="$ca$cm"'$(if (( $? )); then
    		print "'"$ca$ce[31m$caO_O$ca$ce[0m$ca"'"
    	else
    		print "'"$ca$ce[33m$ca^_^$ca$ce[0m$ca"'"
    	fi)'"[$USER@$HOSTNAME:\$(twd)]$PS1 "
    

    I do think Unicode makes things more fun though:
    	mksh$ PS1="$ca$cm$ca$ce["'$(if (( $? )); then
    		print "'"31m$ca☹"'"
    	else
    		print "'"33m$ca☺"'"
    	fi)'"$ca$ce[0m$ca[$USER@$HOSTNAME:\$(twd)]$PS1 "
    
  2. Change color on bad command
    Coming soon... colour changes are okay, but I wonder why anyone would want the history number. I've got to discuss this a little, and think about ways to use such information with mksh, and if I really want to transcribe the prompts 1:1 (it's possible by just incrementing a counter, but, hey, you don't want that). This is actually a csh feature, and who'd really use csh? I'm on my 80486DLC notebook right now, so my debugging methods are a little limited, but I promise a larger article later. I just needed to get started.

This article will be updated in-place.

Due to heavy load at work, as well as some minor things, I'm either taking back interest altogether, involvement altogether, time spent on projects, or any of these on aspects/particulars of projects.

Sometimes, when you're burnt out, it's best to concentrate on living and on core projects. mksh is one of these for me, as is keeping MirBSD userland and MirPorts infrastructure working well, with small, evolving improvements (no big jumps). Other things, no matter how nice, interesting or useful (to me as well as to others) they are, need to stay back. I poured most of the last seven years of my life into MirBSD.

Sometimes, you want to give back, but it's too much effort, or you cannot afford to spend more time on it. I'll close one of my Debian ITP bugs for this reason. (I also rarely send in patches from ports for this reason, but sometimes point upstream to our CVSweb.)

Sometimes, people like Ulrich Drepper, Marco d'Itri, Gerrit Pape let you realise that every project has its Theo de Raadt-alikes. I've still not ported jupp's latest release to Debian (but an OpenSuSE Buildservice SRPM exists), nor uploaded the current mksh(1) version even to my own wtf repo. I will do so, when I feel like spending private time with Debian again, at least for the etch and lenny (and hardy – for work) branches, as dash and mksh in sid have... issues I predicted ages ago. (For one, I'm still waiting for Gerrit to contact me. Maybe our eMail systems don't like each other? Waldi or formorer will probably pass on any messages, as will the trusty BTS.) I'll probably not open any ITP bugs again and send in much less of the bugs I notice, simply because I don't like being ignored (or flamed, but sometimes, being ignored is worse – which is why Benny works on MirPorts, btw). Maybe, if I feel the need to, my wtf repo will grow instead; DDs or DMs are free to take from there if they like.

Sometimes, one realises that he just doesn't fit in. While Cachewolf is a useful project, working together with Java™ developers that communicate over web fora only and don't even see the need for compatibility with Unix or proper processes most of the time (svn:eol-style comes to mind, and switching the source code to UTF-8 is something I've given up to dream of – I would even have fixed bugs where Ewe wouldn't do UTF-8 right, but I run into a wall of bliss ignorance there) proved impossible for me. I won't budge either: web fora are simply not for me to use. Period. This is my fault (for not fitting in) as well as the fault of some of the rest of the team (for ignoring years of experience, or for simply nicht über den eigenen Tellerrand schauen (however one says this in English, I don't know) and not caring of these who do; for supporting the commercial gc.com site over the three alternatives too). I will continue to use it, maybe the iPAQ H3600 a colleague gave me proves useful, otherwise, MirBSD will do just fine.

So, when I leave or pull back a little, no prejudices. Sometimes with reason, but mostly due to lack of available resources on my part. I hope nobody who has been or will be noticing me ceasing to contribute as much as usual thinks ill that's why.

mksh feature weeks

07.09.2009 by tg@
Tags: mksh

It's feature weeks for mksh(1). You almost certainly want to follow the mksh-current changelog. Please test all the new features, as well as make sure that nothing breaks nor I introduced other regressions, such as making resource usage worse (disc, CPU, memory). Please be reminded that many more deep-reaching changes will follow in the next few hacking days.

Thanks to David Korn (the Korn in AT&T Korn Shell, ksh88/ksh93) for discussing features and helping with finding syntax and semantic of some of the more complicated language pecularities.

On an unrelated note, cvs(1), lynx(1), ncurses(3), some other things, and a lot of ports have also been worked on. The next snapshot is planned for mid to end of September, bi-arch. Stay tuned!

MirOS bsd4me – Sane OS on netboot.me ☺

30.08.2009 by tg@
Tags: news snapshot

Christian Hofstaedtler «ch:#grml» pointed me to an article about netboot.me, a service providing bootable Live and Install variants of some OSes over the Internet (via gPXE and HTTP transport, mostly). They offer a bootable floppy/disc, USB stick, and El Torito ISO image. One gets a graphical menu if not careful, out of which there is no escape, and which isn’t compatible with most graphics cards, but if ^B is pressed quickly enough, there will be a usable command line.

An experimental MirOS bsd4me bootable image is available as well:

gPXE> autoboot
gPXE> chain http://netboot.me/2032

This will be loaded via plain, unencrypted HTTP from our main webserver. It’s the usual <5 MiB Mini-ISO for the i386 architecture and contains a full installer, as well as e3 and tinyirc known from MirOS bsd4grml. Source code of the GPL’d parts and overall licencing information is also included inside the ISO image. Please remember this is for beta testing only. We do not support MEMDISK boot methods with our second stage boot(8/i386) loader, as these are not reliable.

On an unrelated side note: dutch Windows XP Professional is weird. Translations were laughed at by several dutch-speaking people in the mksh IRC channel, but I was able to disable Blåtand on “blau” (the new IBM X40, successor of “bleu”). And it works!

A more related side note: when virtualising MirBSD, do not use VirtualBox. If you do have Vanderpool/Pacifica available and enabled, both the OSE and the full edition may be viable, but kvm is the better option. If VT-x/HVM is not used, it does not emulate the i386 architecture properly. Use qemu, kqemu, kvm, bochs, VMware Server 2 (not 1 – timekeeping is broken in there), Microsoft® VirtualPC 2004, Parallels Desktop, VMware Fusion for Macintosh, … instead.

Tonight and the last few days have seen several releases of things like MirMake, the package tools, etc. You might want to upgrade your MirPorts Framework checkout. (We now have enough Lua tools to access the Simtec eKey, I just need to order one. We’re currently at 2¾ interested people, not quite enough to make sense.)

Hot!

20.08.2009 by tg@
Tags: rant snapshot

It’s hot! The weather forecast said 35°C, one of the apothecaries’ thermometres says 40°C, and my body says I’m ill. I also nurse quite some headaches again. So I won’t do/write much. It was hard enough to go outside to buy breakfast.

On the other hand, the SS5 built a mock-snapshot (QUICK_DIST2, so only base+gcc, no X11/ports/htman/…) just fine (with some fixes). Does anybody have any RAM suitable for a SPARCstation 5 for me? I cannot use that from the SS20s because they use different kinds of RAM. I won’t publish this, but instead use it to build another one for real, possibly combined with another i386 snapshot, with all the /etc/* changes after 20090812, and maybe fixing what we will have learned to be broken in the meantime. And Benny’s ports fixes (he got GNOME except nautilus – which has unsafe code in signal handlers – working, part of (or all?) Xfce too, etc).

Luckily, tomorrow is more like 22°C

All 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31

MirOS Logo