Developers’ Weblog

Sponsored by
HostEurope Logo

Developers’ Weblog

All 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31

The GNU Project is famous for its coding style – Linus Torvalds even suggests to print them out, but not read them; burning them is a great symbolic gesture. Legibility issues aside (Linus’ own are interestingly similar to style(9) aka KNF, the BSD coding styles), why is that so?

mksh-current (R40+) now supports pathnames in arbitrary lengths on Debian GNU/Hurd (I think; I only could test on gnubber.bddebian.org that it compiles, seeing that all existing installations set sysconf(_PC_PATH_MAX) to 1024 to be consistent with POSIX PATH_MAX) by using some glibc-only functions. This is because Hurd does not have PATH_MAX (some older systems also don’t have it, but there, we just define it to POSIX 1024 and good is). Now, why? Simple: because the GNU coding style says to have “no arbitrary limits” in your code.

I would like to call that ridiculous, but it’s actually dangerous: if you are on a 32-bit machine and have a pathname of 512 MiB, you’re in danger of freezing your system or at least crashing mksh, even if you have the full 4 GiB worth of RAM, due to the amount of copying and carrying around pathnames. This is a security relevant issue, in my eyes.
Now on to the ridicule part: This is Open Source, people! Change the limit (as it’s a – only one – constant in a header file) and recompile everything! Simple as that! The BSDs do it all the time! In GNU, it’s even simpler because you force developers, redistributors and some users to actually give up freedom and require them to put the source code alongside. Now, why doesn’t anybody see this? I can’t be the only one, can I? (I actually think that changeable limits would suffice the horrid GNU coding style, but find myself reluctant to read it again due to its sheer size – similar to their licences…)

On an unrelated note, I hope to have a bi-arch ISO format snapshot on BitTorrent by end of the month. Mika is also trying to put a new Grml release out by then, of course with an up-to-date MirOS bsd4grml

mksh’s Build.sh can now generate Makefrag.inc snippets that reflect its environment and can be used like Rebuild.sh or integrated in, for example, the Android.mk files.

mksh(1) also is a great compiler testsuite: it regularily(!) points out bugs in gcc’s -fwhole-program --combine and helped to find (and fix) bugs in huge things like SunStudio, old things made modern like pcc, as well as one-man projects such as nwcc
I wish compiler authors would just use mksh as testsuite regularily.

I released a new version of jupp for Unix® today: joe-3.1jupp12 (including a backported hex editing mode, as a late birthday present for waga (from IRC) who asked for it, a plethora of bug fixes, raw octet support in UTF-8 mode, UTF-8 support in the 8-bit modes, and more changes. If you already use it, update please. If not, give it a try! It’s included in at least FreeWRT already, and I build Debian packages (link on the jupp website) in my “play repository”, as well as packages for Univention Corporate Server at work (from these packages). I’m also using it on Solaris 8, Solaris 10 and AIX 5.3 ☻ (and many more)

There are currently quite a lot more things I would like to do but can’t seem to find the time for. I had originally planned a new MirBSD snapshot, including ISO, for mid→end of September – now, end of October seems more likely (if I can get a few days off work and some RAM for the SPARCstation 5). Several things, like the mksh prompt wlog entry and webpages for a few subprojects (BSD::arc4random; arc4random.c; the RANDEX protocol, plugins, implementations, proxies; kwalletcli; keystash) are lacking too, and the Grml project is also expecting code from me. Sorry. I am, after all, human too…

As sent to the mailing list I had a chance to compile mksh for Android today, thanks to Waldemar confirming that it works (with OpenADK's eglibc linked in statically) and CeKay's help in mastering the SDK and NDK. The posting contains all relevant information; it is virtually impossible to write an Android.mk file though.

On the other hand, I did submit a patch for Bionic (the libc) to have sys_signame[] like real BSDs, so that we could simply hard-code the appropriate CFLAGS and CPPFLAGS and let the NDK compile the mksh source *.c files (signames.inc would no longer be needed).

Maybe we'll submit mksh for inclusion somewhere, as this would be the first really usable shell. For this reason, I have uploaded a binary (gunzip(1) first) at ports/mksh-39.9.20090929-android15.tgz (MD5 64ee103453d65e947f2beb1aeb6450d2) which you can place in, for instance, /data/mksh then put a (modified, as the ls(1) and id(1) and possibly other utilities are more than reduced) dot.mkshrc from the source (CVSweb) as /data/.mkshrc and run it from adb (which, by the way, is the one responsible for some cursor keys etc. not working, as Android and mksh(1) do both fine in that regard) with # HOME=/data /data/mksh in the shell. I may post an Android specific patch for dot.mkshrc some day.

Update 10.10. – .gz → .tgz and Cygwin binary now too!

I tried to improve the Font Embedding as well as the CSS on this website again. The fonts actually used should be Gentium Regular, Gentium Italic, Gentium Basic Bold, Gentium Basic Bold Italic. If you have only the Book variants installed, they are fallback. Both CSS3 and IE/EOT embedding styles should work. (Bolding seems to be broken in my Opera-Linux 9.27 though…)

  • b bold
  • strong bold
  • i italic
  • em italic
  • bold italic

Please do report any bugs (possibly with patches) again ☺ Things still look great in Lynx, so I̲ am content in any case…

Getting closer: new snapshots soon

13.09.2009 by tg@
Tags: snapshot

I'm closer to getting a new bi-arch snapshot ISO done: i386 is in a pretty shape, and the sparc looks good, will be compiling this night, and a few subsequent ones... so both architectures are on the same level. I'll need to do clean installation builds for the snapshots of course. There will probably be a ISO like the Easter 2009 snapshot on BitTorrent again (no TPB, mind you), including the (admittedly slightly dusty) MirGRML.

Since we have over 1 GiB worth of binary packages, mostly compiled by bsiegert@, some of which depend on libgcc_s.so.1.1, while the snapshots will have libgcc_s.so.2.0, fixes10.ngz will contain a binary plug of the older one (built just before the ABI change) for convenience in using the binary packages. Be advised to install it.

mksh(1) is currently being actively developed; the snapshot may come before mksh R40 will be released, but contain a lot of changes from R39 (see the changelog).

I feel like I'm going ill again, bed day for me. This sucks. And I even don't know why.

smultron, MidnightBSD developer and project member, and graphician / art master for the MidnightBSD Korn Shell (among other things), has pointed me to a blog article: 8 Useful and Interesting Bash Prompts and asked for an mksh(1) translation. These will not be minimalistic (I can do quite some things), but focus on containing no control characters or other things not easily cut-and-past'able. Made more readable, too.

Small note: all mksh examples assume the following "setup code" in front of the PS1=... line. Yes, PS1 is set twice (makes the code more readable and perform better). Most of this is from the mksh(1) manpage or dot.mkshrc and relatively portable, which is why some things seem more complicated than needed. They do, however, use some rather recent mksh features; if there's real interest in making it work with, say, Debian etch mksh R28, eMail me. Note: that → character is a literal Tab (the "[→·]" Tab-Space occurs often).

	ca=$(print -n '\001'); ce=$(print -n '\033'); cm=$(print -n '\r')
	: ${HOSTNAME:=$(ulimit -c 0; hostname -s 2>&-)}
	[[ $HOSTNAME = @(localhost|*([→·])) ]] && \
	    HOSTNAME=$(ulimit -c 0; hostname 2>&-)
	: ${USER:=$(ulimit -c 0; id -un 2>&- || print \?)}
	if (( USER_ID )); then PS1='$'; else PS1='#'; fi

	function twd {
		typeset d=${PWD:-?} p=~

		[[ $p = ?(*/) ]] || d=${d/#$p/~}
		print -nr -- "$d"
	}

If you do not need exactly the effect of "\w" (tilde unexpansion), use $PWD ipv $(twd) below. Note that $USER and $HOSTNAME are usually evaluated only once; escape them if you need it evaluated every time.

  1. Show Happy face upon successful execution
    	bash$ PS1="\`if [ \$? = 0 ]; then
    		echo \[\e[33m\]^_^\[\e[0m\]
    	else
    		echo \[\e[31m\]O_O\[\e[0m\]
    	fi\`[\u@\h:\w]\$ "
    
    	mksh$ PS1="$ca$cm"'$(if (( $? )); then
    		print "'"$ca$ce[31m$caO_O$ca$ce[0m$ca"'"
    	else
    		print "'"$ca$ce[33m$ca^_^$ca$ce[0m$ca"'"
    	fi)'"[$USER@$HOSTNAME:\$(twd)]$PS1 "
    

    I do think Unicode makes things more fun though:
    	mksh$ PS1="$ca$cm$ca$ce["'$(if (( $? )); then
    		print "'"31m$ca☹"'"
    	else
    		print "'"33m$ca☺"'"
    	fi)'"$ca$ce[0m$ca[$USER@$HOSTNAME:\$(twd)]$PS1 "
    
  2. Change color on bad command
    Coming soon... colour changes are okay, but I wonder why anyone would want the history number. I've got to discuss this a little, and think about ways to use such information with mksh, and if I really want to transcribe the prompts 1:1 (it's possible by just incrementing a counter, but, hey, you don't want that). This is actually a csh feature, and who'd really use csh? I'm on my 80486DLC notebook right now, so my debugging methods are a little limited, but I promise a larger article later. I just needed to get started.

This article will be updated in-place.

Due to heavy load at work, as well as some minor things, I'm either taking back interest altogether, involvement altogether, time spent on projects, or any of these on aspects/particulars of projects.

Sometimes, when you're burnt out, it's best to concentrate on living and on core projects. mksh is one of these for me, as is keeping MirBSD userland and MirPorts infrastructure working well, with small, evolving improvements (no big jumps). Other things, no matter how nice, interesting or useful (to me as well as to others) they are, need to stay back. I poured most of the last seven years of my life into MirBSD.

Sometimes, you want to give back, but it's too much effort, or you cannot afford to spend more time on it. I'll close one of my Debian ITP bugs for this reason. (I also rarely send in patches from ports for this reason, but sometimes point upstream to our CVSweb.)

Sometimes, people like Ulrich Drepper, Marco d'Itri, Gerrit Pape let you realise that every project has its Theo de Raadt-alikes. I've still not ported jupp's latest release to Debian (but an OpenSuSE Buildservice SRPM exists), nor uploaded the current mksh(1) version even to my own wtf repo. I will do so, when I feel like spending private time with Debian again, at least for the etch and lenny (and hardy – for work) branches, as dash and mksh in sid have... issues I predicted ages ago. (For one, I'm still waiting for Gerrit to contact me. Maybe our eMail systems don't like each other? Waldi or formorer will probably pass on any messages, as will the trusty BTS.) I'll probably not open any ITP bugs again and send in much less of the bugs I notice, simply because I don't like being ignored (or flamed, but sometimes, being ignored is worse – which is why Benny works on MirPorts, btw). Maybe, if I feel the need to, my wtf repo will grow instead; DDs or DMs are free to take from there if they like.

Sometimes, one realises that he just doesn't fit in. While Cachewolf is a useful project, working together with Java™ developers that communicate over web fora only and don't even see the need for compatibility with Unix or proper processes most of the time (svn:eol-style comes to mind, and switching the source code to UTF-8 is something I've given up to dream of – I would even have fixed bugs where Ewe wouldn't do UTF-8 right, but I run into a wall of bliss ignorance there) proved impossible for me. I won't budge either: web fora are simply not for me to use. Period. This is my fault (for not fitting in) as well as the fault of some of the rest of the team (for ignoring years of experience, or for simply nicht über den eigenen Tellerrand schauen (however one says this in English, I don't know) and not caring of these who do; for supporting the commercial gc.com site over the three alternatives too). I will continue to use it, maybe the iPAQ H3600 a colleague gave me proves useful, otherwise, MirBSD will do just fine.

So, when I leave or pull back a little, no prejudices. Sometimes with reason, but mostly due to lack of available resources on my part. I hope nobody who has been or will be noticing me ceasing to contribute as much as usual thinks ill that's why.

mksh feature weeks

07.09.2009 by tg@
Tags: mksh

It's feature weeks for mksh(1). You almost certainly want to follow the mksh-current changelog. Please test all the new features, as well as make sure that nothing breaks nor I introduced other regressions, such as making resource usage worse (disc, CPU, memory). Please be reminded that many more deep-reaching changes will follow in the next few hacking days.

Thanks to David Korn (the Korn in AT&T Korn Shell, ksh88/ksh93) for discussing features and helping with finding syntax and semantic of some of the more complicated language pecularities.

On an unrelated note, cvs(1), lynx(1), ncurses(3), some other things, and a lot of ports have also been worked on. The next snapshot is planned for mid to end of September, bi-arch. Stay tuned!

MirOS bsd4me – Sane OS on netboot.me ☺

30.08.2009 by tg@
Tags: news snapshot

Christian Hofstaedtler «ch:#grml» pointed me to an article about netboot.me, a service providing bootable Live and Install variants of some OSes over the Internet (via gPXE and HTTP transport, mostly). They offer a bootable floppy/disc, USB stick, and El Torito ISO image. One gets a graphical menu if not careful, out of which there is no escape, and which isn’t compatible with most graphics cards, but if ^B is pressed quickly enough, there will be a usable command line.

An experimental MirOS bsd4me bootable image is available as well:

gPXE> autoboot
gPXE> chain http://netboot.me/2032

This will be loaded via plain, unencrypted HTTP from our main webserver. It’s the usual <5 MiB Mini-ISO for the i386 architecture and contains a full installer, as well as e3 and tinyirc known from MirOS bsd4grml. Source code of the GPL’d parts and overall licencing information is also included inside the ISO image. Please remember this is for beta testing only. We do not support MEMDISK boot methods with our second stage boot(8/i386) loader, as these are not reliable.

On an unrelated side note: dutch Windows XP Professional is weird. Translations were laughed at by several dutch-speaking people in the mksh IRC channel, but I was able to disable Blåtand on “blau” (the new IBM X40, successor of “bleu”). And it works!

A more related side note: when virtualising MirBSD, do not use VirtualBox. If you do have Vanderpool/Pacifica available and enabled, both the OSE and the full edition may be viable, but kvm is the better option. If VT-x/HVM is not used, it does not emulate the i386 architecture properly. Use qemu, kqemu, kvm, bochs, VMware Server 2 (not 1 – timekeeping is broken in there), Microsoft® VirtualPC 2004, Parallels Desktop, VMware Fusion for Macintosh, … instead.

Tonight and the last few days have seen several releases of things like MirMake, the package tools, etc. You might want to upgrade your MirPorts Framework checkout. (We now have enough Lua tools to access the Simtec eKey, I just need to order one. We’re currently at 2¾ interested people, not quite enough to make sense.)

Hot!

20.08.2009 by tg@
Tags: rant snapshot

It’s hot! The weather forecast said 35°C, one of the apothecaries’ thermometres says 40°C, and my body says I’m ill. I also nurse quite some headaches again. So I won’t do/write much. It was hard enough to go outside to buy breakfast.

On the other hand, the SS5 built a mock-snapshot (QUICK_DIST2, so only base+gcc, no X11/ports/htman/…) just fine (with some fixes). Does anybody have any RAM suitable for a SPARCstation 5 for me? I cannot use that from the SS20s because they use different kinds of RAM. I won’t publish this, but instead use it to build another one for real, possibly combined with another i386 snapshot, with all the /etc/* changes after 20090812, and maybe fixing what we will have learned to be broken in the meantime. And Benny’s ports fixes (he got GNOME except nautilus – which has unsafe code in signal handlers – working, part of (or all?) Xfce too, etc).

Luckily, tomorrow is more like 22°C

Karolina Lesińska writes: “We have placed the first 3 issue of BSDmagazine on our website as free downloads. The issues are:

  1. FreeBSD Ins & Outs
  2. OpenBSD in the Limelight
  3. Explore NetBSD

“The link is: http://bsdmag.org/prt/view/pdf-articles.html

My comment: The BSD magazine is a rather interesting piece of print, which already carried a small article about MirBSD. We have, I admit, been offered to write more which would get published, but, I am ashamed, haven’t quite gotten around to doing so.

Today, it's only been 29°C, and I died. What will I do tomorrow, where it is supposed to climb up to 35°C over today's 32°C?

The subject line says it all…

tg@ss5:~ $ uname -a
MirBSD ss5.mirbsd.org 10 Kv#10uAA-20090810 GENERIC#146 sparc
tg@ss5:~ $ dmesg
MirBSD#10uAA (GENERIC) #146: Mon Aug 10 17:57:16 UTC 2009
        tg@demo.mirbsd.org:/usr/src/distrib/generic/obj/build/GENERIC
real mem = 33165312
avail mem = 24928256
[…]
sd0 at scsibus0 targ 3 lun 0: <IBM, DORS32160SUN2.1G, WA7A> SCSI2 0/direct fixed
sd0: 2063MB, 6703 cyl, 5 head, 126 sec, 512 bytes/sec, 4226725 sec total
[…]
cgsix0 at sbus0 slot 3 offset 0x0: SUNW,501-2325, 1152x900, rev 11
wsdisplay0 at cgsix0: console (std, vt100 emulation), using wskbd0
[…]
tg@ss5:~ $ sysctl hw machdep
hw.machine=sparc
hw.model=SUNW,SPARCstation-5, MB86907 @ 170 MHz, on-chip FPU
[…]
machdep.cputype=2
machdep.v8mul=1
[…]
tg@ss5:~ $ top
[…]
Memory: Real: 17M/26M act/tot  Free: 1444K  Swap: 2640K/133M used/tot
[…]

It may only be a SPARCstation 5, with a rather fast CPU though (the SS20en only have 125 MHz and 150 MHz ROSS CPUs), with a small, slow and lout hard disc and very few RAM, but hey, it installs, the cgsix(4/sparc) scrolls quickly (as the tvtwo(4/sparc) currently is unaccelerated), and builds packages like a charm. Yay! I even have mc (Midnight Commander) now…

All 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31

MirOS Logo