Developers’ Weblog

Sponsored by
HostEurope Logo

Developers’ Weblog

All 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30

With the latest edition of MirOS #10-current’s snapshot (read the announcement for more details), we have trumped OpenBSD once more. While being dual-arch is nothing new (to them), and being runnable as Live-HDD, Live-CF-Card, etc. is nothing new (to us), we haven’t yet had a dual-arch Live-CD/DVD (or HDD or CF/SD/MMC/… Card or USB stick) yet. Well, only one of the two architectures has Live CD capability, but we neither have baselive image capability for sparc yet (though that would be possible, even if somewhat kludgy) nor would it have fit on the ISO 9660 image within the boundaries of a CD-R. And DVDs don’t match well with SPARCstation 20s ☻

Coming to that, I recently recherched if gcc 3.4.6, which we currently use as system compiler, can be used to build gcc 4.3 (with special care on the Ada part). Looks like you can even use gcc 2.95 for that, lucky us. This means I should somehow discover the Ada patch for gdb 6.3, which – again, luckily – we use, and apply it. I had talked with Benny about the compiler issue… originally I wanted to strip down the system compiler to the bare minimum once a more recent gcc (and possibly, LLVM, SUNWcc, pcc, etc.) are in ports and usable, but with the proposals to rewrite gcc in dreaded C++, C++ API issues, licencing issues, Ada bootstrapping, etc. we will probably continue to require GNU make for building the base compiler (the bad part) and be able to announce we’re good for developing (the good part of keeping the full-featured gcc near base). Just the Java™ part will not be re-enabled in near future, as it isn’t used for bootstrapping Iced Tea anyway. (It can be used for simple things though.)

With all the friendly fire, OpenBSD bashing, etc. aside, I would like to use this place to thank a few OpenBSD developers for helping me out with a bug in mksh as well as the issue with sshd(8) on sparc (and probably i386 even if it didn’t show up) I had (which even was my own fault). While you guys still don’t have an mksh port in your tree, some of you are really helpful (also hi Miod) sometimes, and I hope we can drink a few beer together at the next FOSDEM etc. and FrOSCon of course. (Which I hope we will also be able to use to discuss the state of our X11 tree.) To Linus: while OpenBSD developers may be masturbating monkeys, at least they’re able to design stable APIs, drink beer, and use sane version control systems with central quality control while being annoyed by humppa.

The usual unrelated news: Basel, ich komme! Schon mal GRÜẞE im Voraus an die Schweizer, die dies lesen.

mksh side notes: some older versions of SunC 5.9 fail on ({ stuff in acomp, use env HAVE_EXPSTMT=0 to build then. nwcc also has issues.

MirBSD side notes: We build for V8 CPUs on sparc, but still only V7 are emulated by tme. And while V9 CPUs in V8+ mode shouldn’t be a problem, there’s no interest from the OpenBSD camp to run the sparc (not sparc64) port on sun4u or even sun4v machines, so no fast build machines yet… *snief*

Since CacheWolf was the only use case, I decided to remove MirEwe from the repository entirely, as CacheWolf will be rewritten to use the Eve VM, inevitably.

Not only have I removed a last source tarball (distfile) of Lunix Ewe 1.49mb4, with CompileEwe.zip, JavaEwe.zip, and ewe.jar pregenerated, but the comma-v rcsfile(5)s are also archived in a gzipped sv4crc file, gzsig(1)ned, as usual.

Update 06.11.2008: We now have an Ewe icon Icon, thanks to smultron.

And, also as usual, one more unrelated thing: config(8) -ef /bsd should not only be able to handle gzip(1)ped kernels, but also personalise them by rewriting the initial_entropy array. TODO. The snapshot, dualive ISO 9660 image, is almost complete and will be uploaded later.

Update 26.07.2008: personalising done, snapshot broken.

mksh – The iShell

Thanks to help from people in the various IRC channels, and gecko2@ jailbreaking his iPhone 3G (not G3…), I was able to run (albeit not test – there is no Perl for the iPhone yet) two mksh(1) binaries on it: a normal one and a fat binary (aka universal binary – this one, containing code for 5 (five) architectures, can truly be called an universal binary ☺).

I packaged the modified build script, an extra ldid binary (built for OSX86) provided by someone else, and the output of the build process, together with a readme, as a gzsig(1)ned tarball on our server again – note that the cryptographic signature is only for time stamping (and download integrity checking) and does not imply any warranties, especially for the third-party software involved.

While here, we would be pleased if the reader supports our proposal and requests Apple to include mksh in the default installations. We would also be pleased if the mksh-iPhone binary were to make it into one of the Cydia repositories, contact us via IRC or mailing lists for assistance.

If someone were to package Perl 5.8 or so, that’d be great too.

To the readers from ##iphone-hackers on the one and #iphone-dev on the other IRC network: *waves a friendly hello*

My plans are as follows: after squishing out all remaining bugs, I’ll publish another MirOS-current snapshot (i386 baselive, i386 installer, sparc installer, some selected few binary packages for both arches), announce that on the website, and make a CVS repository tarball.

After that, I’ll concentrate on setting up the new servers (tear and euynome) fully, and dispose of rant-featuring-herc. Features I wanted, such as full AES vnd(4) encryption, DJBDNS with no split v4/v6 transport binaries, etc. will have to wait.

With the infrastructure in place, we can start hacking again. We had planned on bringing out MirOS #10bis in time for FOSDEM, but it looks more like just another -stable snapshot, and maybe another month for the second release of -stable; Benny will have to work some on the branch prior to its tagging (I’ll concentrate on -current), especially because we want to use X11 (and gcc) from HEAD, since these are not branched.

mksh – The iShell

Try this:

	$ cvs -d _anoncvs@anoncvs.mirbsd.org:/cvs co -PA mksh
	$ cd mksh
	$ cat >iBuild.sh <<-'EOF'
		DEV=/Developer/Platforms/iPhoneOS.platform/Developer
		SDK=${DEV}/SDKs/iPhoneOS2.0.sdk
		CC="${DEV}/usr/bin/gcc-4.0 -arch arm"
		CPPFLAGS="-I${SDK}/usr/include"
		CPPFLAGS="$CPPFLAGS -I${DEV}/usr/lib/gcc/arm-apple-darwin9/4.0.1/include"
		CPPFLAGS="$CPPFLAGS -F${SDK}/System/library/Frameworks"
		CPPFLAGS="$CPPFLAGS -F${SDK}/System/library/PrivateFrameworks"
		CPPFLAGS="$CPPFLAGS -F/System/library/Frameworks"
		LDFLAGS="-L${SDK}/usr/lib -L${DEV}/usr/lib/gcc/arm-apple-darwin9/4.0.1/lib"

		export CC CPPFLAGS LDFLAGS
		exec "$@"
	EOF
	$ mksh iBuild.sh mksh Build.sh -r
	$ file mksh

Mach-O executable arm

Of course, you need the iPhone 2.0 SDK on your Mac for this (and, presumably, an installed native mksh… otherwise just change the above instructions a little.

We haven't tested it yet, but if gecko2@ can figure out how to run mobileterminal.app inside the emulator, he will… and possibly jailbreak his iPhone G3.

Maybe this fact (availability of mksh) strengthens our point.

In the meanwhile, gecko2@ has found the geocaches I placed.

Update 18.07.2008 12:42 – we have a fat binary with five architectures (amd64 arm i386 powerpc ppc64) now. This is the iShell, indeed.

mksh on… platforms

16.07.2008 by tg@

Okay, so I installed Windows® 2000 and Interix again. And UWIN. And the Microsoft, Digital Mars, and Borland command-line compilers. And mksh worked, of course.

Then I tried DJGPP. Yeah, mksh(1) for MS-DOS®. *sigh* GNU bash, the only shell available, SIGSEGVs on me trying to build anything nōn-trivial, even worse with gcc 4.2.3 than with gcc 3.2.3… but even pdksh could not be made, due to SIGCHLD missing. Too bad, no mksh for you, Mr “I don’t accept bug report mails from your IP, nor in the webinterface” Delorie.

The iPhone… will have to wait. We can install the BSD Subsystem and the Terminal application in gecko2’s SDK’s ARM emulator, then build and test for that. I just gave him the link to the BSD package, but he went to bed shortly before, so maybe tomorrow.

And then, mksh R35b can go out, we fixed some bugs and tested on more platforms (fixing testsuite shortcomings during that time, even).

As requested, Benny has written to Apple, Inc. about inclusion of mksh. Lucas “laffer1” Holt of MidnightBSD has said to follow, as has Andreas “gecko2” Gockel, the Fink maintainer for mksh. Link to this entry.

Lucas actually said he would care more about and use mksh(1) if it were integrated in Mac OSX, as most MidnightBSD developers (like Benny) do most of their work on a Macintosh. He has integrated mksh in his operating system quite some time ago.

mksh is also the default shell, including /bin/sh, on FreeWRT Embedded GNU/Linux.

Andreas already has run tests for using mksh(1) as /bin/sh on Darwin, although I did not request that personally, and brought up the question of /etc/profile adjustments and ~/.mkshrc integration. (FreeWRT has a changed system-wide profile including a system-wide copy of the mkshrc file; other possible ways would involve setting $ENV or (preferred) placing .mkshrc in the New User’s Skeleton and in root’s home directory, if existent.

Strong points of mksh: free (as in BSD, now even without advertising clause), small, fast, portable, easy to use, can run many ksh93 and bash scripts, is actively developed, and benefits from the BSD development style (central development, security focus).

To everyone with an Apple ID: please suggest to Apple to include mksh(1), write to them how good it is despite lacking popularity, how compatible it is, and just how much you like to use it. Even if you do not use it yourself, if you can follow our reasons, write to them. If you have access to other operating systems, do the same, especially with mere porting frameworks, as the OpenBSD ports tree still does not include our port, which was ready-made for committing by me to reduce workload for them.

Some unrelated side notes… Geocacheing continues: OpenCaching.de The Frog Site Rare Jewels Broken Webpages² – Dr. Pfeffer especially liked mine^WWaldemar’s Zaurus SL-C3200 with CacheWolf running on Ewe… even if I still think it’s dead slow. Interestingly, porting Ewe to the iPhone G3 would not violate the clause preventing you from creating “instant messaging or real time navigation software”, and CacheWolf itself would not have to be ported. I’d like to have more RAM on the Z though – maybe via the SD Card slot?
SCSI sucks. And mksh gets better… IRIX, and a bug fix. Just still no SunOS 5.5 (missing /usr on the HDD).

This is the title of the bug report I filed with the Apple Bug Report Tool yesterday. The problem ID is 6069931; I think you can only see your own bug reports in the tracker though, so you may not be able to see my report. FWIW, I filed it against Mac OS X 10.5.4 (the current version), with "Feature (new)" as a severity. For reference, the full text of the bug report follows:

11-Jul-2008 11:58 AM Benny Siegert:

I would like to kindly request the inclusion of mksh as an alternative shell into future versions of Mac OS X.

mksh (the MirOS Korn Shell) is a Korn Shell derivative under a BSD-like licence. It includes the features from the OpenBSD ksh as well as additional bash compatibility. Almost all bash scripts run under mksh, however with a speedup of about a factor 2. The code was checked for security problems by Coverity, Inc. UTF-8 support is fully integrated. With all these features, it is still very small, just 240 KiB as an i386 binary. The build process is done by a script called Build.sh, Mac OS is already a fully supported and maintained platform.

This could potentially replace the ksh93 included now as /bin/ksh or be installed side by side as /bin/mksh. In short, it can be regarded a more modern replacement for older ksh releases.

The homepage of mksh is located at http://mirbsd.de/mksh. The current version is available from https://www.mirbsd.org/MirOS/dist/mir/mksh/mksh-R35.cpio.gz.

continuity

09.07.2008 by tg@

Since quite a while I’ve had anger about BSDstats – their ports reporting stopped including ours (MirPorts Framework, NetBSD® pkgsrc®, OpenBSD ports tree, Debian dpkg), the maintainer was rarely reachable, at most by eMail, and the entire system had issues. Now, the site no longer exists, apparently nobody knows why, and our systems error out. Well, I have disabled BSDstats, this will probably make MirOS appeal more to privacy-conscious geeks again, and save us from getting more errors in our logfiles, especially the Live CD reporting was already totally broken due to weird User-Agent tricks.

I’ll now count on people just using IRC or so to say hi to us if they’re using MirOS, are happy or tell us things we could improve. Or meet us at conferences and pay for our beer ☺ and other beverages (we’re not BeerFSD after all).

Configuring name-based apache vhosts with SSL and IPv6… is hard. Really. This is probably due to it being “a patchy webserver”. Here is a quick cut and paste solution, well, sort of.

Scenario: I have a web server running name-based vhosts on the SSL port. I get a either wildcard certificate, or – in my case – a certificate with some subjectAltName extensions set; CAcert supports these now. All vhosts I wish to serve must be included, and the primary commonName (as part of the distinguishedName) ought to be the host’s primary FQDN, and it must be duplicated as subjectAltName.

I wish to serve a default page (just the standard index.htm from /var/www/htdocs/) to everyone who does not provide a proper host name (e.g. speaking HTTP/1.0 without Host: header, or just probing my IP addresses), and a couple of vhosts to everyone else. I use one IPv4 and one IPv6 address. The vhost configuration is placed in several files in /var/www/conf/vhosts/ called «vhostname».«port»; if some information is to be shared (e.g. vhost on both port 80 and 443), these are placed in «vhostname».common and included by the port configs.

Important: Do not use wildcard vhosts, they won’t work and give errors. Do not use the _default_ vhost in any form, it silently won’t work.

You might want to place directory information in the vhost configuration file instead of in the main configuration file.

Part of httpd.conf is:

	ServerAdmin «mail address for the server itself»
	ServerName «main FQDN»

	NameVirtualHost 192.0.2.1 443
	NameVirtualHost 2001:db8:1234:1::1 443

	<IfDefine SSL>

	<VirtualHost 192.0.2.1:443>
	Include /var/www/conf/vhosts/eurynome.443
	</VirtualHost>

	<VirtualHost [2001:db8:1234:1::1]:443>
	Include /var/www/conf/vhosts/eurynome.443
	</VirtualHost>

	<VirtualHost 192.0.2.1:443>
	Include /var/www/conf/vhosts/call.443
	</VirtualHost>

	<VirtualHost [2001:db8:1234:1::1]:443>
	Include /var/www/conf/vhosts/call.443
	</VirtualHost>

	<VirtualHost 192.0.2.1:443>
	Include /var/www/conf/vhosts/default.443
	</VirtualHost>

	<VirtualHost [2001:db8:1234:1::1]:443>
	Include /var/www/conf/vhosts/default.443
	</VirtualHost>

	</IfDefine>
 

(using documentation addresses)

Make sure every vhost except the default one sets ServerName (and ServerAdmin if it differs), and DocumentRoot etc. as usual. The default vhost does not need to set ServerName, but the line
ServerAlias *
must be present and it must be the last vhost in the list, as above.

If you still have questions – especially now that even OpenBSD has seen the light and applied the IPv6 patch I not only applied to MirBSD some four years ago but also suggested to them some two or three times – come to IRC or drop us an eMail, and we’ll try to fix things. Apache is definitively historic ballast, but still common enough on todays Unices.

So the uppercase eszett – ‘ẞ’ (or ‘ẞ’ if your monospace font has it) – is now an ISO 10646 standard. Your favourite BSD has been one of the first to add support for it to libc, a (contributed) keyboard layout, and even Markus Kuhn’s fixed-misc Unicode fonts, after the proposal with a real codepoint assignment came out. Not all Germans, nor typographers, but historicians, might like it, the discussion has been flamewar-like. But we knew this all along.

Ah, by the way. No, we aren’t dead. Websites are irrelevant, but if you end up here, please read about mksh anyway. Real questions are asked on the mailing lists or in IRC. And we don’t have a handbook (yet), because with two developers, nobody has yet had enough spare time to write one, plus you’d need a couple of handbooks – one for people coming from Open/NetBSD (Lite-based), one for people coming from Free/DragonFly/MidnightBSD (386BSD-based), one for people coming from Unix®, one for the Apple followers, one for the GNU/Leenocksers, etc. You see?

And while here: Xcode 3.1 (on gecko2@’s Macbook) comes with llvm-gcc… but not in /usr/bin/ but in /Developer/usr/bin/. It compiles quickly, produces fast and small code… kind of nice. If Apple brings LLVM+clang into a usable shape to replace our gcc 3.4.6 (gcc4 has unsafe optimisations that cannot be turned off, and all other compilers produce bigger code, which breaks the installation media (especially floppies) and SPARC kernels), in a form usable as compact command-line compilers (and not just libraries for Xcode integration), I’ll be happy. (They’d just need to add mksh to Mac OSX then to make me even happier, but I guess I’m sort of late with that request. Even if it’s fully free – in contrast to GNU bash – and, in its most recent incarnations, even advertising clause free.)

Getting things done… slowly

13.06.2008 by tg@
Tags: hardware

While “the swiss are laggards, and the people from Bern are the slow motion of switzerland”, other things take their time as well. However, vnd(4) updates have started, and other encryption schemes (including a hardware-accelerated AES variant for Columbo0815 from #IceWM) will end up in the code very soon, as promised.

During hacking, I spot other things – bugs in the random(4) code, as well as improvements to the kernel and XFree86® possibly leading to my pcengines.ch ALIX.1c board to be supported (thanks to undeadly.org for keeping me informed). And Benny fixed ports bugs again, as well as put security updates for Freetype2 et al. into the tree. Thanks!

FWIW: Opera 9.50 segfaults on me. It can’t be updated thusly.

All 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30

MirOS Logo