Developers’ Weblog

Sponsored by
HostEurope Logo

Developers’ Weblog

All 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31

Still cats have less sorrows. Now it’s proven that feline specimen can actually spell. Poor “lolcats” who are abused by many.

Discovering Rivella and Tonnerre mentioning that swiss chocolate exported into the EU is artificially made more sweet than inland production (yuck, sweet chocolate) are some promising points voting for working in Switzerland. It’s still expensive and a major change though. But the payment is good, too.

While mkisofs is finally gone, I had to fix lots of bugs and port a new language, lang/pawn, actually better than others, but the interna still look more than very fragile. Same problem as with Ewe: written for Win32-only at first, made portable later by persons who don’t know how to code for Unix. Ugh, and… I’ve got to learn Python.

Either my VPN, gecko2s Server, or the network (at home, Strato, or here) sucks. Hm… maybe it’s the backup Tonnerre makes.

I should search for some more geocaces, but now that I have some more time than at the beginning of the week, the weather is worsening, and we still have the duties. *grml* I need tear set up, too…

I started hacking on the bootloader issue, decided to move out the various libraries into src/kern/ first, and completed that for zlib, began it for the rest. The other places – src/lib/libc/ src/sys/lib/ src/sys/arch/*/stand/lib* – now look much cleaner already, and in fact can do some better. I also merged bootloader and other improvements from OpenBSD before beginning the file move dance, but it still worked for me in qemu for testing, so I didn’t expect much breakage.

Furthermore, MirOS now officially has OPTU-8 locale (no more CESU-8), even if masqueraded as “en_US.UTF-8” for better application support. The CSUR had already, as posted about quite a few times, allocated us a PUA range, which mksh was already using, and now the base system and libc are beginning to catch up. Thanks to a pen and paper “game” I have sort of a defined finite machine modelling the conversion states, and found out I can get by with the mbstate_t we already were using. tr(1) already doesn’t wreck ISO-8859-1 accents any more (nor binary files), although that definitively needs some testing.

I’d just like to warn people that HEAD (MirOS-current) is a pretty moving target at the moment.

… I still wish I were a cat, sometimes. Just have to stay clear of cat abusers.

Tonight, I managed to port Wine 1.0 from pkgsrc®, with thanks to Hubert Feyrer for mentioning, except it seems to stumble about W^X and memory allocation/mapping in general. I guess we need some help from OpenBSD toolchain experts and possibly both FreeBSD® and pkgsrc® porters dealing with this. I can’t, but I tried a lot.

My other project for tonight failed, once because it has just too many unported dependencies, second because one of them requires gcc4, namely webkit (non-Qt). Okay, they say webkit-Qt can be built with gcc 3.4.6, but it seems to be not recommended. Thus no midori for us ☹ (yet).

The third thing was getting rid of J�rg Schilling… but due to the heat I slept bad and could not get up early. Really! (Yeah yeah…)

You all should read Tonnerre’s wonderful blog posting about how he got rid of Subversion, luckily. (I wonder about said “new” developer’s preference as I’d rather use CVS for everything…)

… oder: de vitae felidae


Katze müßte man sein, oder Kater halt, je nachdem. Den ganzen Tag sich faul in der Sonne räkeln, unter dem Heck eines Autos liegen und ab und zu den an ihr vorbeiradelnden Zweibeinern zuzwinkern und sie mit dem Kopf verfolgen; wenn man Hunger hat, reicht es, den Dosenöffner auf zwei Beinen mitleiderregend anzuschauen und/oder zu -miauen… manche haben sogar ein eigenes Blog oder gar ein BSD nach einem benannt; wem das nicht reicht hält sich den Nachbar des eigenen Dosenöffners, nämlich homsn, als zusätzlichen Hausmensch (quasi wie wir uns Haustiere, z.B. Hunde, halten)… der krault einen dann, und man muß nichtmals hingehen.


I just coined a new term: IMACS. The connotation is: I’ll hack the ukbd(4) driver to allow inmetaaltctrlshifting entropy into the kernel too, soon.

Update 26.07.2008: sadly, USB keyboards don’t seem to generate any interrupt requests on modifier key presses or releases. This means you’re stuck to DIN or PS/2 pckbd(4) or Sun (&c.) zskbd(4/sparc) for entropy collection via IMACSing. Can’t be helped…

With the latest edition of MirOS #10-current’s snapshot (read the announcement for more details), we have trumped OpenBSD once more. While being dual-arch is nothing new (to them), and being runnable as Live-HDD, Live-CF-Card, etc. is nothing new (to us), we haven’t yet had a dual-arch Live-CD/DVD (or HDD or CF/SD/MMC/… Card or USB stick) yet. Well, only one of the two architectures has Live CD capability, but we neither have baselive image capability for sparc yet (though that would be possible, even if somewhat kludgy) nor would it have fit on the ISO 9660 image within the boundaries of a CD-R. And DVDs don’t match well with SPARCstation 20s ☻

Coming to that, I recently recherched if gcc 3.4.6, which we currently use as system compiler, can be used to build gcc 4.3 (with special care on the Ada part). Looks like you can even use gcc 2.95 for that, lucky us. This means I should somehow discover the Ada patch for gdb 6.3, which – again, luckily – we use, and apply it. I had talked with Benny about the compiler issue… originally I wanted to strip down the system compiler to the bare minimum once a more recent gcc (and possibly, LLVM, SUNWcc, pcc, etc.) are in ports and usable, but with the proposals to rewrite gcc in dreaded C++, C++ API issues, licencing issues, Ada bootstrapping, etc. we will probably continue to require GNU make for building the base compiler (the bad part) and be able to announce we’re good for developing (the good part of keeping the full-featured gcc near base). Just the Java™ part will not be re-enabled in near future, as it isn’t used for bootstrapping Iced Tea anyway. (It can be used for simple things though.)

With all the friendly fire, OpenBSD bashing, etc. aside, I would like to use this place to thank a few OpenBSD developers for helping me out with a bug in mksh as well as the issue with sshd(8) on sparc (and probably i386 even if it didn’t show up) I had (which even was my own fault). While you guys still don’t have an mksh port in your tree, some of you are really helpful (also hi Miod) sometimes, and I hope we can drink a few beer together at the next FOSDEM etc. and FrOSCon of course. (Which I hope we will also be able to use to discuss the state of our X11 tree.) To Linus: while OpenBSD developers may be masturbating monkeys, at least they’re able to design stable APIs, drink beer, and use sane version control systems with central quality control while being annoyed by humppa.

The usual unrelated news: Basel, ich komme! Schon mal GRÜẞE im Voraus an die Schweizer, die dies lesen.

mksh side notes: some older versions of SunC 5.9 fail on ({ stuff in acomp, use env HAVE_EXPSTMT=0 to build then. nwcc also has issues.

MirBSD side notes: We build for V8 CPUs on sparc, but still only V7 are emulated by tme. And while V9 CPUs in V8+ mode shouldn’t be a problem, there’s no interest from the OpenBSD camp to run the sparc (not sparc64) port on sun4u or even sun4v machines, so no fast build machines yet… *snief*

Since CacheWolf was the only use case, I decided to remove MirEwe from the repository entirely, as CacheWolf will be rewritten to use the Eve VM, inevitably.

Not only have I removed a last source tarball (distfile) of Lunix Ewe 1.49mb4, with CompileEwe.zip, JavaEwe.zip, and ewe.jar pregenerated, but the comma-v rcsfile(5)s are also archived in a gzipped sv4crc file, gzsig(1)ned, as usual.

Update 06.11.2008: We now have an Ewe icon Icon, thanks to smultron.

And, also as usual, one more unrelated thing: config(8) -ef /bsd should not only be able to handle gzip(1)ped kernels, but also personalise them by rewriting the initial_entropy array. TODO. The snapshot, dualive ISO 9660 image, is almost complete and will be uploaded later.

Update 26.07.2008: personalising done, snapshot broken.

mksh – The iShell

Thanks to help from people in the various IRC channels, and gecko2@ jailbreaking his iPhone 3G (not G3…), I was able to run (albeit not test – there is no Perl for the iPhone yet) two mksh(1) binaries on it: a normal one and a fat binary (aka universal binary – this one, containing code for 5 (five) architectures, can truly be called an universal binary ☺).

I packaged the modified build script, an extra ldid binary (built for OSX86) provided by someone else, and the output of the build process, together with a readme, as a gzsig(1)ned tarball on our server again – note that the cryptographic signature is only for time stamping (and download integrity checking) and does not imply any warranties, especially for the third-party software involved.

While here, we would be pleased if the reader supports our proposal and requests Apple to include mksh in the default installations. We would also be pleased if the mksh-iPhone binary were to make it into one of the Cydia repositories, contact us via IRC or mailing lists for assistance.

If someone were to package Perl 5.8 or so, that’d be great too.

To the readers from ##iphone-hackers on the one and #iphone-dev on the other IRC network: *waves a friendly hello*

My plans are as follows: after squishing out all remaining bugs, I’ll publish another MirOS-current snapshot (i386 baselive, i386 installer, sparc installer, some selected few binary packages for both arches), announce that on the website, and make a CVS repository tarball.

After that, I’ll concentrate on setting up the new servers (tear and euynome) fully, and dispose of rant-featuring-herc. Features I wanted, such as full AES vnd(4) encryption, DJBDNS with no split v4/v6 transport binaries, etc. will have to wait.

With the infrastructure in place, we can start hacking again. We had planned on bringing out MirOS #10bis in time for FOSDEM, but it looks more like just another -stable snapshot, and maybe another month for the second release of -stable; Benny will have to work some on the branch prior to its tagging (I’ll concentrate on -current), especially because we want to use X11 (and gcc) from HEAD, since these are not branched.

mksh – The iShell

Try this:

	$ cvs -d _anoncvs@anoncvs.mirbsd.org:/cvs co -PA mksh
	$ cd mksh
	$ cat >iBuild.sh <<-'EOF'
		DEV=/Developer/Platforms/iPhoneOS.platform/Developer
		SDK=${DEV}/SDKs/iPhoneOS2.0.sdk
		CC="${DEV}/usr/bin/gcc-4.0 -arch arm"
		CPPFLAGS="-I${SDK}/usr/include"
		CPPFLAGS="$CPPFLAGS -I${DEV}/usr/lib/gcc/arm-apple-darwin9/4.0.1/include"
		CPPFLAGS="$CPPFLAGS -F${SDK}/System/library/Frameworks"
		CPPFLAGS="$CPPFLAGS -F${SDK}/System/library/PrivateFrameworks"
		CPPFLAGS="$CPPFLAGS -F/System/library/Frameworks"
		LDFLAGS="-L${SDK}/usr/lib -L${DEV}/usr/lib/gcc/arm-apple-darwin9/4.0.1/lib"

		export CC CPPFLAGS LDFLAGS
		exec "$@"
	EOF
	$ mksh iBuild.sh mksh Build.sh -r
	$ file mksh

Mach-O executable arm

Of course, you need the iPhone 2.0 SDK on your Mac for this (and, presumably, an installed native mksh… otherwise just change the above instructions a little.

We haven't tested it yet, but if gecko2@ can figure out how to run mobileterminal.app inside the emulator, he will… and possibly jailbreak his iPhone G3.

Maybe this fact (availability of mksh) strengthens our point.

In the meanwhile, gecko2@ has found the geocaches I placed.

Update 18.07.2008 12:42 – we have a fat binary with five architectures (amd64 arm i386 powerpc ppc64) now. This is the iShell, indeed.

mksh on… platforms

16.07.2008 by tg@

Okay, so I installed Windows® 2000 and Interix again. And UWIN. And the Microsoft, Digital Mars, and Borland command-line compilers. And mksh worked, of course.

Then I tried DJGPP. Yeah, mksh(1) for MS-DOS®. *sigh* GNU bash, the only shell available, SIGSEGVs on me trying to build anything nōn-trivial, even worse with gcc 4.2.3 than with gcc 3.2.3… but even pdksh could not be made, due to SIGCHLD missing. Too bad, no mksh for you, Mr “I don’t accept bug report mails from your IP, nor in the webinterface” Delorie.

The iPhone… will have to wait. We can install the BSD Subsystem and the Terminal application in gecko2’s SDK’s ARM emulator, then build and test for that. I just gave him the link to the BSD package, but he went to bed shortly before, so maybe tomorrow.

And then, mksh R35b can go out, we fixed some bugs and tested on more platforms (fixing testsuite shortcomings during that time, even).

As requested, Benny has written to Apple, Inc. about inclusion of mksh. Lucas “laffer1” Holt of MidnightBSD has said to follow, as has Andreas “gecko2” Gockel, the Fink maintainer for mksh. Link to this entry.

Lucas actually said he would care more about and use mksh(1) if it were integrated in Mac OSX, as most MidnightBSD developers (like Benny) do most of their work on a Macintosh. He has integrated mksh in his operating system quite some time ago.

mksh is also the default shell, including /bin/sh, on FreeWRT Embedded GNU/Linux.

Andreas already has run tests for using mksh(1) as /bin/sh on Darwin, although I did not request that personally, and brought up the question of /etc/profile adjustments and ~/.mkshrc integration. (FreeWRT has a changed system-wide profile including a system-wide copy of the mkshrc file; other possible ways would involve setting $ENV or (preferred) placing .mkshrc in the New User’s Skeleton and in root’s home directory, if existent.

Strong points of mksh: free (as in BSD, now even without advertising clause), small, fast, portable, easy to use, can run many ksh93 and bash scripts, is actively developed, and benefits from the BSD development style (central development, security focus).

To everyone with an Apple ID: please suggest to Apple to include mksh(1), write to them how good it is despite lacking popularity, how compatible it is, and just how much you like to use it. Even if you do not use it yourself, if you can follow our reasons, write to them. If you have access to other operating systems, do the same, especially with mere porting frameworks, as the OpenBSD ports tree still does not include our port, which was ready-made for committing by me to reduce workload for them.

Some unrelated side notes… Geocacheing continues: OpenCaching.de The Frog Site Rare Jewels Broken Webpages² – Dr. Pfeffer especially liked mine^WWaldemar’s Zaurus SL-C3200 with CacheWolf running on Ewe… even if I still think it’s dead slow. Interestingly, porting Ewe to the iPhone G3 would not violate the clause preventing you from creating “instant messaging or real time navigation software”, and CacheWolf itself would not have to be ported. I’d like to have more RAM on the Z though – maybe via the SD Card slot?
SCSI sucks. And mksh gets better… IRIX, and a bug fix. Just still no SunOS 5.5 (missing /usr on the HDD).

This is the title of the bug report I filed with the Apple Bug Report Tool yesterday. The problem ID is 6069931; I think you can only see your own bug reports in the tracker though, so you may not be able to see my report. FWIW, I filed it against Mac OS X 10.5.4 (the current version), with "Feature (new)" as a severity. For reference, the full text of the bug report follows:

11-Jul-2008 11:58 AM Benny Siegert:

I would like to kindly request the inclusion of mksh as an alternative shell into future versions of Mac OS X.

mksh (the MirOS Korn Shell) is a Korn Shell derivative under a BSD-like licence. It includes the features from the OpenBSD ksh as well as additional bash compatibility. Almost all bash scripts run under mksh, however with a speedup of about a factor 2. The code was checked for security problems by Coverity, Inc. UTF-8 support is fully integrated. With all these features, it is still very small, just 240 KiB as an i386 binary. The build process is done by a script called Build.sh, Mac OS is already a fully supported and maintained platform.

This could potentially replace the ksh93 included now as /bin/ksh or be installed side by side as /bin/mksh. In short, it can be regarded a more modern replacement for older ksh releases.

The homepage of mksh is located at http://mirbsd.de/mksh. The current version is available from https://www.mirbsd.org/MirOS/dist/mir/mksh/mksh-R35.cpio.gz.

All 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31

MirOS Logo