Developers’ Weblog

Sponsored by
HostEurope Logo

Developers’ Weblog

All 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31

Today, I was sort of bored, so I enhanced something other than our libc, namely pcc, with support for Unicode/UTF and wide character literals and strings. Kind of fun it was.

Still missing is support for \uXXXX literals in multibyte strings (need ucs2utf function for that, which can be taken from our libc again), and, maybe, automatic UTF-16/CESU-8 conversions, if the target has UCS-2 (MirBSD, Windows) and the host doesn’t, or so.

Every single time Benny works on ports/www/firesomething, it sinks in my esteem. Generating crypto keys during fake install is not how things should be done (use a post-pkg_add script!). And I still think it’s only rumours that it had been rewritten after the Netscape® code was opened – it would be better than before then, not worse. I’d rather shoot myself than publish such filth, if, like Benny says, the code had indeed been rewritten. Well, at least it is aware how insecure it is and kills itself:

14:46⎜«benz» Nov 23 16:40:29 mirbulk firesomething-bin: stack overflow in function virtual nsresult nsCryptoHash::Finish(PRBool, nsACString_internal&)

And if he wouldn’t document this in a wlog entry, I do ☻☺

But then, I don’t believe him totally unaware:

14:09⎜«benz» The name "Minefield" is strangely appropriate

galloc and OPTU

22.11.2008 by tg@
Tags: unicode mksh

aalloc was a failure and prone to mis-type; I’ve started on galloc, which is supposed to be even more type-safe and flexible, yet still segfaults on me as well.

In the meanwhile, to not stay bored, I enhanced the Unicode (MirOS OPTU-8/OPTU-16) function suite in MirBSD and hacked an implementation of Plan 9’s Rune functions – dubbed p9¾_*.c due to usage of our “internationalisation like Plan 9, just on the next layer, and within the confines of UNIX®” approach to internationalisation. These (as well as some well-known ones like wcsrtombs(3)) should be OPTU safe, and we now have a macro telling us if a wide character is part of the OPTU Raw Octet codepoint range (in the CSUR PUA assignment).

I’ve contacted Bruno Haible again because his libutf8 misses the Unicode Title case (complementing Upper and Lower case)… I’ve got an idea how to implement it but would like to double-check with him to ensure nothing breaks. The Plan 9 functions need it (these two are currently implemented as stubs that just throw ENOCOFFEE).

The number of manual pages also raised…

I built XFree86® with a (slightly beefed up) Reiser CCCP in use as sole C Præprocessor now… and have yet to notice failures in operation. Now we’ll have to find out what else stuff is there depending on the existence of /usr/bin/cpp, possibly switching it, like the <> mirmake(1) file, to ${CC} -E - (which is not 100% compatible, because cpp can also read from stdin without the dash, whereas, with the dash, neither can read from only a file). X11 etc. can be switched to /usr/libexec/cpp then.

aalloc and COFF

17.11.2008 by tg@
Tags: mksh

I got (E)COFF executables to work, even though I cannot produce them except with a hand-crafted binary yet. Maybe these produced by kencc work now.

I didn’t get the time or nerve to continue hacking on aalloc today… the whole pointers thing in C is so useless. If I could use assembly, it were so trivial. Anyway, the TPtr data type will probably have to get lost, and I plan on using more temporary local variables for overview.

Besides a beer discussion, IRC channel today featured plans on getting a temporary setup of fast enough machines for crossbuilding OpenJDK; replaced confirmed that someone had success in doing what I planned, although the nōn-free JDK 1.5 only. We’ll see…

small updates

12.11.2008 by tg@
Tags: security

/MirOS/current/i386/fixes10.ngz holds a new libm, with which you can use the three new binary packages (lzma, llvm, llvm-gcc4.2) I’ve just placed in /Pkgs/current/i386/ for your fun. These all have been compiled using mgcc-10uA3, as llvm-gcc4.2 is not up to the task.

The new “aalloc” area-based omalloc-employing pool allocator is still unfinished; I need to make cookies work and perhaps add more ifdefs to make it smaller for MKSH_SMALL users (installer, FreeWRT, …). However, feel free to read it and send me comments. I admit it might want to be cleaned up a little, but first it’s got to be finished. Do not use it yet; it’s intended for both mksh and libobjfw though. Also, I’d like to hear of possible optimisations or more security stuff one can throw in; the current scheme is the compromise looking best to me. Remember that it was written with mmap malloc (and omalloc – Otto Moerbeek zijn malloc) and malloc.conf(5) MALLOC_OPTIONS guard/junk/protection in mind.


10.11.2008 by tg@

First off, the X.509v3 certificate for gzsig(1) has been renewed. While, strictly spoken, this is not necessary, it eases validation.

MirPorts now not only contain ports for various compilers such as pcc (which does not suck anymore), LLVM+Clang (which does suck, because it’s written in ugly CFrustFrust) and llvm-gcc 4.2, it can also peruse them to compile packages:

  • $ mmake USE_COMPILER=system # mgcc (C, C++), Default
  • $ mmake USE_COMPILER=pcc # pcc (C only)
  • $ mmake USE_COMPILER=llvm # llvm-gcc (C, C++, Fortran 77/9x)

While clang cannot yet be used there really, the script of mksh can use its “ccc” compiler driver (which however is experimental) as well as llvm-gcc to build mksh(1)… to native code, or to intermediate bytecode (with link-time intra-module optimisation and final assembly into native code).

The LLVM Interpreter lli also, surprisingly, works. No patch required.

This (LLVM) is for MirOS-current i386 only; Darwin probably just needs to be tested, whereas MidnightBSD and OpenBSD require some more work.

TODO: there is an LGPL’d lzmadec binary, C only even; do something with it, possibly to ease use of .clz binary packages. Recent storage needs have become utterly ridiculous.

website fun

09.11.2008 by tg@
Tags: security

Now that the website is running on MirOS httpd(8), we can have some fun… SSL and such. The following picture may not be displayed if you use a nōn-rsync’d mirror, but can be worth the fun:

connection info

An ASCII connection info version is, of course, available as well, but images can be embedded more easily. For example, if you were to read this wlog entry via SSL (link provided courtesy to readers using Planet Symlink), the information would differ.

And if you still get warnings about an unknown certificate authority, CAcert provides them for downloading into browsers such as M*zilla Firef*x whose idiotic development team opposes against them. For what it’s worth, unchecked SSL is still better than no SSL at all (especially for entropy ☺) unless you think encrypted = authentic identify.

WTF we have a symlink

01.11.2008 by tg@
Tags: bug snapshot

Much closer to a new DuaLive CD. Reading Ecma 119 (ISO 9660), El Torito, and IEEE P1282 (Rock Ridge). Fixing bugs in makefs(8) as if there were no tomorrow. Even ifdef’d MirBSD deps, so that they™ can take everything back. Finding bugs in the kernel (one harmless, one deadly), and amusing myself over certain students. (Bit 0 == 2⁰ == 0x01 not 0x00…)

What’s left? Actual testing (qemu, real hardware), and I suspect that the RR_MOVED creation code does not split the directory correctly at 2048 byte sector boundaries, like all the others. (How many damn academics did work on this part of the code, anyway? Modularisation sucks, major arse.) Being paranoid and using calloc(3) has saved me from most other illnesses since.

Leider mußte ich den Federroten heute alleine trinken, aber immerhin konnte ich wbx@ überzeugen, sich zur mksh zu bekennen – sie kann UTF-8 und überhaupt Umlaute in der Shell, schön mit nur einmal Rücktaste drücken und so. Und seinen GNU screen hab ich ihm auch noch gefixt. Dafür bin ich den Sharp Zaurus SL-C3200 (und somit auch meine einzige OpenBSD-Testkiste) wieder los… eventuell nutzt er ja MirPorts, denn unser screen weiß, obs in einem Unicode- oder 8-Bit-Terminal rennt, wsconfig/chkuterm sei dank.

Warum sagt mir eigentlich niemand, daß mor…^Wheute die Geschäfte zuhaben? Ich bin ja mal voll für src/usr.bin/calendar/calendars/de_DE/calendar.ladenöffnungszeiten.$bundesland (das geht ja sonst mal gar nicht)!

I discovered that the CD drive in my docking station – namely a cd0 at scsibus0 targ 0 lun 0: <HL-DT-ST, DVD-ROM GDR8083N, 0K04> SCSI0 5/cdrom removable – neither is removable (the docking station is not exactly hot-pluggable) nor can it write. It can read various kinds of DVDs, CD-ROM, CD-R, CD-RW, but not burn any ☹ And since the lid seems to be custom-fit, I probably cannot simply take odem’s DVD-ROM/CD-RW slim drive either. That said, the sparc still cannot read CD-RW either… must discover some CD-R media…

I had a good laugh at J�rg “schily” Schilling learning English: &dp[14] /* Hunreds of a Seconds */ – That he doesn’t grok licencing is already well-known, so no need to bash that again.

new developer

26.10.2008 by tg@

Adam “replaced” Hoka, who has advertised and bug-spotted (and fixed) in the past, begun a Haiku port of mksh. He has been given CVS commit access for that purpose (possibly extended later), and as such, is now the 8th developer (counting both active and passive ones) to The MirOS Project.

Apparently, the Haiku port now compiles but does not run. Possibly another buggy operating system to add to the list.

I somehow end up fixing pcc each time I try to build it. But code size on i386 has much improved recently, although still nowhere near gcc ‘-Os’ already much better than its ‘-O2’ counterpart, both gcc3 and gcc4. I’m still waiting for more bug fixes, and more of my fixes integrated into upstream, before updating the pcc mirport though.

While I removed the use of __typeof__ from the mksh(1) source, SUNWcc still explodes on it unless we use HAVE_EXPSTMT=0 *sigh*

There are new “hosted” subprojects in our CVS repository. These are for each committer to place his/her stuff into, which, in contrast to other stuff in the contrib module, does not necessarily need to have something to do with BSD, Unix, shell, etc. However, existing mismatched code (the .exe files of my keyboard layout and Benny’s polymers code) will not be moved, because that doesn’t make sense with cvs(1).

Today, my health has been somewhat better, but I still feel as if I had a cold but hidden, so it does not impact me that much but I cannot lose it either. Headaches started at about 20³⁰, which I consider good, even though I now cannot comply with Jonathan’s wish of having a look at llvm-gcc for spotting a bug in libobjfw.

While I could use newfs(8)+vnd(4)+mount(8) or makefs(8) with a fixed block size for a live CD, or a ports version of mkisofs, or create the image on Darwin, I do not think so. The next snapshot will just not be a Live CD at all. Later, guys. Maybe TNF helps fixing it.

mksh also did not have the newest nifty features, such as <(cmd), applied, due to my ongoing illness feeling and added headaches all of this evening. However, I did finish the OSF/1 V2.0 (MIPS) port, and tomorrow there will be a proper announcement of R36, including ports, source RPMs, etc.

Many thanks to DEChengst from the #!/bin/mksh IRC channel again for giving me access to such machines.

I built LLVM+clang on Debian Lenny, just to make it crash *grin* due to an internal assertion… in mksh(1)’s source code. Gah. Anyway, got the problem reported. I wonder if I should remove Objective-C from the base gcc since it’s not just almost but really unusable anyway (same for Ada unless it can be used to build llvm-gnat-4.2). Jonathan gave up on the GNU runtime for good, or so he told me.

Today I also tried to get a neighbour’s fax-scan-print-copy combo running. Ends up that both the new phone line he bought for it was faulty (I gave him a replacement from my Attic), the AVM Fritzbox has… issues, and when I finally got my own fax from anno 1985 to work, and his Officejet not, we belive that either the ISDN voltage (it was accidentally plugged next to the NTBA first) or, much more likely, a recent thunderstorm b0rked it. So much for high-tech. My low-tech works.

All software sucks…

22.10.2008 by tg@
Tags: bug

I did not get much further. While I had the brilliant idea to embed a native little-endian BSD disklabel inside the first 512 bytes of the first-stage sparc bootloader (as the code searches for it anywhere in sector #1 inside the partition – or disc, if no MBR partition found – aligned to a long boundary), because an MBR partition would fake about two slices in the Sun disklabel which… could cause issues, the software I wanted to use to create a testcase has… bugs.

I ended up creating an install-only ISO and wanted to put the rest of the files into a 4.2FFS filesystem with makefs(8). However, that one did not want as I do, and that for quite a while. Cheers to TNF! grml…

The update of Opera to 9.61 also failed for reasons outlined in the commit message. So we’re still stuck with Opera 9.27…

I did get libpng updated, but had “fun” with that as well. So long.

mksh porting

21.10.2008 by tg@
Tags: mksh

If you read the comp.os.minix newsgroup, please have a look at Message-Id <> and help us porting mksh to Minix 3. (Yes I know I replied to a different thread, but this is just because Pine did not want to let me post using a simple ‘C’ompose in the gabeln.)

If you have an idea how I get the OSF/1 V2.0 cc(1) to correctly fail if it cannot resolve functions, or how we can fix Plan 9 or even Syllable Desktop (which is said to have fixed their signal handling in the kernel, however an actual user would have to test it), also feel free to contact me, you can probably help.

Closer, but no game

21.10.2008 by tg@
Tags: bug

On a somewhat unrelated topic, I would like to remind the readers of this wlog that next weekend, winter time will enact, i.e. you’ve got to add only one instead of two hours to the current UTC time to be able to talk to your “nōn-CS” neighbours. I’m adding it here since I know I would forget it otherwise, and I just saw it in a newspaper I read at the imbiss.

Well, I got closer. So close that dual-boot CDs are possible with the new system. So close that I got everything in place to make both baselive and dualive CDs. But makefs(8) let me down and threw an assertion on – how gdb(1) helped me to find out – a deep directory, usr/libdata/perl5/i386-mirbsd/5.8.8/auto/B/C, thinking it adds the same directory a second time to its node tree (aborting thusly to not end up in an endless loop later). However, adding the allow-deep-trees mount option did not help. So I’m stuck.

There are some alternatives. The one which I like the most would be to further change the Sun disklabel to not only look like an i386 MBR to the BIOS, but also contain a “partition table” with only one partition of type 0x27 (MirBSD), starting at the chain sector (24, in our case) and being two sectors in size. The second one would then contain an i386 disklabel. A 4.2BSD FFS filesystem (created with makefs(8) which should work better there) would just be added after the ISO 9660 filesystem containing the boot stuff (and possibly, the /v10 directory, i.e. nothing less and nothing more than the contents of a normal dual-arch install-only CD, plus the Live kernel, which is just a GENERIC with root set to cd0f… since we’ll be using UFS then, we don’t even need /dev on a ramdisk any more). However, I do not exactly know how this would behave if we mix a filesystem using 2048-byte sectors with a filesystem using 512-byte sectors on the same medium and expect it to work both when burnt on a CD using 2048-byte sectors for the entire medium and when put on a CF card / USB stick / HDD / etc. using 512-byte sectors for all of the two filesystems. The Sun disklabel would also have to be adapted, unless we want to hide the i386-live part from it (or are too lazy to show it… but we’re perfectionists, sometimes).

I know from others that they put /usr on an FFS inside a vnd(4) file instead, but this has not only heavy performance issues, I also know our vnd(4) to behave slightly buggy on media with sector sizes other than 512, with OpenBSD having fixed some of that. I would thusly like to avoid it.

Just putting FFS on the CD is not an option either. Oh, there are endianness issues as well, so there’s probably no point in having the i386 FFS slice being accessible by sparc, since people in all of the BSD camps still haven’t understood how uncool they are (our rewritten elf2aout(1) being a prime example that it is not only possible but also highly useful to have such tools do internal endianness conversions as needed).

Now I either need an OpenBSD guru telling me how to accomplish it, or have to experiment. *sigh*

Does anyone succeed in running MirBSD on qemu 0.9.1? No matter if with or without kqemu, it freezes for me after the pciide(4) probe. The sparc issue also has yet to be resolved.

All 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31

MirOS Logo