Developers’ Weblog

Sponsored by
HostEurope Logo

Developers’ Weblog

All 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31

pcc and Win32

25.11.2008 by tg@
Tags: pcc snippet unicode

I think I’m quite finished with the Unicode stuff. You can now do really weird things like this:

void
今日は(const struct person *$kunde)
{
        printf("Γεια σου %ls!", $kunde->имя);
}

ISO C99 permits it.

On the other hand, pcc on Win32 appears broken, cannot be compiled either natively (e.g. due to use of “long long” and other bogus things which may be C99 but not MS-C90) nor cross from GNU/Cygwin32 (because the Win32 port seems to be orphaned and, even if I try to fix these things, it doesn’t work for me). So I couldn't get it to build a simple MessageBoxW() hello world programme.

I wonder what the reception of my patch will be. I wish support for wchar_t were a release goal, then I could try to tap into that BSD func *smile* but anyway, I was just bored, and we need wchar_t support in pcc anyway, and better I do it than some ISO-2022-JP fanboy. And that above example… well, three more lines of code, so that you can actually write “int blöd;” ipv “int bl\u00F6d;” ☻☺ I think it’s worth it, even if such can confuse the hell out of people trying to read your code.

This thread contains the patches (three versions, until now… use today’s please), in case you want to help testing.

Today, I was sort of bored, so I enhanced something other than our libc, namely pcc, with support for Unicode/UTF and wide character literals and strings. Kind of fun it was.

Still missing is support for \uXXXX literals in multibyte strings (need ucs2utf function for that, which can be taken from our libc again), and, maybe, automatic UTF-16/CESU-8 conversions, if the target has UCS-2 (MirBSD, Windows) and the host doesn’t, or so.

Every single time Benny works on ports/www/firesomething, it sinks in my esteem. Generating crypto keys during fake install is not how things should be done (use a post-pkg_add script!). And I still think it’s only rumours that it had been rewritten after the Netscape® code was opened – it would be better than before then, not worse. I’d rather shoot myself than publish such filth, if, like Benny says, the code had indeed been rewritten. Well, at least it is aware how insecure it is and kills itself:

14:46⎜«benz» Nov 23 16:40:29 mirbulk firesomething-bin: stack overflow in function virtual nsresult nsCryptoHash::Finish(PRBool, nsACString_internal&)

And if he wouldn’t document this in a wlog entry, I do ☻☺

But then, I don’t believe him totally unaware:

14:09⎜«benz» The name "Minefield" is strangely appropriate

galloc and OPTU

22.11.2008 by tg@
Tags: unicode mksh

aalloc was a failure and prone to mis-type; I’ve started on galloc, which is supposed to be even more type-safe and flexible, yet still segfaults on me as well.

In the meanwhile, to not stay bored, I enhanced the Unicode (MirOS OPTU-8/OPTU-16) function suite in MirBSD and hacked an implementation of Plan 9’s Rune functions – dubbed p9¾_*.c due to usage of our “internationalisation like Plan 9, just on the next layer, and within the confines of UNIX®” approach to internationalisation. These (as well as some well-known ones like wcsrtombs(3)) should be OPTU safe, and we now have a macro telling us if a wide character is part of the OPTU Raw Octet codepoint range (in the CSUR PUA assignment).

I’ve contacted Bruno Haible again because his libutf8 misses the Unicode Title case (complementing Upper and Lower case)… I’ve got an idea how to implement it but would like to double-check with him to ensure nothing breaks. The Plan 9 functions need it (these two are currently implemented as stubs that just throw ENOCOFFEE).

The number of manual pages also raised…

I built XFree86® with a (slightly beefed up) Reiser CCCP in use as sole C Præprocessor now… and have yet to notice failures in operation. Now we’ll have to find out what else stuff is there depending on the existence of /usr/bin/cpp, possibly switching it, like the <sys.mk> mirmake(1) file, to ${CC} -E - (which is not 100% compatible, because cpp can also read from stdin without the dash, whereas, with the dash, neither can read from only a file). X11 etc. can be switched to /usr/libexec/cpp then.

aalloc and COFF

17.11.2008 by tg@
Tags: mksh

I got (E)COFF executables to work, even though I cannot produce them except with a hand-crafted binary yet. Maybe these produced by kencc work now.

I didn’t get the time or nerve to continue hacking on aalloc today… the whole pointers thing in C is so useless. If I could use assembly, it were so trivial. Anyway, the TPtr data type will probably have to get lost, and I plan on using more temporary local variables for overview.

Besides a beer discussion, IRC channel today featured plans on getting a temporary setup of fast enough machines for crossbuilding OpenJDK; replaced confirmed that someone had success in doing what I planned, although the nōn-free JDK 1.5 only. We’ll see…

small updates

12.11.2008 by tg@
Tags: security

/MirOS/current/i386/fixes10.ngz holds a new libm, with which you can use the three new binary packages (lzma, llvm, llvm-gcc4.2) I’ve just placed in /Pkgs/current/i386/ for your fun. These all have been compiled using mgcc-10uA3, as llvm-gcc4.2 is not up to the task.

The new “aalloc” area-based omalloc-employing pool allocator is still unfinished; I need to make cookies work and perhaps add more ifdefs to make it smaller for MKSH_SMALL users (installer, FreeWRT, …). However, feel free to read it and send me comments. I admit it might want to be cleaned up a little, but first it’s got to be finished. Do not use it yet; it’s intended for both mksh and libobjfw though. Also, I’d like to hear of possible optimisations or more security stuff one can throw in; the current scheme is the compromise looking best to me. Remember that it was written with mmap malloc (and omalloc – Otto Moerbeek zijn malloc) and malloc.conf(5) MALLOC_OPTIONS guard/junk/protection in mind.

compilers

10.11.2008 by tg@

First off, the X.509v3 certificate for gzsig(1) has been renewed. While, strictly spoken, this is not necessary, it eases validation.

MirPorts now not only contain ports for various compilers such as pcc (which does not suck anymore), LLVM+Clang (which does suck, because it’s written in ugly CFrustFrust) and llvm-gcc 4.2, it can also peruse them to compile packages:

  • $ mmake USE_COMPILER=system # mgcc (C, C++), Default
  • $ mmake USE_COMPILER=pcc # pcc (C only)
  • $ mmake USE_COMPILER=llvm # llvm-gcc (C, C++, Fortran 77/9x)

While clang cannot yet be used there really, the Build.sh script of mksh can use its “ccc” compiler driver (which however is experimental) as well as llvm-gcc to build mksh(1)… to native code, or to intermediate bytecode (with link-time intra-module optimisation and final assembly into native code).

The LLVM Interpreter lli also, surprisingly, works. No patch required.

This (LLVM) is for MirOS-current i386 only; Darwin probably just needs to be tested, whereas MidnightBSD and OpenBSD require some more work.

TODO: there is an LGPL’d lzmadec binary, C only even; do something with it, possibly to ease use of .clz binary packages. Recent storage needs have become utterly ridiculous.

website fun

09.11.2008 by tg@
Tags: security

Now that the website is running on MirOS httpd(8), we can have some fun… SSL and such. The following picture may not be displayed if you use a nōn-rsync’d mirror, but can be worth the fun:

connection info

An ASCII connection info version is, of course, available as well, but images can be embedded more easily. For example, if you were to read this wlog entry via SSL (link provided courtesy to readers using Planet Symlink), the information would differ.

And if you still get warnings about an unknown certificate authority, CAcert provides them for downloading into browsers such as M*zilla Firef*x whose idiotic development team opposes against them. For what it’s worth, unchecked SSL is still better than no SSL at all (especially for entropy ☺) unless you think encrypted = authentic identify.

WTF we have a symlink

01.11.2008 by tg@
Tags: bug snapshot

Much closer to a new DuaLive CD. Reading Ecma 119 (ISO 9660), El Torito, and IEEE P1282 (Rock Ridge). Fixing bugs in makefs(8) as if there were no tomorrow. Even ifdef’d MirBSD deps, so that they™ can take everything back. Finding bugs in the kernel (one harmless, one deadly), and amusing myself over certain students. (Bit 0 == 2⁰ == 0x01 not 0x00…)

What’s left? Actual testing (qemu, real hardware), and I suspect that the RR_MOVED creation code does not split the directory correctly at 2048 byte sector boundaries, like all the others. (How many damn academics did work on this part of the code, anyway? Modularisation sucks, major arse.) Being paranoid and using calloc(3) has saved me from most other illnesses since.

Leider mußte ich den Federroten heute alleine trinken, aber immerhin konnte ich wbx@ überzeugen, sich zur mksh zu bekennen – sie kann UTF-8 und überhaupt Umlaute in der Shell, schön mit nur einmal Rücktaste drücken und so. Und seinen GNU screen hab ich ihm auch noch gefixt. Dafür bin ich den Sharp Zaurus SL-C3200 (und somit auch meine einzige OpenBSD-Testkiste) wieder los… eventuell nutzt er ja MirPorts, denn unser screen weiß, obs in einem Unicode- oder 8-Bit-Terminal rennt, wsconfig/chkuterm sei dank.

Warum sagt mir eigentlich niemand, daß mor…^Wheute die Geschäfte zuhaben? Ich bin ja mal voll für src/usr.bin/calendar/calendars/de_DE/calendar.ladenöffnungszeiten.$bundesland (das geht ja sonst mal gar nicht)!

I discovered that the CD drive in my docking station – namely a cd0 at scsibus0 targ 0 lun 0: <HL-DT-ST, DVD-ROM GDR8083N, 0K04> SCSI0 5/cdrom removable – neither is removable (the docking station is not exactly hot-pluggable) nor can it write. It can read various kinds of DVDs, CD-ROM, CD-R, CD-RW, but not burn any ☹ And since the lid seems to be custom-fit, I probably cannot simply take odem’s DVD-ROM/CD-RW slim drive either. That said, the sparc still cannot read CD-RW either… must discover some CD-R media…

I had a good laugh at J�rg “schily” Schilling learning English: &dp[14] /* Hunreds of a Seconds */ – That he doesn’t grok licencing is already well-known, so no need to bash that again.

new developer

26.10.2008 by tg@

Adam “replaced” Hoka, who has advertised and bug-spotted (and fixed) in the past, begun a Haiku port of mksh. He has been given CVS commit access for that purpose (possibly extended later), and as such, is now the 8th developer (counting both active and passive ones) to The MirOS Project.

Apparently, the Haiku port now compiles but does not run. Possibly another buggy operating system to add to the list.

I somehow end up fixing pcc each time I try to build it. But code size on i386 has much improved recently, although still nowhere near gcc ‘-Os’ already much better than its ‘-O2’ counterpart, both gcc3 and gcc4. I’m still waiting for more bug fixes, and more of my fixes integrated into upstream, before updating the pcc mirport though.

While I removed the use of __typeof__ from the mksh(1) source, SUNWcc still explodes on it unless we use HAVE_EXPSTMT=0 *sigh*

There are new “hosted” subprojects in our CVS repository. These are for each committer to place his/her stuff into, which, in contrast to other stuff in the contrib module, does not necessarily need to have something to do with BSD, Unix, shell, etc. However, existing mismatched code (the .exe files of my keyboard layout and Benny’s polymers code) will not be moved, because that doesn’t make sense with cvs(1).

Today, my health has been somewhat better, but I still feel as if I had a cold but hidden, so it does not impact me that much but I cannot lose it either. Headaches started at about 20³⁰, which I consider good, even though I now cannot comply with Jonathan’s wish of having a look at llvm-gcc for spotting a bug in libobjfw.

While I could use newfs(8)+vnd(4)+mount(8) or makefs(8) with a fixed block size for a live CD, or a ports version of mkisofs, or create the image on Darwin, I do not think so. The next snapshot will just not be a Live CD at all. Later, guys. Maybe TNF helps fixing it.

mksh also did not have the newest nifty features, such as <(cmd), applied, due to my ongoing illness feeling and added headaches all of this evening. However, I did finish the OSF/1 V2.0 (MIPS) port, and tomorrow there will be a proper announcement of R36, including ports, source RPMs, etc.

Many thanks to DEChengst from the #!/bin/mksh IRC channel again for giving me access to such machines.

I built LLVM+clang on Debian Lenny, just to make it crash *grin* due to an internal assertion… in mksh(1)’s source code. Gah. Anyway, got the problem reported. I wonder if I should remove Objective-C from the base gcc since it’s not just almost but really unusable anyway (same for Ada unless it can be used to build llvm-gnat-4.2). Jonathan gave up on the GNU runtime for good, or so he told me.

Today I also tried to get a neighbour’s fax-scan-print-copy combo running. Ends up that both the new phone line he bought for it was faulty (I gave him a replacement from my Attic), the AVM Fritzbox has… issues, and when I finally got my own fax from anno 1985 to work, and his Officejet not, we belive that either the ISDN voltage (it was accidentally plugged next to the NTBA first) or, much more likely, a recent thunderstorm b0rked it. So much for high-tech. My low-tech works.

All software sucks…

22.10.2008 by tg@
Tags: bug

I did not get much further. While I had the brilliant idea to embed a native little-endian BSD disklabel inside the first 512 bytes of the first-stage sparc bootloader (as the code searches for it anywhere in sector #1 inside the partition – or disc, if no MBR partition found – aligned to a long boundary), because an MBR partition would fake about two slices in the Sun disklabel which… could cause issues, the software I wanted to use to create a testcase has… bugs.

I ended up creating an install-only ISO and wanted to put the rest of the files into a 4.2FFS filesystem with makefs(8). However, that one did not want as I do, and that for quite a while. Cheers to TNF! grml…

The update of Opera to 9.61 also failed for reasons outlined in the commit message. So we’re still stuck with Opera 9.27…

I did get libpng updated, but had “fun” with that as well. So long.

mksh porting

21.10.2008 by tg@
Tags: mksh

If you read the comp.os.minix newsgroup, please have a look at Message-Id <Pine.BSM.4.64L.0810202029070.28869@herc.mirbsd.org> and help us porting mksh to Minix 3. (Yes I know I replied to a different thread, but this is just because Pine did not want to let me post using a simple ‘C’ompose in the gabeln.)

If you have an idea how I get the OSF/1 V2.0 cc(1) to correctly fail if it cannot resolve functions, or how we can fix Plan 9 or even Syllable Desktop (which is said to have fixed their signal handling in the kernel, however an actual user would have to test it), also feel free to contact me, you can probably help.

All 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31

MirOS Logo