Developers’ Weblog

Sponsored by
HostEurope Logo

Developers’ Weblog

All 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36

my first advisory

14.04.2008 by tg@

XTaran mentioned that there is a Secunia advisory for mksh: SA29803. Wow. This would then be my first one. But people really cannot read: the advisory states it appies to “Secunia Product #18328: MirBSD Korn Shell 3.x” – WTF? I mean, version numbering surely is difficult these days, with Linux and related software often having four or more decimal points(!) in them… but I reported it to them using their web site form.

I don’t know why Secunia made this advisory, but I suppose the Fedora person has his hands in there, as he told me yesterday that he needs an advisory for marking the update as security fix in their package system. I told him that this is not a lack of wanting to document, or even lack of communication (skills), but that I merely don’t know how this kind of issues is usually handled “out there”. I haven’t dealt with GNU/Linux any more since I started using OpenBSD back then in 2.9 times, and then, this procedere wasn’t widely used.

Ah, speaking of OpenBSD. They are sometimes not even at fault when a bug report is mis-communicated, even if some people don’t believe it. And they don’t do the major/minor game either. They just still have the decimal dot for hysteric raisins.

Actual user feedback matters. My fork (MirEwe, just to annoy Jonathan) of the Ewe VM now has a fix, or rather workaround: when opening serial devices, “/dev/” was prepended to the device name. Now if users enter “/dev/ttyUSB0” (leenocks) things don’t work – but this is hard to debug. MirEwe now warns and does the right thing. The warning is in there for portability ☺

Ah, and: I could finally log my first (locationless, though) TerraCaches. That makes me an active user of all known platforms. Even if it cost me an entire night of re-reading a book I knew from a looong time ago. But actual caches will follow, and I’m even already planning to plant some myself.

Basel is getting nearer. I’m still not too happy with the option to move away from here. We’ll see – at least I’m going to take a big chance when I see one, and peek into how it is over there. Even if Benny is on Malle.

current state: annoyed

13.04.2008 by tg@

How can a bug like this be unnoticed for so long, while the two source code lines in question were specifically touched and diffs reviewed by big names such as deraadt?

Of course it’s me who has to fix longstanding bugs from 386BSD, VIA C3 AES data corruption bugs in the OpenBSD kernel, LP64 bugs in OpenSSL, etc.

a few news items

12.04.2008 by tg@
Tags: geocache

As you can read, quite a few new versions of our portable software has been released. Well, sort-of-portable, but for nroff I plan on improving, and MirMake 2 will be a lot better too, kinda like mksh.

Geocacheing continues: OpenCaching.de The Frog Site Rare Jewels Broken Webpages²
Even Jonathan got hooked now. We found another (his first) two today.

As we’re sort of a big family, you’ll occasionally find German-language postings here. Don’t wonder. We have people who especially read that.

Ah, and while we’re at stats bragging:
BOINC stats
These were made on gecko2’s Intel Mac (Darwin), my laptop (MirBSD), and a few even on hephaistos (GNU/Linux). MidnightBSD needs to use brandelf, thus execution of Linux/i386-ELF binaries fails or, after branding, checksum verification (which BOINC does) fails. Sucks to be FreeBSD derived.

I’ll be in Switzerland (Cōnfœderatio Helvetica) next week, while Benny happens to be on vacation on Mallorca (hopefully not near the war zone called Ballermann). Development may slow down a little due to that (no more 100+ CVS spa^H^H^Hmails per day, yay!) but will not stall.

Willkommen, Schweiz!

12.04.2008 by tg@

Da wir ja alle eine große Familie sind, möchte ich hiermit Grüße (Grüsse?) an die Schweizer Leser, die dieses Weblog (nein, ist kein Blog) auf Planet Symlink entdecken, senden.

Ad auditorēs qui nōn possunt legere linguam germanicam: Salvete amici Cōnfœderationis Helveticæ!

Ich bin übrigens Mitte der Woche in Basel zu Besuch, Benny ist leider zur Zeit auf Mallorca, Urlaub machen. (Hoffentlich fernab vom Kampfgebiet, äh, Ballermann und so.)

I haven’t written here for a while, but I just want to get one thing said. I might have done what Sun feared first thing after I got a Java™ VM (even if it’s only single-threaded Ewe for embedded systems) running stably on MirOS… it now has a native method arc4random_pushb(3)… yeah!

Entropy is determining a lot of my life these days anyway. For example, accidentally sleeping too few, too much or at weird times, being phoned by random people, talking with Vutral about further possible improvements in our RNGs, being asked by CAcert.org if we are suitable OS for their high-security boxen (almost, but we’ll fix the missing parts, and some they’ve got to do themselves as no off-the-shelf OS does), and reporting in huge masses of entropy to CAcert.at Research Lab – some 128 MiB samples with Firesomething “Bon Echo”, as Opera ISE’d out and Lynx just first ate up all CPU then none at all any more without monitorable activity…

Ah, and BOINC is running stable on MirBSD. MidnightBSD has some issues, mainly due to the fucked-up FreeBSD kernel and brandelf.

On Firefox Updates

02.04.2008

When Mozilla Firefox(TM) aka www/firesomething version 1.5.0.10 came out over a year ago, I tried porting it (of course). However, not only did the build not finish, it filled the entire filesystem before failing. At the time, I was updating from something like 1.5.0.4 so I did not know which version exactly caused the problem. Thus, I made incremental updates up to 1.5.0.8, where I seemingly lost interest. Now, with a faster build machine, I continued the updates, discovering that 1.5.0.10 is really the version that fails.

A diff between firefox-1.5.0.9-source.tar.bz2 and firefox-1.5.0.10-source.tar.bz2 is 10 MiB. Most of that is in CVS directories, which are included in their fucking releases, believe it or not. If you leave out those changes, the diff is still 5 MiB and 155000 lines, of which 90% are in security/. Mind you, the whole source tarball, from calling the directory mozilla instead of adding a version number to including CVS directories to using several different build systems, leaving unused configure scripts in the tree, etc., positively REEKS of a flagrant disrespect for those that build from source. All this seems to be meant to encourage you to use official Mozilla builds—which is fine if you happen to be on one of the few supported platforms: Windows, Mac OS X, and Linux i386. We are not.

Anyway: in their infinite wisdom, the Firefox developers chose to upgrade the included NSS libraries (Netscape Security) from 3.10.x to 3.11.4 in a minor security update. This version sports extensive internal restructurations—another nice way of saying "fuck you, porters". Thank you, Mozilla project. Of course, the NSS update is not explicitly mentioned in the release notes.

This tech note says: "The low-level freebl cryptographic code has been separated from softoken on all platforms. Even on platforms for which there is only one implementation of freebl, there is now a separate freebl shared library. The freebl library implements a private interface internal to NSS." This new library is the core of the problem. After the NSS libraries are built, they are cryptographically signed by a program called shlibsign which, in turn, dumps core, generating a 3 GiB core dump in my case!

I had suspected a problem related to our security features, especially W^X. The page about Building on Fedora Core 5 says: "For those with SELinux in enforcing mode, you are likely to run into problems both with the shlibsign during the build process and with the running the final build related to SELinux denying execmod permission ..." However, with some difficulty, I managed to build a debug version of everything for analyis with gdb. During the start of shlibsign, one of the init procedures loads the native freebl (?) module, which promptly loads itself. On and on, until memory exhaustion.

This comment in security/nss/lib/freebl/Makefile provided a clue to the solution:

# The blapi functions are defined not only in the freebl shared
# libraries but also in the shared libraries linked with loader.c
# (libsoftokn3.so and libssl3.so).  We need to use GNU ld's
# -Bsymbolic option or the equivalent option for other linkers
# to bind the blapi function references in FREEBLVector vector
# (ldvector.c) to the blapi functions defined in the freebl
# shared libraries.
ifeq (,$(filter-out BSD_OS FreeBSD Linux NetBSD, $(OS_TARGET)))
    MKSHLIB += -Wl,-Bsymbolic
endif

Adding MirBSD to the OS list fixes the build. And it works!

Now, firefox 1.5 is old and unsupported. So why was this update important? It makes further updates possible. The same bug was holding up my long-finished firesomething-2.0 port. I already have a package for firefox 2.0.0.13 but the port needs just a little more work. I am also confident I will be able to provide a working port for firesomething 3 when released.

Wow, my first posting in this new weblog. My laptop, a new MacBook Pro, spent about 10 days over easter in order to really do a MirPorts bulk build. 1503 binary packages for MirOS #10 have been uploaded to the mirrors. This has also been an occasion to review some largely untouched parts of the tree. Most of the problems seen are related to missing distfiles, changed download URLs, etc. Some packages did not build during th bulk build but when run non-recursively, they worked.

Some bugs in the resulting packages remain: You must manually install the expat package for most of the stuff to work. gtk+ insists on writing to /var/db, which makes it unsuitable for AS_USER builds. firesomething only when works as firefox. Still, this gives you a wide variety of pre-built packages for the new release. Enjoy!

I seem to have a sort of fan club, mostly related to mksh, but also to jupp. Interesting.

Benny has built 1503 binary packages, amounting to about 954 MiB of data. Uploading…

We just got an email about Metalinker, which looks interesting enough to try, from its primal developer Anthony Bryan. We now have it ☺

This night, my internet connection failed every so few minutes (returning a PAP failure, despite me being in midst of a session). I took the chance to upgrade herc’s software (kernel, userland, a few packages) and spotted that sendmail(8) bug. I also took the chance to rebuild the two RAID 1s (one for /, the other for /MirOS and the CVS repositories), which took me a while and about a dozen attempts, but finally I succeeded.

Other than that, I didn’t hack much. I walked about 5‥6 km (single way) to buy myself a Döner though ☺ bad weather doesn’t have to mean stay inside.

We should be at way more than 1500 downloads in total by now… 1405 on the Germany 1 mirror, more than 100 on Germany 2, an unknown lot on Japan, more than 60 BT downloads from me and friends… that accumulates quite.

Happy Spring Equinōx to everyone!

I merged the crypto improvements to HEAD: we now have improved Rijndael CBC code for UVM swapencrypt, and this code uses the VIA C3/C7 PadLock™ ACE if existing – whose code contained a now-fixed data corruption bug.

Next on the TODO list is: make vnd(4)’s encryption use the same code if AES is chosen as cipher algorithm. Allow selection of Blowfish (stay compatible), AES-128, AES-192, AES-256; other algorithms may follow. We will need a new keyfile format and stay backwards-compatible, but this is not a problem.

We got a PUA assignment from U+F900‥F97F (tentatively) for our encoding proposal, and should use “one of the various non-characters” for the NUL encoding. But: “Emoticon U+FDD0 is actually Unicode for the eye of the basilisk…” – U+F000‥F7FF are now reserved for Linux’ straight-to-font map (and some subranges are used by Windows® and Mac), and F800‥F89F for Mac (and possibly, Linux). – Although I got scolded again for chosing a 16 bit wide character type, I believe this compromise with all its good and bad sides is the right way to go for us at the moment. As for how to codify this, I still do not have a final answer, but I think using wrappers for SUSv3 functions which cannot fully support the proposal is a good idea. Most use cases should work with the SUSv3 functions, anyway.

I still haven’t ported mksh to BSD/OS 3.1, OSF/1 V2.0 and Ultrix V4.5…

I’m impressed – 1377 full downloads from the Germany 1 mirror, add to that the partial downloads (wget -c or so-called “download managers”), these from the Germany 2 and Japan mirrors, BitTorrent (which is 36 times from myself, and a couple of dozen times from my friends, and even more from the unknown peers) and you won’t think of us as irrelevant any more.

I’ve hacked on the crypto improvements branch again. I decided that getting tear up and running would be my priority target. It turned out to be a great way to hone my programming skillz as well: I got a lession about pointer aliasing.

Except the actual VIA C7 part, I now tested it quite well on i386-qemu and sparc, where the latter took about one third the time compiling… the new CPU and RAM pay off.

Oh yeah, and hacking on stuff always points out unrelated bugs… such as NO_GZIP=Yes for kernels not working, or <bsd.lkm.mk> being out of date (I had to MFC that, even). And that src/kern/z needs tender care – the transition is still not finished even for zlib.

I hope the discussion about our charset/encoding proposal will find a solution… I feel like constructing a bikeshed, as you get about that much feedback there too ☺ Benny said he doesn’t grok the SUSv3 functions, like mbrtowc(3), enough to follow – and I can totally understand that. Maybe a Unicode guru person like Markus Kuhn can help – I asked him.

My primary SPARC build box demo now has a HyperSPARC 150 MHz ROSS CPU, instead of a SuperSPARC 75 MHz, and 512 MiB RAM instead of a mere 128 MiB (and with that, twice the RAM of my primary i386 box).

But then, I got three SCSI HDDs from the same stone age, and each of them has a different connector. This sucks.

All 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36

MirOS Logo