When doing porter uploads, one must not forget to pass -m"My Name <firstname.lastname@example.org>" to dpkg-buildpackage, e.g. with --debbuildopts for cowbuilder. Thanks Aurélien and sorry to everyone who got the upload mails.
(How are the rules for sponsored uploads? I get conflicting info on these, and indeed, the one I sponsored never showed up on my QA DDPO page…)
I’m almost finished with “sort of re-bootstrapping” Debian/m68k (I can use etch-m68k as well as what was in unstable at the moment as dependencies, so it was not that much, still, 305 binary packages build from 84 source packages, most for unstable (very few for unreleased, with very responsive maintainers, thanks all, who will include the patches in their next uploads) is a bit… including rebuilds with newer versions, more patches, more testing or newer dependencies installed. I’ll probably upload on Sunday evening, as I’ll be off for 2-4 days at least from then (see below). Ingo tried to test on real hardware, but as Murphy wants a hard disc failed… we’ll still try to get something done over the weekend. If you want to have a look, see my repository index (sources.txt contains a sample sources.list file, 0-NOTE.txt some hints, including the right debootstrap/cowbuilder magic and speed tricks). I’ll need to learn how to use LVM and set up a buildd now…
I’ve not been in much of a hacking mood recently – all these visits to the dentist leave me in unrest and disturb my equilibrium. Hence, not much activity even in mksh even if there was need, almost none in MirBSD. This is only temporary, but I won’t attend OpenRheinRuhr, or, if I come at all, it’ll be for socialising only and probably only one day. Benny’s done with his Doctor (in France, no idea whether it’s one in Germany as well) of Chemistry and has returned to hacking some (World of) Google-Go(o) code. I expect MirBSD activity to slowly raise once we can come back. Please accept our apologies.
I’m currently working on something which will eventually amount to a re-bootstrapping of sorts of the Debian/m68k (Linux) port – patches to the Linux kernel, gcc, etc. are prepared (some have been accepted into upstream or the packages already). I will probably have more, once the compile processes finish, anyway (even emulated, it’s slow).
I think that, once I get past that TLS (thread-local storage, needed by eglibc) migration, I will try to find out a list of packages needed for debootstrap (AFAICT: all packages marked Essential, or of priority important, and all marked Build-Essential (for the *-builder variant), and their dependencies (although I’ll substitute sysv-rc with file-rc, which is better and needs less deps)) and pull arch:all from sid, then build the rest myself using a consistent snapshot of sid possibly with patches going to unreleased. Then I can use cowbuilder to make cleaner packages, which can eventually be uploaded (once I get enough to get a buildd running – kernel, bootloader, etc) – binNMUs are way to go here I suppose. I will only upload once it’s self-hosted, installable (seen by edos-debcheck), clean, etc. (i.e. I’ll rebuild all binaries) and probably keep a bootstrap repo around (until m68k caught up) so that unstable (possibly amended by unreleased for a while) will not again become uninstallable, e.g. if arch:all packages change their dependencies (Python, gcc-defaults are some I’ve seen). That bootstrap repository is needed anyway because debootstrap can’t install from two separate repositories (unstable+unreleased for example).
Progress is slow because I try to keep as close to official packages as possible, refuse to cross-compile, and try to produce uploadable if possible packages all the time. Getting patches into packages, so that I can build from unstable, instead of debian-ports.org unreleased, has proven time-consuming and occasionally frustrating as well. Although I would like to thank the people who helped me on the way already. (I am not naming any in fear of forgetting some, but you know who you are ☺) They are among the Debian (gcc, kernel, m68k) and Linux-68k crowd.
(Why does genattrtab in gcc-4.4 take 3½ hours when it took less than half an hour in gcc-4.3 anyway?)
Well, did I mention dentists are sadists?
openbittorrent.com has, apparently, bitten the dust as well. Oh the joy. DHT to the rescue, over the last few days, for those who could do it. I’m now running bttrack.py, yes the original, on eurynome and have (again) reannounced this project’s torrents. Please download the *.torrent files again. And don’t bother asking, I’m not running a public tracker. Clarification: The content is unchanged, only the torrent metafile has changed!
The updated CVS and new RNG code seem to behave well; I also fixed old bugs in the process. I will probably update our main server within some foreseeable future (this would be the ideal time to push out a snapshot again as well, even if it’s “just” netinstall).
People have shown interest in my djbdns patches. Consider forking, even putting it into the base system. I need to solve the problem of the remaining non-v4-transport-capable v6-transport binaries though, I think only dnscachet6 is remaining, so we’ll get only one set of binaries again. Also look for SRV RR patches. I wonder whether someone will code DNSSEC support…
The msdosfs LFN code is also still on my TODO, as are some other things. But hey, at least there’s movement; even Benny, despite being offline, unreachable by phone, etc. commits Google-Goo code. (Hi!)
mksh currently is being reviewed by the Android Security team, who like it on a first look. I’ve already addresses the first concerns even. I might release R39d soonish, also because I’d like a stable release before going on to associative, and since it’s easier to do than prohibit, multidimensional arrays – which have been welcomed in #ksh already…
You might want to update src/sys/net/netisr.h if running #10-stable, or upgrade to the latest kernel. I ran dieharder, and the results look good. The latest RNG subsystem pulls from many more sources and mixes better; I’ll summarise it later probably.
I’m currently working on two very important subsystems to MirBSD: the entropy subsystem arc4random(3), arandom(4), arc4random(9); the cvs(GNU) implementation. That’s why it’s extremely encouraged to not update to -rHEAD right now.
The entropy subsystem receives completely – except arc4random_uniform(3) – rewritten arcfour and arc4random* code (userspace already done) including quite a speedup, and a new structure of the kernel pools and how they interact (also for speedup, but better hashing as well).
Our GNU CVS implementation has received a number of patches from Debian’s, and not only did I synchronise the port with base again, but also created I an (unofficial) “WTF” *.deb package from it, since Debian’s has, as we discovered years ago, some broken (but hey, ISO compliant!) date format.
I’d suggest let me finish doing, unbreaking ;-) and testing it.
mksh was merged into Android (both AOSP and Google’s internal master tree) in the night 24/25th August, and is expected to be the one shell to rule them all, for Gingerbread.
mksh(1) now also has a cat builtin, for here documents mostly. It calls the cat(1) command if it receives any options. The shell is nevertheless smaller than yesterday because of improved string pooling.
There’s another reason to use the MirOS OPTU-16 encoding instead of PEP 383, on which I already wrote: try passing a wide-char filename to a function such as MessageBoxW, or create a filename on a system using wide chars, such as FAT’s LFN or ISO 9660’s Joliet, or one that only allows Unicode (canonically decomposed – ü → ü – out of all things) like HFS+. OPTU-8 at least maps to somewhat reserved codepoints (would, of course, be better to get an official 128 codepoint block, but the chance’s small of getting that in the BMP). Still.
Oh well, the torrents. I’ve remade them all, using one DHT seed node and OpenBitTorrent as tracker and put them on a very rudimentary BT page that will be completely redone soonish. Please re-download them. I currently do not believe f.scarywater.net will return.
Finally, I fell victim to a selling-out and may have just bought a Windows Mobile 6 based phone (Glofiish X650) and an SDHC card and an extra battery with double capacity. Well, at least it’s said to run CacheWolf well. I still would like to have something like Interix, Cygwin, UWIN, coLinux, or maybe some qemu-for-WinCE variant that runs Android, Maemo, Debian/armhf (or armel or arm) at near-native speed (and is usable – the device sadly doesn’t have a hardware keyboard, but it comes with SiRFstar Ⅲ GPSr). It only has 64 MiB RAM, like the Zaurus SL-C3200 and the jesusPhone, though. ☹ Any chance to get MirWorldDomination onto that device as well?
Tomorrow, eight years ago, is the date we now use as birthing point for MirOS. The thing is, we did not really want to create a BSD of our own, fork, or whatnot. We were mostly happy OpenBSD users (really happy before the first eMail exchange with its developers, where Theo de Raadt did indeed stand out but was not the only one – just the one with the authority to deny us), improved it locally and submitted patches and ports. We were flamed for that or, worse, ignored. I begun putting up my “OpenBSD patchkit” on my homepage (back then, at Tripod) and still tried to feed things to upstream and OpenBSD. Then, at some point, Theo de Raadt made it clear he did not want me and the patch kit had grown (from one 4M file into several of them), so I ended up doing a “cvs -d /cvs init” and went from there. Benny’s story is similar – he laughed at me while trying to get ports added to OpenBSD, then discovered his ports were added to the MirPorts Framework and getting commit access there was easier than getting some random developer to commit something of his to OpenBSD. (This trend ended there though… every single person I approached since has become OpenBSD ports committer – I wonder whether they used my invitation letter to blackmail Theo?) It’s often thought that there was a clash of opinions between Theo and me. I think while we might disagree in certain aspects or priorities things should have, in the end we both wanted the same thing, I just was promised to never become a member of the OpenBSD project, so it’s really just “them” being uncooperative. (They (Henning and others) did burn the T-Shirt I gave Theo as a gift some day for making OpenBSD what it was. I won’t comment on that, again, now.)
FrOSCon was a blast. I had two booths of my own – MirBSD and FreeWRT – as well as shares of Debian and Grml. Well, MirBSD was run by Benny and gecko2 because I just didn’t have any time for it, despite XTaran’s help with the FreeWRT booth. All I did was the initial setup of both booths, while at the same time answering about three questions regarding FreeWRT in parallel. Wow. What a little small, open hardware can do to you. XTaran and I had fun and we’ll do FreeWRT booths again; I managed to flash my two FON2100 devices (“La Fonera” – the FON2200 can use the same image, says nbd of OpenWrt) and will fix the port’s remaining few bugs I found; XTaran will try to push the WL-500gPv2 development. The social part was nice as well, although I think the greek restaurant in the city will not be visited by me again. Anyway, if you didn’t attend FrOSCon, your own fault…
Since the BitTorrent tracker used by MirOS is down, here’s the link to the [updated 2010-08-28] current (FrOSCon 2010 Edition) snapshot’s torrent, Triforce as usual. We’ll probably rewrite torrent files for all our ISOs and publish them on the MirBSD website. I’m currently considering OpenBitTorrent plus one or two DHT seed nodes with no statistics. Maybe with webseed. (Need to update the libtorrent/rtorrent MirPorts first, though…) Other options would be different trackers or running one of our own. I will announce the outcome as news entry, once done.
On the plus side, the review process of mksh(1) in Android continues, and I fixed the realpath builtin to behave even more POSIX-ish.
I’m currently putting the finishing touches on a new MirBSD snapshot for i386, targetting FrOSCon. Due to German patent 69429378 (EP0618540) I had to update the sparc kernel (GENERIC and RAMDISK) as well though. For now, mount_msdos(8) LFN support is disabled entirely, but a code rewrite to support LFNs in FAT in a non-infringing way (although for a file with LFN, no SFN will be generated any more then) is planned already (just not done before FrOSCon). This situation, in general, sucks.
The snapshot will, as usual, end up on the mirrors, i.e.
The FrOSCon 5 - 21./22. August 2010 booth plans have finalised, I am rather content:
This is especially good, as XTaran will be shared among (at least) Debian, Grml, FreeWRT; same for me plus of course MirBSD; kimnotyze is FreeWRT but may help with MirBSD; benz and gecko2 probably are MirBSD but gecko2 could help with FreeWRT, tokkee was interested in FreeWRT too… anyway.
Some days, you just love software.
Aug 6 13:55:01 blau firesomething-bin: stack overflow in function VFY_EndWithSignature Aug 6 13:55:01 blau /bsd: signal 6 received by (firesomething-bi:1146) UID(2999) EUID(2999), parent (sh:9059) UID(2999) EUID(2999)
Thus, let me reïterate it for all of you:
Well, now that the Debian Release Managers have sent their freezing bits around… *shudder*… Squeeze is frozen. Well, at least everything I have my hands in has migrated. I’m still… not persuaded. I also can’t decide which looks worse (KDE 4 or Win 7), tending towards KDE 4…
Why does all the horrid software (Solaris, Java™, OpenSSO, MySQL,
etc.) tend to end up at Oracle at the moment? Let me quote from some
Debian mailing list:
>>What happened to the Unix philosophy?
>Modern Solaris engineers
Is that similar to high-speed horse carriages?
My RPM repository has been pimped a bit – I ported some stuff from my DEB repository and updated them in both (rdate(8) and ntpd(8), specifically). Still ought to work more on them, but currently MirBSD base is most important, although I’m dying for mksh associative and multi-dimensional arrays, as well as more sh(1) conformance assorted bug fixes.
Well, there’s a life besides the computer. I’ve taken today off, wanting to hack on MirBSD’s most urgent problems (but probably end up doing that tomorrow), slept long, and will meet with cnuke@ and gecko2@ for Greek style dinner. The latter will almost certainly end up with a long-time work contract at the same place where I run a lot of things already, so congratulations. In the meanwhile, bsiegert@ has almost become a Doctor of Chemistry, and my brother’s finished his Maths and Economics diploma.
Also, I’ve put up the logo of the company where my new dedicated server is hosted; they reduce the monthly fee in exchange for this, so humour me and pay them a visit. They’re IPv6 pioneers, actually. (The server is now installed but not completely set up yet, and I have yet to begin moving services; it’ll be better than the VM eurynome is, but the clock could use the new timekeeping subsystem in the kernel as well as socket send (ÆrieBSD) and receive timestamps as it’s off by 0-1000 ms.)
Speaking of kernel stuff, yesterday I considered moving wscons(4) to UTF-8 again (since everything is CESU-8, we need to take raw octets into account also). I’ve seen OpenBSD begun importing Citrus… *shudder* Anyway, that’s my part, but I’d like volunteers for backporting things like the timekeeping stuff (and possibly more hardware support), and writing a pivot_root like thing (explained on some mailing list already) so we can use ramdisc root to do loopback root.
FrOSCon 5 - 21./22. August 2010 is approaching rapidly. I’m a bit envious at some of the tracks (I mean, really, geocaching (ok, I did the surrounging caches over the last years but still), learning python by means of game programming, etc. really sounds interesting – and I know people who could benefit from a non-kids version of that as well) but this year’s FrOSCon is nothing for me to curse about either: I managed to get both a booth for The MirOS Project (MirBSD, mksh and other subprojects) as well as one for Waldemar’s FreeWRT (although wbx@ – if he comes – won’t join there since he forked his own fork since its conception). Booth staff are, currently: tg@ and bsiegert@ (Developer), gecko2@ (Staff) for MirOS, tg@ and “XTaran” abe@ (Developer), kimnotyze (Hacker) for FreeWRT. (XTaran will probably be helping Debian/Grml too.) This year, it’ll be my job (after 2 years of aptituz) to keep the Altbier-Fraktion watered, I’m thinking one crate of Schlösser Alt and one crate of Hannen Alt?
Have a look at the Program and don’t tell me you won’t come! It will rock! (Except there won’t be Formorer’s Chilli, but that’s no reason, there’s enough other stuff in manageable distance.)
Besides interesting booths and talks, FrOSCon is still looking for helpers who will not only get free entrance but also catering during operation.
Well, I suppose I should be happy that mksh is actually used…
- [tg] Correct shf buffer I/O routines to avoid a memory corruption bug discovered by Waldemar Brodkorb and other bad effects
- [tg] Fix NULL pointer dereference during iteration loop when checking for alias recursion; discovered by Michal Hlavinka
That’s OpenADK (Waldemar’s fork of FreeWRT, which is Waldemar’s fork of OpenWrt), and Red Hat Enterprise Linux, respectively. Popcon in Debian and its derivates is also pleasant.
I could use some help bugfixing this though:
(sleep 3; exit 12) & bgprocpid=$! sleep 6 # background process is done by now wait $bgprocpid # POSIX mandates that, since $! was asked # for, wait is to reply its errorlevel
Somehow, JF_KNOWN is never set – and I can’t debug this with gdb(1).
(There’s also a dashism in some *buntu start scripts that does pretty much the same except it uses “wait %1” there. In fact it doesn’t even seem to use $! – no idea whether we can support that at all in a POSIX shell – which dash clearly isn’t… – without keeping track of background processes forever.)
I’ve got some interesting results using r1.1 of an example test programme (r1.2 got cleaned up and more output) on various systems, regarding ASLR. The 1.1 revision tests everything mksh R40+ will use (except there will probably no larger than page sized allocations) for its LCG PRNG. On OpenBSD (MirBSD, ÆrieBSD) malloc(3) uses in fact mmap(2), which is randomised. (Though -pie doesn’t yet work as it’s supposed to.) Some OSes are better than others… but look for yourself. (Read on to continue, not part of the RSS for size reasons. This wlog entry may be updated – with bumped date – unperiodically.)
tg@blau:~ $ mgcc -static x.c x.c:0: note: someone does not honour COPTS correctly, passed 0 times x.c: In function `foo': x.c:27: warning: function returns address of local variable tg@blau:~ $ ./a.out 0xa9332000 0xaba65000 0xa0ae7000 0xcfbed990 0xcfbed994 tg@blau:~ $ ./a.out 0xa91b4000 0xa02b1000 0xa1602000 0xcfbf8680 0xcfbf8684 tg@blau:~ $ ./a.out 0x9f731000 0x9cb2a000 0xa94ca000 0xcfbf5840 0xcfbf5844 tg@blau:~ $ ./a.out 0x9c2af000 0xa6a0b000 0xa4ce1000 0xcfbefac0 0xcfbefac4 tg@blau:~ $ ./a.out 0xa3b61000 0xa96de000 0xa96df000 0xcfbedcc0 0xcfbedcc4
tg@frozenfish:~ $ gcc -static x.c x.c: In function ‘foo’: x.c:27: warning: function returns address of local variable x.c: In function ‘bar’: x.c:33: warning: function returns address of local variable tg@frozenfish:~ $ ./a.out 0x80b2a20 0x80b2a30 0xb7745008 0xbf985ce0 0xbf985cd4 tg@frozenfish:~ $ ./a.out 0x80b2a20 0x80b2a30 0xb7726008 0xbfb911b0 0xbfb911a4 tg@frozenfish:~ $ ./a.out 0x80b2a20 0x80b2a30 0xb7784008 0xbf83d040 0xbf83d034 tg@frozenfish:~ $ ./a.out 0x80b2a20 0x80b2a30 0xb77e8008 0xbfc0f840 0xbfc0f834 tg@frozenfish:~ $ sid I: [sid chroot] Running command: “mksh -l” tg@frozenfish:~ $ gcc -static x.c x.c: In function ‘foo’: x.c:27: warning: function returns address of local variable x.c: In function ‘bar’: x.c:33: warning: function returns address of local variable tg@frozenfish:~ $ ./a.out 0x80c86a8 0x80c86b8 0xb77c3008 0xbfaa1900 0xbfaa18f4 tg@frozenfish:~ $ ./a.out 0x80c86a8 0x80c86b8 0xb77d2008 0xbfcc0260 0xbfcc0254 tg@frozenfish:~ $ ./a.out 0x80c86a8 0x80c86b8 0xb77c1008 0xbfbe2120 0xbfbe2114
uname: Linux frozenfish 2.6.18-6-686 #1 SMP Fri Feb 19 23:40:03 UTC 2010 i686 GNU/Linux
tg@stinky:~ $ gcc -static x.c x.c: In function `foo': x.c:27: warning: function returns address of local variable tg@stinky:~ $ ./a.out 595f0 59bf0 59d00 ffbefbb4 ffbefb5c tg@stinky:~ $ ./a.out 595f0 59bf0 59d00 ffbefbb4 ffbefb5c tg@stinky:~ $ ./a.out 595f0 59bf0 59d00 ffbefbb4 ffbefb5c tg@stinky:~ $ gcc x.c x.c: In function `foo': x.c:27: warning: function returns address of local variable tg@stinky:~ $ ./a.out 20950 20f50 21060 ffbefb3c ffbefae4 tg@stinky:~ $ ./a.out 20950 20f50 21060 ffbefb3c ffbefae4
mirabilos@stargazer:~ $ gcc -static x.c x.c: In function 'foo': x.c:27: warning: function returns address of local variable x.c: In function 'bar': x.c:33: warning: function returns address of local variable mirabilos@stargazer:~ $ ./a.out 0x800603080 0x800605040 0x800700000 0x7fffffffe62c 0x7fffffffe62c mirabilos@stargazer:~ $ ./a.out 0x800603080 0x800605040 0x800700000 0x7fffffffe63c 0x7fffffffe63c mirabilos@stargazer:~ $ ./a.out 0x800603080 0x800605040 0x800700000 0x7fffffffe62c 0x7fffffffe62c
uname: MidnightBSD stargazer.midnightbsd.org 0.3-CURRENT MidnightBSD 0.3-CURRENT #1: Thu May 27 22:13:45 EDT 2010 email@example.com:/usr/obj/usr/src/sys/GENERIC amd64
(QEMU, thanks to Aurélien! Debian unstable from approx. Jan 2010)
root@debian-mipsel:~ # gcc-4.4 -static x.c x.c: In function 'foo': x.c:27: warning: function returns address of local variable x.c: In function 'bar': x.c:33: warning: function returns address of local variable root@debian-mipsel:~ # ./a.out 0x4aa740 0x4aa750 0x2aaa8008 0x7fa417e8 0x7fa417d8 root@debian-mipsel:~ # ./a.out 0x4aa740 0x4aa750 0x2aaa8008 0x7fc67708 0x7fc676f8 root@debian-mipsel:~ # ./a.out 0x4aa740 0x4aa750 0x2aaa8008 0x7fb68238 0x7fb68228 root@debian-mipsel:~ # ./a.out 0x4aa740 0x4aa750 0x2aaa8008 0x7fc586c8 0x7fc586b8
uname: Linux debian-mipsel 2.6.32-trunk-4kc-malta #1 Mon Jan 11 03:45:08 UTC 2010 mips GNU/Linux
Gentoo GNU/Linux on amd64
gcc-4.4.4, glibc-2.11.2-r0, 2.6.35-rc4 x86_64
0x20cc010 0x20cc030 0x7fef0c497010 0x7fff32148fec 0x7fff32148fec 0xa35010 0xa35030 0x7f575d0e4010 0x7fff0dd7220c 0x7fff0dd7220c 0x1f90010 0x1f90030 0x7f8657107010 0x7fff6116813c 0x7fff6116813c 0x9dd010 0x9dd030 0x7f1eab0a6010 0x7fff3dcc638c 0x7fff3dcc638c
Not everyone does ASLR… but there’s enough variety (and with eglibc’s AT_RANDOM even proper entropy) inside for our purposes. On OpenBSD and MirBSD, we’ll still use KERN_ARND as it’s extremely cheap entropy (code paths checked on both) but not for every call of $RANDOM. On things like Debian/m68k mksh(1) ought to have gained a possibly noticeable speed-up.