mksh-current has just gained an experimental recursive parser for command substitutions, fixing RedHat BZ#496791 and decades-old complaints about the pdksh codebase, compared to AT&T ksh93. (GNU bash could also do the example, but not some other things mksh(1) parses fine now.)
This means that things like the following work now.
# POSIX, should “always” work echo $(case 1 in (1) echo yes;; (2) echo no;; esac) # POSIX optional, works now in mksh, works in GNU bash echo $(case 1 in 1) echo yes;; 2) echo no;; esac) # GNU bash seems to choke on comments ending with backslash # a comment with " ' \ x=$( echo yes # a comment with " ' \ ) # No non-recursive COMSUB parser can pass all of the above # tests and these below at the same time (some extensions) echo $(typeset -i10 x=16#20; echo $x) echo $(typeset -Uui16 x=16#$(id -u) ) . echo $(c=1; d=1 typeset -Uui16 a=36#foo; c=2 typeset -Uui16 b=36 #foo; d=2 echo $a $b $c $d) # the ‘#’ is especially tricky, that’s why the above cases
Next on my TODO is the complete rewrite of the read built-in command, as well as its documentation. I think that the (reduced) goals for mksh R40 will have been met by then, except porting to LynxOS and MPE, but we’re working on it, and re-testing Syllable and Plan 9). Of course, a release implies testing on a lot of the supposedly supported platforms, so it won’t be out “immediately”. Though, associative arrays have been removed from the R40 goals, so that I can at least get a new release out. Note that Debian and OpenSuSE Buildservice users have been provided with somewhat well-tested mksh-current snapshots for a while already, and Gentoo users can use the “live ebuild”; there’s always compiling from source too…
Warning: this is a rant against BSD (specifically FreeBSD®, but don’t let me get started on DragonFly, who think it’s wise to drop all shells except ash from the base system and rely on pkgsrc® – yay let’s compile a dozen packages just to get a shell with tab completion, not to mention boxen with no network access – for the task – although others seem to go into that direction too…; you know, there’s BSD, and then there’s FreeBSD…) – don’t like, don’t read.
If you want to change something in the BSD world, you gotta fork your own BSD – no other way around the thickheads. Ok, back then, I ran into a particularily thick one, but others tend to not be much better. Users share the thickness. If you want to change something in the GNU/Linux world, just make a package, have someone upload it, prod (or pay, Hanno got a Radler) people to do it, or just upload it yourself.
At the BSD booth at FOSDEM, despite me bringing the Windows® Mobile 6 Professional devive, strictly for Geocaching mind you, Macintosh boxen had a share of more than 50% – I didn’t manage to tip the scale. At the Debian booth, almost everyone had a “I want to buy a new laptop some day, but it just keeps on working and doesn’t break” pre-Lenovo IBM laptop. No hyping of Google either. (Last year’s CLT saw BSD people advocating pro-Schily – the guy with the broken encoding in his name – shockingly.)
Honestly, tcsh, FreeBSD® people? Sorry. While I agree that there is merit on having the same script and interactive shell, as someone has pointed out (copy-paste examples into the command line), there’s those zsh users who use mksh or GNU bash for scripting. Or just POSIX shell. And that’s with an interactive shell which can be used for scripting. On the other hand, the C shell (both csh and tcsh) cannot.
And what’s with pretending the accent gravis is non-combining, called “backtick” (such a thing does not exist); and advocating it? Sorry, if your csh/tcsh doesn’t handle the POSIX $(…) you should just drop it. (By the way, there is a convention that example command lines are prefixed with % for csh and $ for sh (or # but we write $ sudo instead, these days). Use it. Or leave it. If you have examples that substitute another process’ output, be specific.) It’s funny to see how one person tries to defuse my arguments against csh by telling me “it’s just an interactive shell”, while the other argues that people copy-paste between them, to which that was my response. Read the thread!
And please, get your facts right. “I would prefer that the standard shell be at least Bourne-compatible.” You don’t want Bourne (“^” instead of “|” for pipes), you want POSIX. That GNU bash is called the Bourne-Again Shell in one of their usual semi-bad puns doesn’t help the global perception of such things any. Also, the root shell and /bin/sh are disjunct.
(Plus, why change the root shell, use sudo(8), plain and simple.)
ObNote: in jupp (should I package that
for Debian, btw? rather upload, packages are ready…) the ‘`’ key is used
as præfix for Ctrl-X (`X) or to directly enter numerical (decimal,
octal, sedecimal/hexadecadic) ASCII, 8-bit or Unicode codepoints. Yay!
And even the FSF has seen the light; for a few releases already, GCC uses “'…'” instead of “`…'” for quoting in messages, even without locales. Great job there! (LC_MESSAGES=en_GB.UTF-8 usually works, too, though.)
ObDisclaimer: I have an (yes, Google…) Alert on the word “mksh”, so I know when it’s being discussed. This obviously includes certain fora. Also, I’m a shell implementer and bound to know a certain amount of details. Plus, mksh’s build script runs with pretty much any Bourne/POSIX/Z Shell which has functions and not too many bugs. I wrote it. Go figure. No lowly trolling.
FWIW, mksh(1) has the cat(1) builtin both because Android has no cat(1), and as speed hack. Almost all other shells have worse speed hacks, like a printf(1) builtin. And recently, builtins have become direct-callable, so this actually reduces the overall system footprint. (Its inclusion also provides for some other possibilities, internally.) And as two final side notes, if you haven’t seen this: determine which shell we are run under (CVS) and I still offer a prompt conversion service (send me any GNU bash or oksh $PS1 and I’ll send that to you in mksh(1) syntax – optionally with adjustments/improvements, like cwd uses only up to 1/3 of screen width).
To reproduce, I just uploaded mksh_39.3.20110218-1.dsc and you can run DEB_BUILD_OPTIONS=mksh-static=klibc,dietlibc,eglibc dpkg-buildpackage -rfakeroot to verify it, once you have mkstemp(3). (I will probably send a smaller implementation of that in, later.) I have that and the open fix and the m68k patch applied, nothing else… where did my bug go?
ObQuestion: what’s the legal (copyright/trademark) status of the Atari logo (the one in rainbow colours, with three things going up, right and left “leg” looking like an umbrella stand’s)?
I just need to work more on bilocality. While I did find two geocaches, one at the South/Noon Train Station (taalverwarringen…), one in the buurt of the University, I did manage to miss the AW building completely and utterly. Wow. Except, that Haiku guy came over to talk for a bit (nice). And I drew. An Atari logo with swirl, for that weird stuff I recently have been found doing.
More mksh-current news coming soon, stay tuned. In the meanwhile, I met bonsaikitten IRL (at FOSDEM, yes, too) who kindly made a “live ebuild”, i.e. a source package building -current.
Finally let’s say a big thank you to the person mostly manning our booth, gecko2, and to Benny for talking to people, getting That Other Packaging Thingy working, and pimping the website a bit.
Dear Opera Software A.S.A.
It’s nice that you employ SecureAPT for your package repository, however, the effect is slightly lost by you replacing the key each year, never signing any of them, and putting them up on an http but not https site.
Update 18.08.2011 – please refrain from putting a file /etc/apt/sources.d/opera.list inside your binary DEB packages, as well. As a system administrator, we may very well have mirrored the binary packages, and do not want accesses to external APT repos, for a shitload of reasons. Honestly. (Lintian could warn about it…)
If you don’t get the message, please contact me. Or any of my fellow Debian Developers. Thank you very much.
❧ Who’s not? ☺ Same procedure as every year.
(okay, lolando prefers skiïng but…)
Anyway. A cow‐orker told me that Belgium again/still has no gouvernment, and they have been asked to grow out their beards until they do. I found “evidence” on the ’net but won’t link it here, also it’s on German… anyway. Let’s all join in. (Besides, I now have an excuse to not shave, maybe even my grandmother will accept this one…)
My Debian/m68k stuff is coming around nicely, but I still haven’t gotten around to do everything planned, plus I need to grow a new kernel and eglibc, after the latest uploads, and the 2.6.37 based one panics. Also I’ve got to take care to not overwork myself. (And make a MirBSD ISO for FOSDEM.) But hey, it’s been not working for some time and better now. And slow anyway ☺ yet we’re progressing. Does anyone know how to debug that a C programme only calling res_init(3) segfaults?
Benny is apparently not just working on making NetBSD® pkgsrc® available on MirOS BSD (picking up my work from 4+ years ago) but also replacing The MirPorts Framework with it. Sad, as I got a request for a gajim MirPort over a cocktail just this evening…
When doing porter uploads, one must not forget to pass -m"My Name <email@example.com>" to dpkg-buildpackage, e.g. with --debbuildopts for cowbuilder. Thanks Aurélien and sorry to everyone who got the upload mails.
(How are the rules for sponsored uploads? I get conflicting info on these, and indeed, the one I sponsored never showed up on my QA DDPO page…)
I’m almost finished with “sort of re-bootstrapping” Debian/m68k (I can use etch-m68k as well as what was in unstable at the moment as dependencies, so it was not that much, still, 305 binary packages build from 84 source packages, most for unstable (very few for unreleased, with very responsive maintainers, thanks all, who will include the patches in their next uploads) is a bit… including rebuilds with newer versions, more patches, more testing or newer dependencies installed. I’ll probably upload on Sunday evening, as I’ll be off for 2-4 days at least from then (see below). Ingo tried to test on real hardware, but as Murphy wants a hard disc failed… we’ll still try to get something done over the weekend. If you want to have a look, see my repository index (sources.txt contains a sample sources.list file, 0-NOTE.txt some hints, including the right debootstrap/cowbuilder magic and speed tricks). I’ll need to learn how to use LVM and set up a buildd now…
I’ve not been in much of a hacking mood recently – all these visits to the dentist leave me in unrest and disturb my equilibrium. Hence, not much activity even in mksh even if there was need, almost none in MirBSD. This is only temporary, but I won’t attend OpenRheinRuhr, or, if I come at all, it’ll be for socialising only and probably only one day. Benny’s done with his Doctor (in France, no idea whether it’s one in Germany as well) of Chemistry and has returned to hacking some (World of) Google-Go(o) code. I expect MirBSD activity to slowly raise once we can come back. Please accept our apologies.
I’m currently working on something which will eventually amount to a re-bootstrapping of sorts of the Debian/m68k (Linux) port – patches to the Linux kernel, gcc, etc. are prepared (some have been accepted into upstream or the packages already). I will probably have more, once the compile processes finish, anyway (even emulated, it’s slow).
I think that, once I get past that TLS (thread-local storage, needed by eglibc) migration, I will try to find out a list of packages needed for debootstrap (AFAICT: all packages marked Essential, or of priority important, and all marked Build-Essential (for the *-builder variant), and their dependencies (although I’ll substitute sysv-rc with file-rc, which is better and needs less deps)) and pull arch:all from sid, then build the rest myself using a consistent snapshot of sid possibly with patches going to unreleased. Then I can use cowbuilder to make cleaner packages, which can eventually be uploaded (once I get enough to get a buildd running – kernel, bootloader, etc) – binNMUs are way to go here I suppose. I will only upload once it’s self-hosted, installable (seen by edos-debcheck), clean, etc. (i.e. I’ll rebuild all binaries) and probably keep a bootstrap repo around (until m68k caught up) so that unstable (possibly amended by unreleased for a while) will not again become uninstallable, e.g. if arch:all packages change their dependencies (Python, gcc-defaults are some I’ve seen). That bootstrap repository is needed anyway because debootstrap can’t install from two separate repositories (unstable+unreleased for example).
Progress is slow because I try to keep as close to official packages as possible, refuse to cross-compile, and try to produce uploadable if possible packages all the time. Getting patches into packages, so that I can build from unstable, instead of debian-ports.org unreleased, has proven time-consuming and occasionally frustrating as well. Although I would like to thank the people who helped me on the way already. (I am not naming any in fear of forgetting some, but you know who you are ☺) They are among the Debian (gcc, kernel, m68k) and Linux-68k crowd.
(Why does genattrtab in gcc-4.4 take 3½ hours when it took less than half an hour in gcc-4.3 anyway?)
Well, did I mention dentists are sadists?
mksh was merged into Android (both AOSP and Google’s internal master tree) in the night 24/25th August, and is expected to be the one shell to rule them all, for Gingerbread.
mksh(1) now also has a cat builtin, for here documents mostly. It calls the cat(1) command if it receives any options. The shell is nevertheless smaller than yesterday because of improved string pooling.
There’s another reason to use the MirOS OPTU-16 encoding instead of PEP 383, on which I already wrote: try passing a wide-char filename to a function such as MessageBoxW, or create a filename on a system using wide chars, such as FAT’s LFN or ISO 9660’s Joliet, or one that only allows Unicode (canonically decomposed – ü → ü – out of all things) like HFS+. OPTU-8 at least maps to somewhat reserved codepoints (would, of course, be better to get an official 128 codepoint block, but the chance’s small of getting that in the BMP). Still.
Oh well, the torrents. I’ve remade them all, using one DHT seed node and OpenBitTorrent as tracker and put them on a very rudimentary BT page that will be completely redone soonish. Please re-download them. I currently do not believe f.scarywater.net will return.
Finally, I fell victim to a selling-out and may have just bought a Windows Mobile 6 based phone (Glofiish X650) and an SDHC card and an extra battery with double capacity. Well, at least it’s said to run CacheWolf well. I still would like to have something like Interix, Cygwin, UWIN, coLinux, or maybe some qemu-for-WinCE variant that runs Android, Maemo, Debian/armhf (or armel or arm) at near-native speed (and is usable – the device sadly doesn’t have a hardware keyboard, but it comes with SiRFstar Ⅲ GPSr). It only has 64 MiB RAM, like the Zaurus SL-C3200 and the jesusPhone, though. ☹ Any chance to get MirWorldDomination onto that device as well?
Tomorrow, eight years ago, is the date we now use as birthing point for MirOS. The thing is, we did not really want to create a BSD of our own, fork, or whatnot. We were mostly happy OpenBSD users (really happy before the first eMail exchange with its developers, where Theo de Raadt did indeed stand out but was not the only one – just the one with the authority to deny us), improved it locally and submitted patches and ports. We were flamed for that or, worse, ignored. I begun putting up my “OpenBSD patchkit” on my homepage (back then, at Tripod) and still tried to feed things to upstream and OpenBSD. Then, at some point, Theo de Raadt made it clear he did not want me and the patch kit had grown (from one 4M file into several of them), so I ended up doing a “cvs -d /cvs init” and went from there. Benny’s story is similar – he laughed at me while trying to get ports added to OpenBSD, then discovered his ports were added to the MirPorts Framework and getting commit access there was easier than getting some random developer to commit something of his to OpenBSD. (This trend ended there though… every single person I approached since has become OpenBSD ports committer – I wonder whether they used my invitation letter to blackmail Theo?) It’s often thought that there was a clash of opinions between Theo and me. I think while we might disagree in certain aspects or priorities things should have, in the end we both wanted the same thing, I just was promised to never become a member of the OpenBSD project, so it’s really just “them” being uncooperative. (They (Henning and others) did burn the T-Shirt I gave Theo as a gift some day for making OpenBSD what it was. I won’t comment on that, again, now.)
FrOSCon was a blast. I had two booths of my own – MirBSD and FreeWRT – as well as shares of Debian and Grml. Well, MirBSD was run by Benny and gecko2 because I just didn’t have any time for it, despite XTaran’s help with the FreeWRT booth. All I did was the initial setup of both booths, while at the same time answering about three questions regarding FreeWRT in parallel. Wow. What a little small, open hardware can do to you. XTaran and I had fun and we’ll do FreeWRT booths again; I managed to flash my two FON2100 devices (“La Fonera” – the FON2200 can use the same image, says nbd of OpenWrt) and will fix the port’s remaining few bugs I found; XTaran will try to push the WL-500gPv2 development. The social part was nice as well, although I think the greek restaurant in the city will not be visited by me again. Anyway, if you didn’t attend FrOSCon, your own fault…
Since the BitTorrent tracker used by MirOS is down, here’s the link to the [updated 2010-08-28] current (FrOSCon 2010 Edition) snapshot’s torrent, Triforce as usual. We’ll probably rewrite torrent files for all our ISOs and publish them on the MirBSD website. I’m currently considering OpenBitTorrent plus one or two DHT seed nodes with no statistics. Maybe with webseed. (Need to update the libtorrent/rtorrent MirPorts first, though…) Other options would be different trackers or running one of our own. I will announce the outcome as news entry, once done.
On the plus side, the review process of mksh(1) in Android continues, and I fixed the realpath builtin to behave even more POSIX-ish.
The FrOSCon 5 - 21./22. August 2010 booth plans have finalised, I am rather content:
This is especially good, as XTaran will be shared among (at least) Debian, Grml, FreeWRT; same for me plus of course MirBSD; kimnotyze is FreeWRT but may help with MirBSD; benz and gecko2 probably are MirBSD but gecko2 could help with FreeWRT, tokkee was interested in FreeWRT too… anyway.
Some days, you just love software.
Aug 6 13:55:01 blau firesomething-bin: stack overflow in function VFY_EndWithSignature Aug 6 13:55:01 blau /bsd: signal 6 received by (firesomething-bi:1146) UID(2999) EUID(2999), parent (sh:9059) UID(2999) EUID(2999)
Thus, let me reïterate it for all of you:
Well, now that the Debian Release Managers have sent their freezing bits around… *shudder*… Squeeze is frozen. Well, at least everything I have my hands in has migrated. I’m still… not persuaded. I also can’t decide which looks worse (KDE 4 or Win 7), tending towards KDE 4…
Why does all the horrid software (Solaris, Java™, OpenSSO, MySQL,
etc.) tend to end up at Oracle at the moment? Let me quote from some
Debian mailing list:
>>What happened to the Unix philosophy?
>Modern Solaris engineers
Is that similar to high-speed horse carriages?
My RPM repository has been pimped a bit – I ported some stuff from my DEB repository and updated them in both (rdate(8) and ntpd(8), specifically). Still ought to work more on them, but currently MirBSD base is most important, although I’m dying for mksh associative and multi-dimensional arrays, as well as more sh(1) conformance assorted bug fixes.
Well, there’s a life besides the computer. I’ve taken today off, wanting to hack on MirBSD’s most urgent problems (but probably end up doing that tomorrow), slept long, and will meet with cnuke@ and gecko2@ for Greek style dinner. The latter will almost certainly end up with a long-time work contract at the same place where I run a lot of things already, so congratulations. In the meanwhile, bsiegert@ has almost become a Doctor of Chemistry, and my brother’s finished his Maths and Economics diploma.
Also, I’ve put up the logo of the company where my new dedicated server is hosted; they reduce the monthly fee in exchange for this, so humour me and pay them a visit. They’re IPv6 pioneers, actually. (The server is now installed but not completely set up yet, and I have yet to begin moving services; it’ll be better than the VM eurynome is, but the clock could use the new timekeeping subsystem in the kernel as well as socket send (ÆrieBSD) and receive timestamps as it’s off by 0-1000 ms.)
Speaking of kernel stuff, yesterday I considered moving wscons(4) to UTF-8 again (since everything is CESU-8, we need to take raw octets into account also). I’ve seen OpenBSD begun importing Citrus… *shudder* Anyway, that’s my part, but I’d like volunteers for backporting things like the timekeeping stuff (and possibly more hardware support), and writing a pivot_root like thing (explained on some mailing list already) so we can use ramdisc root to do loopback root.