mksh tag cloud

Sponsored by
HostEurope Logo

mksh tag cloud

All 1 2 3 4 5 6 7 8

mksh R50d released

07.10.2014 by tg@
Tags: bug debian mksh news pcli

The last MirBSD Korn Shell update broke update-initramfs because I accidentally introduced a regression in field splitting while fixing other bugs – sorry!

mksh R50d was just released to fix that, and a small NULL pointer dereference found by Goodbox on IRC. Thanks to my employer tarent for a bit of time to work on it.

mksh R50c released, security fix

03.10.2014 by tg@
Tags: android bug debian mksh news pcli release security

The MirBSD Korn Shell has got a new security and maintenance release.

This release fixes one mksh(1)-specific issue when importing values from the environment. The issue has been detected by the main developer during careful code review, looking at whether the shell is affected by the recent “shellshock” bugs in GNU bash, many of which also affect AT&T ksh93. (The answer is: no, none of these bugs affects mksh.) Stephane Chanzelas kindly provided me with an in-depth look at how this can be exploited. The issue has not got a CVE identifier because it was identified as low-risk. The problem here is that the environment import filter mistakenly accepted variables named “FOO+” (for any FOO), which are, by general environ(7) syntax, distinct from “FOO”, and treated them as appending to the value of “FOO”. An attacker who already had access to the environment could so append values to parameters passed through programs (including sudo(8) or setuid) to shell scripts, including indirectly, after those programs intended to sanitise the environment, e.g. invalidating the last $PATH component. It could also be used to circumvent sudo’s environment filter which protected against the vulnerability of an unpatched GNU bash being exploited.

tl;dr: mksh not affected by any shellshock bugs, but we found a bug of our own, with low impact, which does not affect any other shell, during careful code review. Please do update to mksh R50c quickly.

mksh R50b released

03.09.2014 by tg@
Tags: mksh news pcli

The MirBSD Korn Shell has got a new bugfix release. Thought you’d want to know ☺

mksh R50, jupp 27 released

29.06.2014 by tg@
Tags: jupp mksh news pcli

Both the MirBSD Korn Shell and jupp – the editor which sucks less have seen new releases today. Please test them, report all bugs, and otherwise enjoy all the bugfixes.

Other subprojects will also have new releases… once I get around doing so after hacking them…

Update 03.07.2014: New release for MirCPIO, that is, cpio(1) and pax(1) and tar(1) in a somewhat portable package.

-r--r--r-- 4 tg miros-cvssrc 141973 Jul 3 19:56 /MirOS/dist/mir/cpio/paxmirabilis-20140703.cpio.gz

mksh build problem workaround

26.01.2014 by tg@
Tags: mksh

As a workaround to the build problem of mksh R49 with some host shells, you can try removing every sequence of backslash + newline in rlimits.opt and sh_flags.opt, for example with the following sequence:

	cd /path/to/mksh
	for f in *.opt; do
		tr '\n' '`' <"$f" | \
		    sed 's/\\`//g' | \
		    tr '`' '\n' >"$f.out"
		mv "$f.out" "$f"
	done
 

This will apply to every upcoming mksh(1) release until such time as this code has been rewritten in (host) C. Another thing that could be done is to add -r to both IFS= read line occurences in Build.sh, function do_genopt.

FOSDEM preparations… done.

20.01.2014 by tg@
Tags: debian event fun grml mksh twitxr work

I’ve produced several pin-on buttons to take with me to FOSDEM for giving away (as long as there are any left):

Several pin-on buttons I made

First row (nice projects), from left to right: MidnightBSD; Glenda, the Plan 9 bunny; Teckids e.V.

Second row (The MirOS Project): mksh; the Shilouette Dæmon; the “Triforce” (Live+Install CDs for i386 and sparc, with MirGrml); “the m” (alternative logo, vector)

Third row (things originating from tarent): Freedroidz (now a Teckids project); OSIAM (Identity and Access Management); tarent (tarent AG, tarent GmbH), who sponsored production of these buttons

Hm… jupp needs a button’able logo!


FOSDEM meetup

mksh-current version "R48 2013/11/17" (only this version, no others) suffer from a now-known regression which can make your MirBSD system unbootable. Please immediately either revert to an earlier version (such as a stable release) or update to "R48 2013/11/29" from AnonCVS.

This reverts the fix for “x="X 1 2"; showargs ${x#X}”, which nevertheless is a genuine bug. Contributions fixing it without introducing any such regressions welcome…

FrOSCon is approaching, and all MirBSD developers will attend… but why’s there no MirBSD exhibit? The answer to that is a bit complex. First let’s state that of course we will participate in the event as well as the Open Source world. We’ll also be geocaching around the campus with other interested (mostly OSS) people (including those we won for this sport) and helping out other OSS projects we’ve become attached to.

MirOS BSD, the operating system, is a niche system. The conference on the other hand got “younger” and more mainstream. This means that almost all conference visitors do not belong to the target group of MirOS BSD which somewhat is an “ancient solution”: the most classical BSD around (NetBSD® loses because they have rc.d and PAM and lack sendmail(8), sorry guys, your attempt at being not reformable doesn’t count) and running on restricted hardware (such as my 486SLC with 12 MiB RAM) and exots (SPARCstation). It’s viable even as developer workstation (if your hardware is supported… otherwise just virtualise it) but its strength lies with SPARC support and “embedded x86”. And being run as virtual machine: we’re reportedly more stable and more performant than OpenBSD. MirBSD is not cut off from modern development and occasionally takes a questionable but justified choice (such as using 16-bit Unicode internally) or a weird-looking but beneficial one (such as OPTU encoding saving us locale(1) hassles) or even acts as technological pioneer (64-bit time_t on ILP32 platforms) or, at least, is faster than OpenBSD (newer GNU toolchain, things like that), but usually more conservatively, and yes, this is by design, not by lack of manpower, most of the time.

The MirPorts Framework, while technically superiour in enough places, is something that just cannot happen without manpower. I (tg@) am still using it exclusively, continuing to update ports I use and occasionally creating new ones (mupdf is in the works!), but it’s not something I’d recommend someone (other than an Mac OSX user) to use on a nōn-MirBSD system (Interix is not exactly thriving either, and the Interix support was only begun; other OSes are not widely tested).

The MirBSD Korn Shell is probably the one thing I will be remembered for. But I have absolutely no idea how one would present it on a booth at such an exhibition. A talk is much more likely. So no on that front too.

jupp, the editor which sucks less, is probably something that does deserve mainstream interest (especially considering Natureshadow is using it while teaching computing to kids) but probably more in a workshop setting. And booth space is precious enough in the FH so I think that’d be unfair.

All the other subprojects and side projects Benny and I have, such as mirₘᵢₙcⒺ, josef stalin, FreeWRT, Lunix Ewe, Shellsnippets, the fonts, etc. are interesting but share few, if any, common ground. Again, this does not match the vast majority of visitors. While we probably should push a number of these more, but a booth isn’t “it” here, either.

MirOS Linux (“MirLinux”) and MirOS Windows are, despite otherwise-saying rumours called W*k*p*d*a, only premature ideas that will not really be worked on (though MirLinux concepts are found in mirₘᵢₙcⒺ and stalin).

As you can see, despite all developers having full-time dayjobs, The MirOS Project is far from being obsolete. We hope that our website visitors understand our reasons to not have an exhibition booth of our own (even if the SPARCstation makes for a way cool one, it’s too heavy to lift all the time), and would like to point out that there are several other booths (commercial ones, as well as OSS ones such as AllBSD, Debian and (talking to) others) and other itineries we participate in. This year both Benny and I have been roped into helping out the conference itself, too (not exactly unvoluntarily though).

The best way to talk to us is IRC during regular European “geek” hours (i.e. until way too late into the night – which Americans should benefit from), semi-synchronously, or mailing lists. We sort of expect you to not be afraid to RTFM and look up acronyms you don’t understand; The MirOS Project is not unfriendly but definitely not suited for your proverbial Aunt Tilly, newbies, “desktop” users, and people who aren’t at least somewhat capable of using written English (this is by design).

mksh/Win32

18.07.2013 by tg@

Michael Langguth and Scalaris AG asked me to publish the mksh/Win32 Beta 14 source and binary archive, and it is with joy I’m doing this.

Checksums and Hashes

  • RMD160 (ports/mksh-w32-beta14.zip) = 0dc8ef6e95592bd132f701ca77c4e0a3afe46f24
  • TIGER (ports/mksh-w32-beta14.zip) = 966e548f9e9c1d5b137ae3ec48e60db4a57c9a0ed15720fb
  • 1181543005 517402 /MirOS/dist/mir/mksh/ports/mksh-w32-beta14.zip
  • MD5 (ports/mksh-w32-beta14.zip) = b57367b0710bf76a972b493562e2b6b5

Just a few words on it (more in the README.1st file included): this is a port of The MirBSD Korn Shell R39 to the native WinAPI; it’s not quite got the full Unix feel (especially as it targets the Weihenstephan unxutils instead of a full Interix or Cygwin environment) but doesn’t need a full POSIX emulation layer either. It’s intended to replace MKS ksh and the MKS Toolkit. Source for the compatibility library is also included under The MirOS Licence; we aim at publishing it as OSI Certified Open Source Software like mksh itself. (There is a situation with dlmalloc/nedmalloc being resolved, and the icon is derived from the BSD dæmon which is a protected unregistered trademark, but we’re not Mozilla and allow distro packages to keep using it ☺) Rebasing it on a newer mksh(1) followed by (partial) integration into the main source code is a goal.

Have fun trying it out and hacking on it. It’s currently built with -DMKSH_NOPROSPECTOFWORK (so coprocesses and a few other minor things won’t work), but a SIGCHLD emulation is being worked on – but if you want to help out, I’m sure it’s welcome, just come on IRC or post on the mailing list, and I’ll forward things to Michael as needed. Reports on testing with other toolchain and OS versions are also welcome.

mksh R45 released

26.04.2013 by tg@

The MirBSD Korn Shell R45 has been released today, and R44 has been named the new stable/bugfix-only series. (That’s version 45.1, not 0.45, dear Homebrew/MacOSX packagers.)

Packagers rejoice: the -DMKSH_GCC55009 dance is no longer needed, and even the run-time check for integer division is gone. Why? Because I realised one cannot use signed integers in C, at all, and rewrote the mksh(1) arithmetics code to use unsigned integers only. Special thanks to the people from musl libc and, to some lesser amount, Natureshadow for providing me with ideas what algorithms to replace some functionality with (signed shell arithmetic is, of course, still usable, it is just emulated using unsigned C integers now).

The following entertainment…

	tg@blau:~ $ echo foo >/bar\ baz
	/bin/mksh: can't create /bar baz: Permission denied
	1|tg@blau:~ $ doch
	tg@blau:~ $ cat /bar\ baz
	foo
 

… was provided by Tonnerre Lombard; like Swedish, German has got a number of words that cannot be expressed in English so I feel not up to the task of explaining this to people who don’t know the German word “doch”, just rest assured it calls the last input line (be careful, this is literally a line, so don’t use backslash-newline sequences) using sudo(8).

mksh R42b and R41c released

15.02.2013 by tg@
Tags: mksh

The MirBSD Korn Shell must-have bugfix releases R42b (for the current formal release series) and R41c (for the last bugfix-only stable series) have been issued. Debian testing/unstable users get the fixes through their package management, versioned 40.9.20120630-7 targetting wheezy (hopefully). Debian experimental users will receive a 42b-1 upload, closer to what other distro packages provide, soonish, targetting jessie.

Since we lack anything better (that, unless Freshmeat, is actually usable), by request of our packagers, new mksh(1) releases will be announced on our miros-mksh@ mailing list starting now.

mksh made quite some waves (machine translation of the third article) recently. Let’s state it’s not just Amigas – ara5 is a buildd running the Atari kernel, an emulated though. On the other hand, the bare-metal Ataris used to be the fastest buildds, so I expect we get them back online soonish. I’m currently fighting with some buildd software bugfixes, but once they’re in, we will make more of them. Oh, and porterboxen! Does anyone want to host a VM with a porterbox? Requirements: wheezy host system (can be emulated), 1 GiB RAM, one CPU core with about 6500 BogoMIPS or more (so the emulated system has decent speed; an AMD Phenom II X4 3.2 GHz does just fine). Oh, and mksh is ported to more and more platforms, like 386BSD 0.0 with GCC 1.39, and QNX 4 with Watcom… and more bugfixes are also being worked on. And let’s not forget features!

jupp got refreshed: it’s got a bracketed paste mode, which is even auto-enabled on xterm-xfree86 (though the xterm(1) in MirBSD’s a tad too old to know it; will update that later, just imported sendmail(8) 8.14.6 and lynx(1) 2.8.8dev.15 into base, more to come) and will be enhanced later (should disable auto-indent, wordwrap, status line updates, and possibly more), lots of new functions and bindings, now uses mkstemp(3) to create backup files race-free, and more (read the NEWS file).

In MirBSD, Benny and I just added a number of errnos, mostly for SUSv4 compliance and being able to compile more software from pkgsrc® without needing to patch. This is being tested right now (although I should probably go out and watch fireworks in less than a half-hour), together with the new imports and the bunch of small fixes we accumulate (even though most development in MirBSD is currently in mksh(1) and similar doesn’t mean that all is, or worse, we were dead, which we aren’t). I’ll publish a new snapshot some time in January. The Grml 2012.12 also contains a pretty up-to-date MirBSD, with a boot(8/i386)loader that now ignores GUID partition table entries when deciding what to use for the ‘a’ slice.

If you haven’t already done so, read Benjamin Mako Hill’s writings!

mksh R41 released

30.11.2012 by tg@
Tags: mksh

The MirBSD Korn Shell R41 has finally been released. This is a major “everything” version with important fixes as well as new features and behavioural changes. Sorry for the delay.

See also the:

Originally posted by bubulle on Planet Debian, a shell prompt that displays the current git branch, in colour on some terminals, after the current working directory. The following snippet does similar things for mksh users, except it doesn’t redefine your prompt but amend it – just throw it at the bottom of your ~/.mkshrc before that last line beginning with a colon (copy from /etc/skel/.mkshrc if you haven’t done that yet):

	function parse_git_branch {
		git branch 2>/dev/null | sed -n '/^\* \(.*\)/s//(\1)/p'
	}

	function amend_prompt_with_git {
		local p q='$(parse_git_branch)' r

		if [[ $TERM = @(xterm-color|xterm|screen*) ]]; then
			if [[ ${PS1:1:1} = $'\r' ]]; then
				p=${PS1:0:1}
			else
				p=$'\001'
				PS1=$p$'\r'$PS1
			fi
			q=$p$'\e[1;33m'$p$q$p$'\e[0m'$p
		fi

		p=${PS1%%*( )[#$]*( )}
		if [[ $p != "$PS1" ]]; then
			# prompt ends with space + #-or-$ + space, we can amend
			r=${PS1: ${#p}}
			PS1=$p$q$r
		fi
	}
	amend_prompt_with_git
	unset -f amend_prompt_with_git
 

The indirection by use of a function is not strictly necessary but allows the use of locals. I took the liberty of adding an asterisk after “screen” to match the GNU/Linux nonsense of having TERM=screen.xterm or somesuch.

While wtf(1) always has been a bit central to MirBSD, and the acronym database has been accessible by CVSweb, what we never had was a DAU compatible (and shellsnippets compatible) lookup. This has now changed: the above link to the acronyms file is a persistent link to its latest version (well, latest when the website was last recompiled), tooltips may very well follow soon, and we’ve got an online WTF lookup service.
Contributions to the acronym database are welcome, of course; just eMail them to tg@mirbsd.org.

Not to stop there, our online HTML manpage search is also new, shiny, and should replace the “!mbsdman” DuckDuckGo hash-bang shortly. (Both of these services offer a DDG search as fallback. Note that DDG is an external service included herein by linking, under their request to spread it, and not affiliated with The MirOS Project. They do, however, donate some advertising money to Debian.)
For all those who didn’t know: only manpages for software in the MirOS BSD base system and for the MirPorts Framework package tools are listed, not for third-party applications installable using ports or, recently, pkgsrc®. Still, if you want to have a peek at a modern classic BSD’s documentation, you’re welcome. (Not to mention content like re_format(7) and style(9) and that some of our documentation is much more legible than others.)

And because writing all that perl(1) made me ill, not to mention I don’t even know that language, I’ve hacked a bit more in the mirmake(1) and mksh(1) parts of the MirWebsite, finally implementing pointing out where in the navigation sidebar the visitor currently is.

We also have exciting mksh porting news involving RT trying a larger number of ancient platforms than I dare count, me fixing bugs in Linux klibc and diving into other things, learning more about why I consider me lucky for hacking a BSD operating system… sorry, I want to keep this short as it’s mostly an announcement.

The MirWebsite source code is, of course, also available. Improvements welcome. Except for these three CGIs, our website is fully statically precompiled, and that’s a good thing. Please help in making the CGIs secure.

This is both a release announcement for the next installment of The MirBSD Korn Shell, mksh R40b, and a follow-up to Sune’s article about small tools of various degrees of usefulness.

I hope I don’t need to say too much about the first part; mksh(1) is packaged in a gazillion of operating environments (dear Planet readers, that of course includes Debian, which occasionally gets a development snapshot; I’ll wait uploading R40c until that two month fixed gcc bug will finally find its way into the packages for armel and armhf). Ah, we’re getting Arch Linux (after years) to include mksh now. (Probably because they couldn’t stand the teasing that Arch Hurd included it one day after having been told about its existence, wondering why it built without needing patches on Hurd…) MSYS is a supposedly supported target now, people are working on WinAPI and DJGPP in their spare time, and Cygwin and Debian packagers have deprecated pdksh in favour of mksh (thanks!). So, everything looking well on that front.

I’ve started a collection of shell snippets some time ago, where most of “those small things” of mine ends up. Even stuff I write at work – we’re an Open Source company and can generally publish under (currently) AGPLv3 or (if extending existing code) that code’s licence. I chose git as SCM in that FusionForge instance so that people would hopefully use it and contribute to it without fear, as it’s hosted on my current money source’s servers. (Can just clone it.) Feel free to register and ask for membership, to extend it (only if your shell-fu is up to the task, KNOPPIX-style scripts would be a bad style(9) example as the primary goal of the project is to give good examples to people who learn shell coding by looking at other peoples’ code).

Maybe you like my editor, too? At OpenRheinRuhr, the Atari people sure liked it as it uses WordStar® like key combinations, standardised across a lot of platforms and vendors (DR DOS Editor, Turbo Pascal, Borland C++ for Windows, …)

ObPromise: a posting to raise the level of ferrophility on the Planet aggregators this wlog reaches (got pix)

mksh R40b (nowadays with filled in user’s caveats (for R40, too!) and packager’s upgrade hints) has just been released. This is a should-have upgrade, fixing a number of – admittedly some obscure – bugs, changing things begun in R40, improving upon others. Thanks to the PLD Linux guys for spotting all these errors; thanks to them and phpnet.org both for adopting mksh so well.

I have also fixed a bug in nroff(1) which will lead to an even nicer looking HTML manpage mksh(1) (after the next rebuild and upload of a MirBSD snapshot – scheduled RSN).

jupp 3.1.16 took on the task of merging Debian joe changes (aiming at an upload). I also split the jupprc file into three versions (2.8 generic/DOS, 3.1+jupp and 3.7/Unix) because of the differences in the baseline executables making rc files partially mutually incompatible (think Insert key), annoyingly warning (think syntax, hmsg), or less usable (joe’s new menu system).

jupp 2.8.2 is a companion to jupp 3.1.16 – mostly because of the new help window “character map” ☺

Binaries for jupp should be updated RSN too.

Considering Banja Luka is arriving quickly, the “r” in RSN should be taken with a few grains of salt. I’ve also scheduled working on the pcc Debian package for the next future; updating lynx and maybe others like OpenSSH in MirBSD is also due; cvs(1) will receive more of my time, but before the next Upload I’d like to fix LP#12230 once verified.

Builds for Debian/m68k are also still running. I note I did in fact not manage to make a new base image, yet (but 2.6.39 kernels miss a patch, anyway, so waiting for 3.0 is ok). It’s still using gcc-4.4 because nobody tests gcc-4.6 and gcj-4.6 FTBFS due to SIGSEGV, but that’s ok in my books. rsyslog is broken but sysklogd works.

The #ksh|Freenode page finally got a well-deserved link to Planet Commandline. Throw more my way!

Acronyms and translations, too. (Got Norwegian and Rumanian covered in the meantime. No idea whether any RTL languages will work in that beast. But I’m young and need the money)

Since I’m writing a wlog entry anyway… let me thank Gunnar for a nice summary on the current Free Culture discussion; my comments on Nina’s site seem to be eaten, but let me support it fully, although, of course, I normally use a copycenter style licence, which is specifically written for general works of authorship under copyright law, not limited to software. I did in fact have that in mind. Maybe some people will like it (it’s less than one Kibibyte long) either generally or just for their everyday random musings (they can then keep CC-BY-SA for the “big works” if they so desire).

Wouter, grass background makes green headlines illegible. I’ve never liked, and never installed manually, cups either. (Benny tells me that Apple’s new version refuses to talk with a non-Apple cups, kinda defeating the whole idea I think.) Port 9100 is JetDirect (probably with an HP in front and some subset of ©®™ trailing) and just nice. (Being able to talk ESC/P with your printer like print '\033K\x07\0\x3E\x81\x99\xA5\xA5\x81\x3E' >/dev/lpa too rocks though, IMHO. Yes, mine can, and I still can. /dev/lpa is BSD.)

Kai, thanks for your vimrc lines:

	:highlight TrailWhitespace ctermbg=red guibg=red
	:match TrailWhitespace /\s\+$\| \+\ze\t/
 

Automatic removal is harmful, though – I just fell into the trap since jupprc contains needed whitespace at EOL… but manual removal (bound to ^K] in jupp) rocks. And I like that your solution uses such strong a colour – vim users are the single most represented offender group for actually leaving the redundant whitespace at EOL there, and it should hurt their eyes. (Sadly there is some vehement disagreement preventing them from inclusion in grml-etc-core – but that’s why I re-post them here.) Ah, and jupp can of course display whitespace visibly (although it uses ‘·’/‘→’, replacing the arrow with ‘¬’ if no UTF-8, not ‘»’), accessible with ^Ov.

Steve, want to put up a checklist for sites? We can “crowdsource” the… testing… to maybe get some interesting results…

Some other people would get more comments if they were idling in IRC (Freenode) or allow comments on their blog, specifically without too high an entrance barrier – OpenID is ok, but many other things, and ECMAscript, are not; but I can’t really say that loud because our wlog is static HTML compiled from a flat plaintext data source so it doesn’t allow such either. I often forget what I wanted to add if I can’t get it out quickly enough (especially at work). Sowwy…

Me like the cat picture postings (Amayita, Tiago, ¡Gracias!).

New releases

11.07.2011 by tg@
Tags: debian mksh

You might have noticed the release of mksh R40 recently, after more than a year of development. Well, stay tuned for both R40b (with accumulated fixes) and R41 (intent to speed up array handling a lot and prepare for what we postponed to mksh R42 now – associative, multi-dimensional arrays).

You should also upgrade, if you have not yet done so, to kwalletcli 2.11.

Finally, jupp 3.1.15 was left out to the world, including Minix 3 users this time, by special request of one of these on our mailing list. In addition to the MidnightBSD mport – which has been there in like forever – and the MirPort and the FreeWRT package, in December 2011 a user submitted it to FreeBSD® ports, and Benny is going to add it to NetBSD® pkgsrc® soon, he said. (He also updated their mksh source package. Thanks!) I’ve been asked by two people, independent from each other, when I’ll upload it to Debian proper, instead of the private-repo packaging. Maybe I should indeed do that, comments?

mksh R40 released

12.06.2011 by tg@
Tags: mksh

The MirBSD Korn Shell R40 has finally been released. This is a must-upgrade, a major everything version. See the changelog for now, until caveats are filled in, due to the sheer amount of changes. The diff between the last and the new release is more than half of the last release’s size, after all – it’s been 15½ months.

The plan for the next two releases is:

  • mksh R41 will fix all bugs popping up within the next few weeks, as they usually show up shortly after a release only, when things are getting adopted. Furthermore, it removes deprecated code that emits a warning in R40: several Build.sh options, plus set ±o arc4random. Finally, I plan on speed-up for arrays by using hashtables internally.
  • mksh R42 is where we tentatively target associative arrays for, now. Multidimensional arrays should be introduced at the same time.

(First posting to Plänet Commandline! Tag: pcli)

Vutral asked in IRC how to synchronise two shells’ environment while they’re running. As you may know, POSIX systems cannot change a process’ environment vector after it has been started, only the process itself can. Well, the shell can, and we’ll use a variety of things for this.

This trick assumes you have $HISTFILE set to the same pathname in both shells (obviously, they run under the same user). It uses export -p to render the current list of exported variables, then transforms the list from newline-separated to a single big one-line export statement.
Then it transforms all remaining newlines (which will be part of a single-quoted string, since that’s mksh(1)’s export format) into the sequence '$'\n'' which means: terminate current single-quoted string, append $'\n' and open up a new single-quoted string immediately; concatenate these three.
Now, $'\n' is just a fancy way of saying newline, and part of mksh because David Korn (yes, the Korn in Korn Shell) strongly suggested to me that this functionality be included – but, as we can see here, it pays off.
Finally, the so transformed string is prepended by unset \$(export); which, when executed, will cause the shell to unset (and unexport) all currently exported variables. The shell parameters that are not exported, i.e. not in the environment, are not affected by this code (except for $x and $nl, but… whatever).
This string is then passed to read -s (plus -r and clearing IFS to enable raw mode), which means, read into the parameter $REPLY (which we conveniently don’t use – but it’s trashed too, thus) but store into history at the same time.

Ah hah! Now, the persistent history feature comes into effect! After running the below statement in the “source” shell, switch into the terminal running the “destination” shell, press Enter once on the empty line (Ctrl-U to empty it if it wasn’t), then Cursor-Up (↑) to recall… voilà, an insanely large line with the previously created string sorta expanded… and press Enter again to run it. Now your set of exported parameters is the exact same (minus if you exported IFS, nl, x or REPLY) as in the “source” shell.

I’ve added extra spaces and a linewrap below, this is really just one big line:

nl=$'\n'; x=$(export -p); x=${x//${nl}export/}; IFS= read -rs <<<"unset \\\$(export);${x//$nl/\'\$\'\\\\n\'\'}"

Of course, this makes a nice function, for your ~/.mkshrc or somesuch.

mksh-current has just gained an experimental recursive parser for command substitutions, fixing RedHat BZ#496791 and decades-old complaints about the pdksh codebase, compared to AT&T ksh93. (GNU bash could also do the example, but not some other things mksh(1) parses fine now.)

This means that things like the following work now.

	# POSIX, should “always” work
	echo $(case 1 in (1) echo yes;; (2) echo no;; esac)
	# POSIX optional, works now in mksh, works in GNU bash
	echo $(case 1 in 1) echo yes;; 2) echo no;; esac)
	# GNU bash seems to choke on comments ending with backslash
	# a comment with " ' \
	x=$(
	echo yes
	# a comment with " ' \
	)
	# No non-recursive COMSUB parser can pass all of the above
	# tests and these below at the same time (some extensions)
	echo $(typeset -i10 x=16#20; echo $x)
	echo $(typeset -Uui16 x=16#$(id -u)
	) .
	echo $(c=1; d=1
	typeset -Uui16 a=36#foo; c=2
	typeset -Uui16 b=36 #foo; d=2
	echo $a $b $c $d)
	# the ‘#’ is especially tricky, that’s why the above cases
 

Next on my TODO is the complete rewrite of the read built-in command, as well as its documentation. I think that the (reduced) goals for mksh R40 will have been met by then, except porting to LynxOS and MPE, but we’re working on it, and re-testing Syllable and Plan 9). Of course, a release implies testing on a lot of the supposedly supported platforms, so it won’t be out “immediately”. Though, associative arrays have been removed from the R40 goals, so that I can at least get a new release out. Note that Debian and OpenSuSE Buildservice users have been provided with somewhat well-tested mksh-current snapshots for a while already, and Gentoo users can use the “live ebuild”; there’s always compiling from source too…

(Free)BSD vs. Linux

03.03.2011 by tg@
Tags: debian event ill mksh rant

Warning: this is a rant against BSD (specifically FreeBSD®, but don’t let me get started on DragonFly, who think it’s wise to drop all shells except ash from the base system and rely on pkgsrc® – yay let’s compile a dozen packages just to get a shell with tab completion, not to mention boxen with no network access – for the task – although others seem to go into that direction too…; you know, there’s BSD, and then there’s FreeBSD…) – don’t like, don’t read.

If you want to change something in the BSD world, you gotta fork your own BSD – no other way around the thickheads. Ok, back then, I ran into a particularily thick one, but others tend to not be much better. Users share the thickness. If you want to change something in the GNU/Linux world, just make a package, have someone upload it, prod (or pay, Hanno got a Radler) people to do it, or just upload it yourself.

At the BSD booth at FOSDEM, despite me bringing the Windows® Mobile 6 Professional devive, strictly for Geocaching mind you, Macintosh boxen had a share of more than 50% – I didn’t manage to tip the scale. At the Debian booth, almost everyone had a “I want to buy a new laptop some day, but it just keeps on working and doesn’t break” pre-Lenovo IBM laptop. No hyping of Google either. (Last year’s CLT saw BSD people advocating pro-Schily – the guy with the broken encoding in his name – shockingly.)

Honestly, tcsh, FreeBSD® people? Sorry. While I agree that there is merit on having the same script and interactive shell, as someone has pointed out (copy-paste examples into the command line), there’s those zsh users who use mksh or GNU bash for scripting. Or just POSIX shell. And that’s with an interactive shell which can be used for scripting. On the other hand, the C shell (both csh and tcsh) cannot.

And what’s with pretending the accent gravis is non-combining, called “backtick” (such a thing does not exist); and advocating it? Sorry, if your csh/tcsh doesn’t handle the POSIX $(…) you should just drop it. (By the way, there is a convention that example command lines are prefixed with for csh and for sh (or but we write $ sudo  instead, these days). Use it. Or leave it. If you have examples that substitute another process’ output, be specific.) It’s funny to see how one person tries to defuse my arguments against csh by telling me “it’s just an interactive shell”, while the other argues that people copy-paste between them, to which that was my response. Read the thread!

And please, get your facts right. “I would prefer that the standard shell be at least Bourne-compatible.” You don’t want Bourne (“^” instead of “|” for pipes), you want POSIX. That GNU bash is called the Bourne-Again Shell in one of their usual semi-bad puns doesn’t help the global perception of such things any. Also, the root shell and /bin/sh are disjunct.

(Plus, why change the root shell, use sudo(8), plain and simple.)

ObNote: in jupp (should I package that for Debian, btw? rather upload, packages are ready…) the ‘`’ key is used as præfix for Ctrl-X (`X) or to directly enter numerical (decimal, octal, sedecimal/hexadecadic) ASCII, 8-bit or Unicode codepoints. Yay!
And even the FSF has seen the light; for a few releases already, GCC uses “'…'” instead of “`…'” for quoting in messages, even without locales. Great job there! (LC_MESSAGES=en_GB.UTF-8 usually works, too, though.)

ObDisclaimer: I have an (yes, Google…) Alert on the word “mksh”, so I know when it’s being discussed. This obviously includes certain fora. Also, I’m a shell implementer and bound to know a certain amount of details. Plus, mksh’s build script runs with pretty much any Bourne/POSIX/Z Shell which has functions and not too many bugs. I wrote it. Go figure. No lowly trolling.

FWIW, mksh(1) has the cat(1) builtin both because Android has no cat(1), and as speed hack. Almost all other shells have worse speed hacks, like a printf(1) builtin. And recently, builtins have become direct-callable, so this actually reduces the overall system footprint. (Its inclusion also provides for some other possibilities, internally.) And as two final side notes, if you haven’t seen this: determine which shell we are run under (CVS) and I still offer a prompt conversion service (send me any GNU bash or oksh $PS1 and I’ll send that to you in mksh(1) syntax – optionally with adjustments/improvements, like cwd uses only up to 1/3 of screen width).

Eh. Why does mksh built with (a patched: mkstemp(3) added) klibc work suddenly, unexpectedly?

To reproduce, I just uploaded mksh_39.3.20110218-1.dsc and you can run DEB_BUILD_OPTIONS=mksh-static=klibc,dietlibc,eglibc dpkg-buildpackage -rfakeroot to verify it, once you have mkstemp(3). (I will probably send a smaller implementation of that in, later.) I have that and the open fix and the m68k patch applied, nothing else… where did my bug go?

ObQuestion: what’s the legal (copyright/trademark) status of the Atari logo (the one in rainbow colours, with three things going up, right and left “leg” looking like an umbrella stand’s)?

FOSDEM was a blast!

13.02.2011 by tg@

I just need to work more on bilocality. While I did find two geocaches, one at the South/Noon Train Station (taalverwarringen…), one in the buurt of the University, I did manage to miss the AW building completely and utterly. Wow. Except, that Haiku guy came over to talk for a bit (nice). And I drew. An Atari logo with swirl, for that weird stuff I recently have been found doing.

More mksh-current news coming soon, stay tuned. In the meanwhile, I met bonsaikitten IRL (at FOSDEM, yes, too) who kindly made a “live ebuild”, i.e. a source package building -current.

Finally let’s say a big thank you to the person mostly manning our booth, gecko2, and to Benny for talking to people, getting That Other Packaging Thingy working, and pimping the website a bit.

mksh wins

28.01.2011 by tg@
Tags: debian mksh

The Android 3.0 (Honeycomb) Preview [German news] comes with a much more sensible /system/bin/sh than previous released versions… Android-x86 and AOSP have had it for a while, but this sees the general adoption of mksh(1) as the only POSIX/Korn Shell on the Android OS! Thanks to those who helped to make this reality.

I’ve just prepared an ISO for FOSDEM Ⅺ which we might export on BitTorrent soon. Benny has provided an upgraded desktop background image, thanks.

The next mksh release… well I’ll bite the sour apple and will release it without associative arrays but hope I get around to hack a few things (especially the read and sleep builtins) before releasing. One lession learned, don’t brag with oh-so-big plans when you haven’t got a deliverable yet. Real Life will interfere. Oh, and Murphy, of course. (mksh R41 might have the associative arrays then. But mksh-current has ${foo@#} which is hash($foo).

FOSDEM, the Free and Open Source Software Developers' European Meeting

❧ Who’s not? ☺ Same procedure as every year.

(okay, lolando prefers skiïng but…)

Anyway. A cow‐orker told me that Belgium again/still has no gouvernment, and they have been asked to grow out their beards until they do. I found “evidence” on the ’net but won’t link it here, also it’s on German… anyway. Let’s all join in. (Besides, I now have an excuse to not shave, maybe even my grandmother will accept this one…)

RT said on IRC that mksh will probably work on MSYS.

My Debian/m68k stuff is coming around nicely, but I still haven’t gotten around to do everything planned, plus I need to grow a new kernel and eglibc, after the latest uploads, and the 2.6.37 based one panics. Also I’ve got to take care to not overwork myself. (And make a MirBSD ISO for FOSDEM.) But hey, it’s been not working for some time and better now. And slow anyway ☺ yet we’re progressing. Does anyone know how to debug that a C programme only calling res_init(3) segfaults?

Benny is apparently not just working on making NetBSD® pkgsrc® available on MirOS BSD (picking up my work from 4+ years ago) but also replacing The MirPorts Framework with it. Sad, as I got a request for a gajim MirPort over a cocktail just this evening…

I’m almost finished with “sort of re-bootstrapping” Debian/m68k (I can use etch-m68k as well as what was in unstable at the moment as dependencies, so it was not that much, still, 305 binary packages build from 84 source packages, most for unstable (very few for unreleased, with very responsive maintainers, thanks all, who will include the patches in their next uploads) is a bit… including rebuilds with newer versions, more patches, more testing or newer dependencies installed. I’ll probably upload on Sunday evening, as I’ll be off for 2-4 days at least from then (see below). Ingo tried to test on real hardware, but as Murphy wants a hard disc failed… we’ll still try to get something done over the weekend. If you want to have a look, see my repository index (sources.txt contains a sample sources.list file, 0-NOTE.txt some hints, including the right debootstrap/cowbuilder magic and speed tricks). I’ll need to learn how to use LVM and set up a buildd now…

I’ve not been in much of a hacking mood recently – all these visits to the dentist leave me in unrest and disturb my equilibrium. Hence, not much activity even in mksh even if there was need, almost none in MirBSD. This is only temporary, but I won’t attend OpenRheinRuhr, or, if I come at all, it’ll be for socialising only and probably only one day. Benny’s done with his Doctor (in France, no idea whether it’s one in Germany as well) of Chemistry and has returned to hacking some (World of) Google-Go(o) code. I expect MirBSD activity to slowly raise once we can come back. Please accept our apologies.

mksh, encodings, MirBSD, BitTorrent, WinCE

28.08.2010 by tg@
Tags: android debian geocache hardware mksh news release snapshot

mksh was merged into Android (both AOSP and Google’s internal master tree) in the night 24/25th August, and is expected to be the one shell to rule them all, for Gingerbread.

mksh(1) now also has a cat builtin, for here documents mostly. It calls the cat(1) command if it receives any options. The shell is nevertheless smaller than yesterday because of improved string pooling.

There’s another reason to use the MirOS OPTU-16 encoding instead of PEP 383, on which I already wrote: try passing a wide-char filename to a function such as MessageBoxW, or create a filename on a system using wide chars, such as FAT’s LFN or ISO 9660’s Joliet, or one that only allows Unicode (canonically decomposed – ü → ü – out of all things) like HFS+. OPTU-8 at least maps to somewhat reserved codepoints (would, of course, be better to get an official 128 codepoint block, but the chance’s small of getting that in the BMP). Still.

Oh well, the torrents. I’ve remade them all, using one DHT seed node and OpenBitTorrent as tracker and put them on a very rudimentary BT page that will be completely redone soonish. Please re-download them. I currently do not believe f.scarywater.net will return.

Finally, I fell victim to a selling-out and may have just bought a Windows Mobile 6 based phone (Glofiish X650) and an SDHC card and an extra battery with double capacity. Well, at least it’s said to run CacheWolf well. I still would like to have something like Interix, Cygwin, UWIN, coLinux, or maybe some qemu-for-WinCE variant that runs Android, Maemo, Debian/armhf (or armel or arm) at near-native speed (and is usable – the device sadly doesn’t have a hardware keyboard, but it comes with SiRFstar Ⅲ GPSr). It only has 64 MiB RAM, like the Zaurus SL-C3200 and the jesusPhone, though. ☹ Any chance to get MirWorldDomination onto that device as well?

Tomorrow, eight years ago, is the date we now use as birthing point for MirOS. The thing is, we did not really want to create a BSD of our own, fork, or whatnot. We were mostly happy OpenBSD users (really happy before the first eMail exchange with its developers, where Theo de Raadt did indeed stand out but was not the only one – just the one with the authority to deny us), improved it locally and submitted patches and ports. We were flamed for that or, worse, ignored. I begun putting up my “OpenBSD patchkit” on my homepage (back then, at Tripod) and still tried to feed things to upstream and OpenBSD. Then, at some point, Theo de Raadt made it clear he did not want me and the patch kit had grown (from one 4M file into several of them), so I ended up doing a “cvs -d /cvs init” and went from there. Benny’s story is similar – he laughed at me while trying to get ports added to OpenBSD, then discovered his ports were added to the MirPorts Framework and getting commit access there was easier than getting some random developer to commit something of his to OpenBSD. (This trend ended there though… every single person I approached since has become OpenBSD ports committer – I wonder whether they used my invitation letter to blackmail Theo?) It’s often thought that there was a clash of opinions between Theo and me. I think while we might disagree in certain aspects or priorities things should have, in the end we both wanted the same thing, I just was promised to never become a member of the OpenBSD project, so it’s really just “them” being uncooperative. (They (Henning and others) did burn the T-Shirt I gave Theo as a gift some day for making OpenBSD what it was. I won’t comment on that, again, now.)

FrOSCon was a blast. I had two booths of my own – MirBSD and FreeWRT – as well as shares of Debian and Grml. Well, MirBSD was run by Benny and gecko2 because I just didn’t have any time for it, despite XTaran’s help with the FreeWRT booth. All I did was the initial setup of both booths, while at the same time answering about three questions regarding FreeWRT in parallel. Wow. What a little small, open hardware can do to you. XTaran and I had fun and we’ll do FreeWRT booths again; I managed to flash my two FON2100 devices (“La Fonera” – the FON2200 can use the same image, says nbd of OpenWrt) and will fix the port’s remaining few bugs I found; XTaran will try to push the WL-500gPv2 development. The social part was nice as well, although I think the greek restaurant in the city will not be visited by me again. Anyway, if you didn’t attend FrOSCon, your own fault…

Since the BitTorrent tracker used by MirOS is down, here’s the link to the [updated 2010-08-28] [deleted 2014-05013] current (FrOSCon 2010 Edition) snapshot’s torrent, Triforce as usual. We’ll probably rewrite torrent files for all our ISOs and publish them on the MirBSD website. I’m currently considering OpenBitTorrent plus one or two DHT seed nodes with no statistics. Maybe with webseed. (Need to update the libtorrent/rtorrent MirPorts first, though…) Other options would be different trackers or running one of our own. I will announce the outcome as news entry, once done.

On the plus side, the review process of mksh(1) in Android continues, and I fixed the realpath builtin to behave even more POSIX-ish.

Well, I suppose I should be happy that mksh is actually used…

  • [tg] Correct shf buffer I/O routines to avoid a memory corruption bug discovered by Waldemar Brodkorb and other bad effects
  • [tg] Fix NULL pointer dereference during iteration loop when checking for alias recursion; discovered by Michal Hlavinka

That’s OpenADK (Waldemar’s fork of FreeWRT, which is Waldemar’s fork of OpenWrt), and Red Hat Enterprise Linux, respectively. Popcon in Debian and its derivates is also pleasant.

I could use some help bugfixing this though:

	(sleep 3; exit 12) &
	bgprocpid=$!
	sleep 6
	# background process is done by now
	wait $bgprocpid
	# POSIX mandates that, since $! was asked
	# for, wait is to reply its errorlevel

Somehow, JF_KNOWN is never set – and I can’t debug this with gdb(1).

(There’s also a dashism in some *buntu start scripts that does pretty much the same except it uses “wait %1” there. In fact it doesn’t even seem to use $! – no idea whether we can support that at all in a POSIX shell – which dash clearly isn’t… – without keeping track of background processes forever.)

I’ve got some interesting results using r1.1 of an example test programme (r1.2 got cleaned up and more output) on various systems, regarding ASLR. The 1.1 revision tests everything mksh R40+ will use (except there will probably no larger than page sized allocations) for its LCG PRNG. On OpenBSD (MirBSD, ÆrieBSD) malloc(3) uses in fact mmap(2), which is randomised. (Though -pie doesn’t yet work as it’s supposed to.) Some OSes are better than others… but look for yourself. (Read on to continue, not part of the RSS for size reasons. This wlog entry may be updated – with bumped date – unperiodically.)

MirBSD-current/i386

tg@blau:~ $ mgcc -static x.c
x.c:0: note: someone does not honour COPTS correctly, passed 0 times
x.c: In function `foo':
x.c:27: warning: function returns address of local variable
tg@blau:~ $ ./a.out
0xa9332000 0xaba65000 0xa0ae7000 0xcfbed990 0xcfbed994
tg@blau:~ $ ./a.out
0xa91b4000 0xa02b1000 0xa1602000 0xcfbf8680 0xcfbf8684
tg@blau:~ $ ./a.out
0x9f731000 0x9cb2a000 0xa94ca000 0xcfbf5840 0xcfbf5844
tg@blau:~ $ ./a.out
0x9c2af000 0xa6a0b000 0xa4ce1000 0xcfbefac0 0xcfbefac4
tg@blau:~ $ ./a.out
0xa3b61000 0xa96de000 0xa96df000 0xcfbedcc0 0xcfbedcc4

Debian Ätsch/i386

tg@frozenfish:~ $ gcc -static x.c
x.c: In function ‘foo’:
x.c:27: warning: function returns address of local variable
x.c: In function ‘bar’:
x.c:33: warning: function returns address of local variable
tg@frozenfish:~ $ ./a.out
0x80b2a20 0x80b2a30 0xb7745008 0xbf985ce0 0xbf985cd4
tg@frozenfish:~ $ ./a.out
0x80b2a20 0x80b2a30 0xb7726008 0xbfb911b0 0xbfb911a4
tg@frozenfish:~ $ ./a.out
0x80b2a20 0x80b2a30 0xb7784008 0xbf83d040 0xbf83d034
tg@frozenfish:~ $ ./a.out
0x80b2a20 0x80b2a30 0xb77e8008 0xbfc0f840 0xbfc0f834

tg@frozenfish:~ $ sid
I: [sid chroot] Running command: “mksh -l”
tg@frozenfish:~ $ gcc -static x.c
x.c: In function ‘foo’:
x.c:27: warning: function returns address of local variable
x.c: In function ‘bar’:
x.c:33: warning: function returns address of local variable
tg@frozenfish:~ $ ./a.out
0x80c86a8 0x80c86b8 0xb77c3008 0xbfaa1900 0xbfaa18f4
tg@frozenfish:~ $ ./a.out
0x80c86a8 0x80c86b8 0xb77d2008 0xbfcc0260 0xbfcc0254
tg@frozenfish:~ $ ./a.out
0x80c86a8 0x80c86b8 0xb77c1008 0xbfbe2120 0xbfbe2114

uname: Linux frozenfish 2.6.18-6-686 #1 SMP Fri Feb 19 23:40:03 UTC 2010 i686 GNU/Linux

Solaris 8/sparc64

tg@stinky:~ $ gcc -static x.c
x.c: In function `foo':
x.c:27: warning: function returns address of local variable
tg@stinky:~ $ ./a.out
595f0 59bf0 59d00 ffbefbb4 ffbefb5c
tg@stinky:~ $ ./a.out
595f0 59bf0 59d00 ffbefbb4 ffbefb5c
tg@stinky:~ $ ./a.out
595f0 59bf0 59d00 ffbefbb4 ffbefb5c
tg@stinky:~ $ gcc x.c
x.c: In function `foo':
x.c:27: warning: function returns address of local variable
tg@stinky:~ $ ./a.out
20950 20f50 21060 ffbefb3c ffbefae4
tg@stinky:~ $ ./a.out
20950 20f50 21060 ffbefb3c ffbefae4

MidnightBSD/amd64

mirabilos@stargazer:~ $ gcc -static x.c
x.c: In function 'foo':
x.c:27: warning: function returns address of local variable
x.c: In function 'bar':
x.c:33: warning: function returns address of local variable
mirabilos@stargazer:~ $ ./a.out
0x800603080 0x800605040 0x800700000 0x7fffffffe62c 0x7fffffffe62c
mirabilos@stargazer:~ $ ./a.out
0x800603080 0x800605040 0x800700000 0x7fffffffe63c 0x7fffffffe63c
mirabilos@stargazer:~ $ ./a.out
0x800603080 0x800605040 0x800700000 0x7fffffffe62c 0x7fffffffe62c

uname: MidnightBSD stargazer.midnightbsd.org 0.3-CURRENT MidnightBSD 0.3-CURRENT #1: Thu May 27 22:13:45 EDT 2010 root@stargazer.midnightbsd.org:/usr/obj/usr/src/sys/GENERIC amd64

Debian sid/mipsel

(QEMU, thanks to Aurélien! Debian unstable from approx. Jan 2010)

root@debian-mipsel:~ # gcc-4.4 -static x.c
x.c: In function 'foo':
x.c:27: warning: function returns address of local variable
x.c: In function 'bar':
x.c:33: warning: function returns address of local variable
root@debian-mipsel:~ # ./a.out
0x4aa740 0x4aa750 0x2aaa8008 0x7fa417e8 0x7fa417d8
root@debian-mipsel:~ # ./a.out
0x4aa740 0x4aa750 0x2aaa8008 0x7fc67708 0x7fc676f8
root@debian-mipsel:~ # ./a.out
0x4aa740 0x4aa750 0x2aaa8008 0x7fb68238 0x7fb68228
root@debian-mipsel:~ # ./a.out
0x4aa740 0x4aa750 0x2aaa8008 0x7fc586c8 0x7fc586b8

uname: Linux debian-mipsel 2.6.32-trunk-4kc-malta #1 Mon Jan 11 03:45:08 UTC 2010 mips GNU/Linux

Gentoo GNU/Linux on amd64

gcc-4.4.4, glibc-2.11.2-r0, 2.6.35-rc4 x86_64

0x20cc010 0x20cc030 0x7fef0c497010 0x7fff32148fec 0x7fff32148fec
 0xa35010  0xa35030 0x7f575d0e4010 0x7fff0dd7220c 0x7fff0dd7220c
0x1f90010 0x1f90030 0x7f8657107010 0x7fff6116813c 0x7fff6116813c
 0x9dd010  0x9dd030 0x7f1eab0a6010 0x7fff3dcc638c 0x7fff3dcc638c

Conclusion

Not everyone does ASLR… but there’s enough variety (and with eglibc’s AT_RANDOM even proper entropy) inside for our purposes. On OpenBSD and MirBSD, we’ll still use KERN_ARND as it’s extremely cheap entropy (code paths checked on both) but not for every call of $RANDOM. On things like Debian/m68k mksh(1) ought to have gained a possibly noticeable speed-up.

Back home

11.07.2010 by tg@
Tags: bug debian event geocache mksh news release snapshot

Bordeaux was very nice (and towards the end much cooler… it’s actually hotter here at more than 50½° north – too warm to think, or do anything) but the LSM/RMLL was very french. They’ll be in Straßburg and Lüttich the next two years so we can probably be expected to attend. I don’t think I can eat duck (which, in south-west france, is a vegetable) or like all that classic french multi-course food so much, but I had enough Couscous Merguez and Thé à la menthe fraîche… and similar good stuff. Many people spoke English and actually asked me whether I do (probably they couldn’t bear me trying to spea^W^W^Wbutchering the language of the Grande Nation) and in general were a friendly bunch. I did see some people with machine guns in the city on the last day, though. No idea what/why… didn’t dare asking ☻

Just another reason to boycott flying: Mario Lang (one of the speakers) was apparently held on the airport and treated as a terrorist due to his Braille line… they thought it was a bomb or somesuch thing.

Read on for more…

Travelling with the Thalys and TGV was nice (but I loathe the Métro parisienne… they should build a ring train like the Berlin S-Bahn and just put another stop before Paris Nord and Montparnasse for people who just want to switch trains to take the ring train to the other line). And I want air conditioned trams in Germany too!

I met Uriel (invited him for some food and talked lengthy with him and some 9grid guy), XTaran (who was rather busy organising things), and a number of other people. Did some PGP keysigning as well. There’s now an experimental MirOS presence at Launchpad, not sure what exactly we’re going to do with it but, as Canonical does not care (as Jonathan said in his talk – great slides, by the way, really impressive), there’s no harm in having it. Some Perl guy from America (USA… just to make sure ☺) wanted a photograph of me with a sign “I love CVS” just so people back at home would believe him he’s met such a person *grins* of course I plugged in a little advertising but cvs(GNU) is honestly good. The forge hacking session was a little under-visited (but still a success in terms of getting more communication and maybe collaboration underways, especially thinking of common interfaces, DC, semantic web, OSLC-CM) and since the room was (in contrast to my hotel room and the trams!) not air conditioned we didn’t get much hacking done. The Debian booth was about 40% of one FOSDEM style table wide… and subsequently crowded. There were more people (of course, I was trying to get mksh into Haikuports, Mandriva, and other things; talked about KDE 3.5.11 (Trinity), Qt 3 vs Qt 4, and kwalletcli, and in general to a not-so-usual bunch of suspects – like I said, LSM/RMLL really is pretty french-only).

It is too hot, but I still committed src/etc/rc,v version 1.110 which you want to upgrade your /etc/rc to before upgrading mksh(1) in MirBSD. (All in the name of better performance on platforms such as Debian/m68k and not raiding Linux’ inferior RNG… but it does simplify things.)

I could probably write more but at the moment just want to lie down and die until it gets cooler… even the rain didn’t help. My feet hurt (Montparnasse-Bienvenue didn’t help) too.

The current version of mksh had use of arc4random(3) removed, including “set -o arc4random”, to speed it up (on some architectures, a lot) – this will break some existing scripts (such as /etc/rc *cough* on MirBSD…). Hence I decided to publish the next version of mksh(1) as R40 based upon current development, and defer plans for associative arrays (and multidimensional arrays) for mksh R41. There’s also already the change to Build.sh arguments, so this suits me quite fine.

(Read: if running MirBSD, don’t upgrade mksh at the moment.) There will be a new MirBSD snapshot once this is fixed, maybe a few more changes to the shell for better POSuX compliance, and the recently mentioned patent on LFNs (long filename) in FAT will be taken into account with a patch to msdosfs.

I’ll travel to LSM/RMLL 2010, the Libre Software Meeting (Rencontres Mondiales du Logiciel Libre) tomorrow until the weekend, to hack some on FusionForge (this is worktime for me), visit XTaran, Uriel, and maybe a couple of other “usual people”.

Thundersday, between 10:00 UTC and 12:00 UTC, eurynome will be shut down by gecko2@ due to power supply maintenance on the host system data centre.

We have a new mirror in the Americas, thanks a lot to Mike 'Fuzzy' Partin! Benny will mention it on the webpages once it’s working.

In response to a planet.d.o series (mentioned in #grml on IRC) of postings: In a sensible shell, Esc+# not only pushes it back but also re-enables the command. Try it out: l s Esc # Cursor-Up Esc #

tg@blau:~ $ #ls
tg@blau:~ $ ls

The command sticks in the history, and is not immediately shown in the next interactive input line, which I consider a plus in most use cases. Anyway, try mksh (just a-g i it), there are a lot of goodies. I found out about Ctrl-O only a year or so ago myself…

I wonder why schizo didn’t write about how to do it in posh tho ☺

Hello, World!

22.05.2010 by tg@
Tags: mksh

Just a random status update: I’ve been too busy with the dayjob, now ill for about two weeks already. There have been some minor mksh fixes but I’d still like to catch up on the Austin ML postings before releasing R39d; Android discussion is live. The base system has some issues found, but I will take a while (no hacking mood, even when not ill).

Scan this! and Google’s playable logo is just weird (at least no sound in Opera) but at least it’s not Fläsh. I played it but there’s no second level. Tonnerre likes «Forth op de Fiets» (as a pun on Ruby on Rails), which lets me remember the Forth Glockenspiel.

Sorry for the lack of updates, but MirBSD is still pretty much a fun project, and Benny is working on his Doctor thesis too.

FWIW: Patents on software are evil and times are worsening. I suppose, if we’ll be able to continue MirBSD at all, I need to disable FAT LFN support. Sucks.

mksh R39c released

25.02.2010 by tg@
Tags: mksh

The MirBSD Korn Shell R39c has been released. This upgrade is strongly recommended for everyone. Focus is on minor but important bug fixes. The recently introduced list of caveats contains language-relevant user-visible changes.

We are proud to announce that the android-x86 project's /bin/sh is now an mksh(1) as well.

Update: This wlog entry uses aggressive tone because I somehow needed to vent frustration from using some of the tools. I should probably provide some constructive critics, too... but this is a rant. Be warned.

Keysigning is useless. I boot up a suitable live GNU/Linux system, install signing-party, take the trouble that is to set up caff, transfer my secret key from the secure box, sign. I think caff providing the keys in a different order than they're given on the command line sucks and just run caff once per key. I did even start Postwreck. But no, people just don't accept any mail from "EHLO grml" systems, and I still cannot control my reverse DNS despite having a static IPv4 address (and IPv6, which looks to be unused). People also pretend I'm on dial-up. Great!

I will no longer participate in any (mass) PGP keysigning but will continue to do so on a per-person basis. Probably sign but one uid, either apply common sense and upload it to t̲h̲e̲ keyserver, or mail the entire signed key to one address.

By the way, how crazy is it that I need to use the deprecated $CONFIG{'mailer-send'} to pass an envelope-from to the mailer? It also suffers from the same delusion as e.g. nmudiff, namely that my Debian box is a fully set up workstation able to send out eMail and configured correctly. At least, it, unlike a number of others, does not assume I use mud (Mutt). grml…

Oh, and caff does a protocol violation (by always sending out GnuPG/MIME and not offering the standardised Inline OpenPGP), I think people just don't care about such. (There is a notation people can use to signal they want PGP/MIME, Inline PGP – which is called "partitioned" – or both (and which order of priority) but, alas, despite Inline PGP being the only one useful for the MUAs without integratin, and being more widely spread than that PGP/MIME crap, the followers of the latter do some (FSF-style?) kind of vendor lockin by not speaking anything else.

Anyway. I'm all for X.509 except there seems to be no sane CA (Startcom is... trouble, even with Opera; CAcert.org is dying). I'll just buy a certificate (not from Verisign though) for www, and roll my own again (I can do it, I have experience with that actually).

On an unrelated side note, still waiting for an OpenSSL patch for that recent TLS extension...

ObRant: password policies, be they required characters or any kind of length restrictions, suck. People I will eventually end up with less secure passwords on such systems, because even if some of mine may appear to be derived from some kind of dictionary (what language that is I'd be interested in, though...) they aren't, and I have my schemes. You got to have them with a gazillion of passwords used. And I probably will forget them more often (and sending them via eMail is also not a solution).

Unrelated notice: mksh R39c with bug fixes coming RSN.

(Updated 24.02. because I was, rightfully, told the language, and the title, were too strong. I also would like to excuse for going so low as to write an ad-hominem attack, which I've since redacted.

The MirBSD Korn Shell R39b has been released. This upgrade is strongly recommended for everyone. While being a stable series release there are, due to standards compliance and bug fixes, a number of caveats users should be aware of when upgrading. Also new, the list of full terms and conditions applying to it. Users (and distributors intending to support mksh for their own customers) should definitively read the caveats, although only corner cases are incompatible (ask for details).

The arc4random.c page now at least has some content, and a lot of links, too.

The kwalletcli page has been completely written by now. I'm proud to announce the availability of the CLI for the KDE Wallet, as distfile, as Debian squeeze/sid package (it's already in testing, yes), and as Debian lenny package, soon to be in backports (currently only in my own play repo, as I'm waiting for bpo upload rights – apparently, my PGP key wrecked the software).

I would like to apologise for the delay; I've been more-than-busy at first (preparing MirBSD for FOSDEM), then in foreign countries where people talk in weird tongues, then ill. I'm still not totally recovered, and there is also much catching-up work to do.

mksh R39b released

29.01.2010 by tg@
Tags: mksh

The MirBSD Korn Shell R39b has been released. This upgrade is strongly recommended for everyone. While being a stable series release there are, due to standards compliance and bug fixes, a number of caveats users should be aware of when upgrading; these shall be documented on the webpage RSN. (In fact I simply do not have the time to do so now, but will do it later.)

To do.

28.01.2010 by tg@

I’m going to FOSDEM, as usual

The MirOS Project will have a booth at FOSDEM 2010, business as usual. If you thought otherwise, you’re crazy ☺

I know I should write a wlog entry about the BSP, write more, release mksh R40, fix the TaC of it and the kwalletcli webpage (thanks again, it’s now in Debian sid!) etc.pp but I also need to prepare an ISO for FOSDEM, etc. Heck, I should prepare a talk for FOSDEM, but I’m not going to. If I need to stand there and talk, I’ll talk, not hold a presentation. I’ll just see what people are interested in, talk about The MirOS Project, and improvise.

I’m busy, and there’s only so much computing you can do in a day. This does include the dayjob. At least, my NMUs are in Debian now and probably can help people (and I submitted info about other bugs too).

Anyway, watch the news in the months to follow… can’t talk about everything now.

Marc Fleury, JBoss founder joins the ranks of Tonnerre, me, and other people requesting that MySQL (and MariaDB!) please finally die. Everyone, don't even fork it. Use a real database instead. Or, at least, SQLite. Really.

We're going to FOSDEM 2010 (of course – I've been at every FOSDEM that was not just an OSDEM, Benny and gecko2 are regular attendees as well, as are other projects of mine such as FreeWRT and Debian GNU/kFreeBSD, by proxy). There will be a recent MirBSD snapshot I've yet got to build, with the new floppy format ustarfs (idea, but no single line of their stinking ridiculously huge code, stolen from NetBSD®) and other improvements (albeit less than I wanted to get done by then). The days before, I'll attend the first FusionForge meeting to break up the French Cabal, with my work hat on. That is also my first time in France (outside of the Elsaß). People, make a good impression on me to overcome the classic prejudices ;-)

This weekend I'm going to meet my Debian Application Manager zack, have some good beer (ugh... first this, then Paris, then good belgian beer...) and fix some bugs, all while learning even more. Sounds like fun, but I almost feel overwhelmed, in contrast to the years of much less travelling from my past. I've also started sort-of mentoring Simon, one of our apprentices at work, into the Debian processes. (On an unrelated side note, formorer recently said bpo will become bp.d.o soon. Great!)

Please don't laugh at this excuse for a webpage, as I've yet to fill it in, but my CLI for the KDE Wallet is hereby deemed ready for public consumption, with a bug-fix release 2.01 (bugs actually found during preparation of a port to Debian sid and KDE 4, which is much much worse than KDE 3, plus it looks so absolutely disgusting I'm not even sure Windows® Mistda is worse). I hope the package will end up in NEW soon (and once progressed to testing I may be able to make the KDE 3 variant official via lenny backports; my WTF *.deb repo will hold them until then.

There are more webpages I need to fill in... mksh's TaC, arc4random (which needs some major redesign as well) and BSD::arc4random, the RANDEX protocol (entropy exchange over IRC) and its plugins and patches, ...

Not just Mac OSX (and, I hope, iPhoneOS) will soon come with mksh(1), but also Android (I prepared patches to make it /bin/sh, which works quite well – although I need to find out how to make a hardlink so that #!/bin/mksh scripts will run) and Maemo, for which I wrote an mksh package in a garage project, which also needs some love w.r.t. testing on actual devices, menu integration, etc. (Please contact me if you can help with either of the three.) We also have «lewellyn:#ksh» making a package for the new OpenSolaris system (thanks again). People persuading Apple to put it on the jesusPhone are also welcome. (This does not mean I endorse any of these – right now, I'd probably get the most of a WinCE PDA with built-in GPSr, WLAN and maybe GSM/GPRS.)

English and French native speakers, please review, and Dutch native speakers may contribute a translation of, our flyers. (Source code for these is not available, sorry. Benny makes them in Quark on System 7 in Basilisk II, used to be Classic until Apple yanked it. But still, they use only free fonts, free imagery or such the MirOS Project is allowed to use, and beat every single other FOSS project flyer I've ever seen by far!)

There's probably more I could write, I bet I forgot half of it anyway, but I'll leave it at that for now. Get yourself a nice cup of hot chocolate, pour an Espresso into it, and enjoy the mix with a piece of cake (I'd say strawberry or mousse-pear but all they had was cassis-créme) and pity me for not knowing any French next month.

I smell an antitrust case coming up

12.12.2009 by tg@
Tags: mksh rant security

Oh the joy…

20:54⎜«smultron» mira: i just upgraded the big server from 10.5 to 10.6... and apparently the upgrade script removed /bin/mksh... now I don't have a shell and quits immediately... any ideas?

My suggestion – ssh -t servername /bin/bash --login – doesn’t seem to help:

20:59⎜«smultron» oh great
20:59⎜«smultron» ssh just keeps asking for the password
20:59⎜«smultron» then gives this:
20:59⎜«smultron» Permission denied (publickey,gssapi-keyex,gssapi-with-mic,keyboard-interactive).

Maybe gecko2@ can help… or bsiegert@… who knows.

In other news, no reaction at all on the Android front on my proposal to get mksh either to replace NetBSD® 0.x ash, or, at least, add it for developers.

I managed to create an avd "Android 2.0-current", with stuff completely built by myself. Now I "just" need to get project/external/mksh.git to be created and writable by me. Or, even better, nuke that NetBSD® ash they're currently using and replace it with a sensible shell, at least mksh-small. Then adb can be built without -DSH_HISTORY (which, with mksh, is required for usability).

I wonder if I could take over Mæmo as well... *grins*

On unrelated side notes, I'm trying to get the "debian" tagged entries aggregated on Plänet Debian, and I'm – again – in the NM process trying to become a DD, with slightly different goals this time. (But I'd also like these porting machines... 'sides, there's still an mksh+dietlibc on hppa bug open...)

I also got HP-UX back at HP PvP (not player versus player though ;) for mksh(1) porting/testing. Sadly, Itanic only, no humppa machines.

In case someone ever needs it, a collection of scripts called BitWeaver → MediaWiki does exactly that and has been released under GNU GPLv2 (only). Cheers!

The GNU Project is famous for its coding style – Linus Torvalds even suggests to print them out, but not read them; burning them is a great symbolic gesture. Legibility issues aside (Linus’ own are interestingly similar to style(9) aka KNF, the BSD coding styles), why is that so?

mksh-current (R40+) now supports pathnames in arbitrary lengths on Debian GNU/Hurd (I think; I only could test on gnubber.bddebian.org that it compiles, seeing that all existing installations set sysconf(_PC_PATH_MAX) to 1024 to be consistent with POSIX PATH_MAX) by using some glibc-only functions. This is because Hurd does not have PATH_MAX (some older systems also don’t have it, but there, we just define it to POSIX 1024 and good is). Now, why? Simple: because the GNU coding style says to have “no arbitrary limits” in your code.

I would like to call that ridiculous, but it’s actually dangerous: if you are on a 32-bit machine and have a pathname of 512 MiB, you’re in danger of freezing your system or at least crashing mksh, even if you have the full 4 GiB worth of RAM, due to the amount of copying and carrying around pathnames. This is a security relevant issue, in my eyes.
Now on to the ridicule part: This is Open Source, people! Change the limit (as it’s a – only one – constant in a header file) and recompile everything! Simple as that! The BSDs do it all the time! In GNU, it’s even simpler because you force developers, redistributors and some users to actually give up freedom and require them to put the source code alongside. Now, why doesn’t anybody see this? I can’t be the only one, can I? (I actually think that changeable limits would suffice the horrid GNU coding style, but find myself reluctant to read it again due to its sheer size – similar to their licences…)

On an unrelated note, I hope to have a bi-arch ISO format snapshot on BitTorrent by end of the month. Mika is also trying to put a new Grml release out by then, of course with an up-to-date MirOS bsd4grml

New MirMake (mandatory update)

20.10.2009 by tg@
Tags: bug mksh

Due to a bug mksh fixed after inheriting it from pdksh via OpenBSD ksh (oksh), which probably got it from AT&T ksh88 (ksh93 exhibits the correct behaviour, as does posh), coupled with the unfortunate lines

	CC=${CC:Q} ${MKDEP_SH} -a ${MKDEP} ${CFLAGS:M-[ID]*} \
	    ${CPPFLAGS} $$files; \

in <bsd.dep.mk, where ${MKDEP} can be the mkdep(1) option -p as well as additional CPPFLAGS like -I... (which I actually found in our tree), you absolutely must upgrade your MirMake package, as well as mkdep(1) in the base system, before upgrading to mksh-20091015 or newer. (Note that R40, which will carry the breaking fix, has not been released yet, but FreeWRT uses an mksh-current snapshot bearing it with still major 39 enacted.) It is actually pretty hard to work around, see the mkdep source code for details. There are basically two things to take care of:

  • For each x in getopts "...x..." c, make sure you not only case (x) ... ;; but also either case (+x) ... ;; or have a (*) ... ;; default trailing block, because mksh(1) getopts will also catch foo -x +y -z sanely. This is, in contrast to oksh, not disablable with a shell option.
  • If you case (\?) cmds ;; (either explicitly or via a default block), special rules apply: if you do anything other than exiting from there (e.g. via usage), $OPTIND will differ: newer shells count this option, olders don't.

The new distfile {RMD160 (/MirOS/dist/mir/make/mirmake-20091020.cpio.gz) = b9ac1258bc66b3d0d63537cc82d02c91408d1ba8} has been uploaded for your convenience already and will be integrated (after testing) into both The MirPorts Framework and FreeWRT as soon as we get to it, probably tomorrow.

mksh’s Build.sh can now generate Makefrag.inc snippets that reflect its environment and can be used like Rebuild.sh or integrated in, for example, the Android.mk files.

mksh(1) also is a great compiler testsuite: it regularily(!) points out bugs in gcc’s -fwhole-program --combine and helped to find (and fix) bugs in huge things like SunStudio, old things made modern like pcc, as well as one-man projects such as nwcc
I wish compiler authors would just use mksh as testsuite regularily.

I released a new version of jupp for Unix® today: joe-3.1jupp12 (including a backported hex editing mode, as a late birthday present for waga (from IRC) who asked for it, a plethora of bug fixes, raw octet support in UTF-8 mode, UTF-8 support in the 8-bit modes, and more changes. If you already use it, update please. If not, give it a try! It’s included in at least FreeWRT already, and I build Debian packages (link on the jupp website) in my “play repository”, as well as packages for Univention Corporate Server at work (from these packages). I’m also using it on Solaris 8, Solaris 10 and AIX 5.3 ☻ (and many more)

There are currently quite a lot more things I would like to do but can’t seem to find the time for. I had originally planned a new MirBSD snapshot, including ISO, for mid→end of September – now, end of October seems more likely (if I can get a few days off work and some RAM for the SPARCstation 5). Several things, like the mksh prompt wlog entry and webpages for a few subprojects (BSD::arc4random; arc4random.c; the RANDEX protocol, plugins, implementations, proxies; kwalletcli; keystash) are lacking too, and the Grml project is also expecting code from me. Sorry. I am, after all, human too…

As sent to the mailing list I had a chance to compile mksh for Android today, thanks to Waldemar confirming that it works (with OpenADK's eglibc linked in statically) and CeKay's help in mastering the SDK and NDK. The posting contains all relevant information; it is virtually impossible to write an Android.mk file though.

On the other hand, I did submit a patch for Bionic (the libc) to have sys_signame[] like real BSDs, so that we could simply hard-code the appropriate CFLAGS and CPPFLAGS and let the NDK compile the mksh source *.c files (signames.inc would no longer be needed).

Maybe we'll submit mksh for inclusion somewhere, as this would be the first really usable shell. For this reason, I have uploaded a binary (gunzip(1) first) at ports/mksh-39.9.20090929-android15.tgz (MD5 64ee103453d65e947f2beb1aeb6450d2) which you can place in, for instance, /data/mksh then put a (modified, as the ls(1) and id(1) and possibly other utilities are more than reduced) dot.mkshrc from the source (CVSweb) as /data/.mkshrc and run it from adb (which, by the way, is the one responsible for some cursor keys etc. not working, as Android and mksh(1) do both fine in that regard) with # HOME=/data /data/mksh in the shell. I may post an Android specific patch for dot.mkshrc some day.

Update 10.10. – .gz → .tgz and Cygwin binary now too!

smultron, MidnightBSD developer and project member, and graphician / art master for the MidnightBSD Korn Shell (among other things), has pointed me to a blog article: 8 Useful and Interesting Bash Prompts and asked for an mksh(1) translation. These will not be minimalistic (I can do quite some things), but focus on containing no control characters or other things not easily cut-and-past'able. Made more readable, too.

Small note: all mksh examples assume the following "setup code" in front of the PS1=... line. Yes, PS1 is set twice (makes the code more readable and perform better). Most of this is from the mksh(1) manpage or dot.mkshrc and relatively portable, which is why some things seem more complicated than needed. They do, however, use some rather recent mksh features; if there's real interest in making it work with, say, Debian etch mksh R28, eMail me. Note: that → character is a literal Tab (the "[→·]" Tab-Space occurs often).

	ca=$(print -n '\001'); ce=$(print -n '\033'); cm=$(print -n '\r')
	: ${HOSTNAME:=$(ulimit -c 0; hostname -s 2>&-)}
	[[ $HOSTNAME = @(localhost|*([→·])) ]] && \
	    HOSTNAME=$(ulimit -c 0; hostname 2>&-)
	: ${USER:=$(ulimit -c 0; id -un 2>&- || print \?)}
	if (( USER_ID )); then PS1='$'; else PS1='#'; fi

	function twd {
		typeset d=${PWD:-?} p=~

		[[ $p = ?(*/) ]] || d=${d/#$p/~}
		print -nr -- "$d"
	}

If you do not need exactly the effect of "\w" (tilde unexpansion), use $PWD ipv $(twd) below. Note that $USER and $HOSTNAME are usually evaluated only once; escape them if you need it evaluated every time.

  1. Show Happy face upon successful execution
    	bash$ PS1="\`if [ \$? = 0 ]; then
    		echo \[\e[33m\]^_^\[\e[0m\]
    	else
    		echo \[\e[31m\]O_O\[\e[0m\]
    	fi\`[\u@\h:\w]\$ "
    
    	mksh$ PS1="$ca$cm"'$(if (( $? )); then
    		print "'"$ca$ce[31m$caO_O$ca$ce[0m$ca"'"
    	else
    		print "'"$ca$ce[33m$ca^_^$ca$ce[0m$ca"'"
    	fi)'"[$USER@$HOSTNAME:\$(twd)]$PS1 "
    

    I do think Unicode makes things more fun though:
    	mksh$ PS1="$ca$cm$ca$ce["'$(if (( $? )); then
    		print "'"31m$ca☹"'"
    	else
    		print "'"33m$ca☺"'"
    	fi)'"$ca$ce[0m$ca[$USER@$HOSTNAME:\$(twd)]$PS1 "
    
  2. Change color on bad command
    Coming soon... colour changes are okay, but I wonder why anyone would want the history number. I've got to discuss this a little, and think about ways to use such information with mksh, and if I really want to transcribe the prompts 1:1 (it's possible by just incrementing a counter, but, hey, you don't want that). This is actually a csh feature, and who'd really use csh? I'm on my 80486DLC notebook right now, so my debugging methods are a little limited, but I promise a larger article later. I just needed to get started.

This article will be updated in-place.

mksh feature weeks

07.09.2009 by tg@
Tags: mksh

It's feature weeks for mksh(1). You almost certainly want to follow the mksh-current changelog. Please test all the new features, as well as make sure that nothing breaks nor I introduced other regressions, such as making resource usage worse (disc, CPU, memory). Please be reminded that many more deep-reaching changes will follow in the next few hacking days.

Thanks to David Korn (the Korn in AT&T Korn Shell, ksh88/ksh93) for discussing features and helping with finding syntax and semantic of some of the more complicated language pecularities.

On an unrelated note, cvs(1), lynx(1), ncurses(3), some other things, and a lot of ports have also been worked on. The next snapshot is planned for mid to end of September, bi-arch. Stay tuned!

I took the weekend to release a couple of things… MirBSD™ base code stuff and base releases (printf.c for mksh in Debian; arc4random.c for Win32 and other non-BSD OSes, now with a HKCU key used in addition to the HKLM seed key which may not be writable; MirMake; MirPorts Package Tools; mirdate – rdate(8)), jupp, mksh including a new PDF manpage, KWalletCLI, and the RANDEX plugin for XChat (Win32, BSD and *nix). I also prepared for the inclusion of more Debian source packages in my CVS “home subtree” and creation of SRPMs for more software (not in CVS though). I cleaned up the mess that were the X11 dist sets in base, cleaned up compiler warnings on half a dozen or more platforms in several parts of the code, fixed bugs in a lot of subprojects, integrated things better, updated the BSD::arc4random MirPort as well as TinyIRC, MirSirc, the irssi and XChat RANDEX plugins to include better version reporting and, for XChat, seedfile support and better responsibility. Now all I need to do is build more binaries and ports (DEB RPM OpenBSD FreeWRT etc.) of the subprojects, update Lynx in base and ports (there is a new major release out, even), update MirGRML, fix the HDD in my sparc, compile stuff, … you see I’m busy.

Here’s a “checksum and link collection” for today’s finest:

  • Simple CVS file drops
  • RMD160 (printf.c.1.10) = 8e8b88401a04474db973be07540a79b129919ff5
  • TIGER (printf.c.1.10) = 3cec4bc24074e88c7889143d19f7659ced17482115ea5afb
  • 3098389975 10575 /MirOS/dist/hosted/other/printf.c.1.10
  • MD5 (printf.c.1.10) = d09ae97aebac104f834d3d3ddd1702ca
  • RMD160 (arc4random.c.1.16) = b0caa3509d2cade6d86cb2c13e6b8817ced2d9a9
  • TIGER (arc4random.c.1.16) = ef6d7a281d451e28434b0e003990eebb47edd0cd4d899fd1
  • 2199066621 12558 /MirOS/dist/hosted/other/arc4random.c.1.16
  • MD5 (arc4random.c.1.16) = e8376a9b51c0ce08f5ed20722b05cad3
  • Simple subproject checkouts
  • RMD160 (mirmake-20090801.cpio.gz) = 79e0d15aab4c7a05690e66769c12dbeb3d99daa1
  • TIGER (mirmake-20090801.cpio.gz) = 2c6642b9515f38e736386945e72c06f402134ebf898613de
  • 788720631 372063 /MirOS/dist/mir/make/mirmake-20090801.cpio.gz
  • MD5 (mirmake-20090801.cpio.gz) = 47c63503210054d86db80040474f1f71
  • RMD160 (pkgtools-20090801.mcz) = 482dcf4b915a10bb6b76859f0c1755b67d6343bb
  • TIGER (pkgtools-20090801.mcz) = 3a622ac3c895c4af9df719dd30cfd3fe45e6d719cc34db5e
  • 2864495035 180188 /MirOS/dist/mir/pkgtools/pkgtools-20090801.mcz
  • MD5 (pkgtools-20090801.mcz) = 87378c95bde1c219d4a09e6bb8ccb897
  • RMD160 (rdate-20090802.tar.gz) = abac9ae8a08ac566d6c0396d39cd5d2cd724f7b0
  • TIGER (rdate-20090802.tar.gz) = f38a164e9d77412203349f79e8033c413335dd6f43a5cbf5
  • 3840714105 11987 /MirOS/dist/mir/rdate/rdate-20090802.tar.gz
  • MD5 (rdate-20090802.tar.gz) = a8fa4550b5a77cff6db1ed0a9d8aa357
  • JUPP (including Win32 binary)
  • PDF manpage
  • HTML manpage
  • RMD160 (joe-3.1jupp11.cpio.gz) = 7ade55cb8511600b3a9d77f37bc581b2d09ab2aa
  • TIGER (joe-3.1jupp11.cpio.gz) = b7bb4aa464b705e697ab2a52ad75fc8755a5817bfb83e09a
  • 805235529 419484 /MirOS/dist/jupp/joe-3.1jupp11.cpio.gz
  • MD5 (joe-3.1jupp11.cpio.gz) = 1e2f21a6fdebe678b125e96806267f33
  • RMD160 (JWIN31B.EXE) = f9eb9f6b3bd2a1bb5874e36d2dcc6dbdaabf75cc
  • TIGER (JWIN31B.EXE) = 771461b752114978ed64f67c01e3ef22a9a9cdf76fda6b11
  • 674256238 948176 /MirOS/dist/jupp/JWIN31B.EXE
  • MD5 (JWIN31B.EXE) = b2d3f1044221fdea76f15621e94e1ae4
  • mksh (including Cygwin package)
  • PDF manpage
  • RMD160 (mksh-R39.cpio.gz) = 5a5bcbe288e722f9772e27d2fdc36eee174bbb7b
  • TIGER (mksh-R39.cpio.gz) = 2a2c08ccf5e27365aa652663629789ade93b3d30c0d1d51f
  • 4103085544 278476 /MirOS/dist/mir/mksh/mksh-R39.cpio.gz
  • MD5 (mksh-R39.cpio.gz) = b2eeb4fe4ccac2704e1440e53cd2672c
  • RMD160 (mksh-39.1-cygwin.tgz) = 0cecd4ffb72f2d51a5c935da58e67350fab10e81
  • TIGER (mksh-39.1-cygwin.tgz) = 3157abadc40696bcb8df1d3574df571b728bef3d4d2ac2f2
  • 2818578374 144625 mksh R39 for Cygwin
  • MD5 (mksh-39.1-cygwin.tgz) = ca949841e39721be666e6a82803e7769
  • KWalletCLI
  • RMD160 (kwalletcli-1.00.tar.gz) = f04ebd39e9714212a915b6d7d4524c8cc2daaee7
  • TIGER (kwalletcli-1.00.tar.gz) = 0fc673c0c813608f0f0d863dfd924a6d62a8507c7bdf361b
  • 2355082724 11524 /MirOS/dist/hosted/kwalletcli/kwalletcli-1.00.tar.gz
  • MD5 (kwalletcli-1.00.tar.gz) = 76ef3c1d611a11ea13dc805d67d82208
  • RANDEX plugin for XChar (including Win32 binary)
  • RMD160 (xchat-randex-1.10.tar.gz) = fd61babbf4e5189f69dae8eb664ee2780433bf4b
  • TIGER (xchat-randex-1.10.tar.gz) = 6bd888b157fcd931e54b71e9778950cbfa675ae6b784ddd5
  • 2651117045 8702 /MirOS/dist/hosted/xchat-randex/xchat-randex-1.10.tar.gz
  • MD5 (xchat-randex-1.10.tar.gz) = d1585c5fae3ee531deeffc8314910553
  • RMD160 (randex.dll.gz.1.10) = a4aaa67cfdad1f9a1bcdc3eea797aff3a30703c4
  • TIGER (randex.dll.gz.1.10) = 55b2dcd7d790d28944d7424121cf5c6d4d386a99751fb556
  • 972086546 23998 /MirOS/dist/hosted/xchat-randex/randex.dll.gz.1.10
  • MD5 (randex.dll.gz.1.10) = 793ce548256efc6a23f7a37dde9215a2

An observation… mksh “print $RANDOM” on Cygwin is very slow, and the HKCU seed changes each time. This should be debugged, it shan’t unless RANDOM is being written to or 400k calls are done.

mksh has just been adjusted to behave as future POSIX will demand, after a lengthy discussion (on the bug-bash and miros-discuss mailing lists as well as the Debian bug tracker), for “set -u” (-o nounset). This, as well as the “stop () {…}” fix, must be tested extensively.

Therefore I urge all of you to

 % CVS_RSH=ssh; export CVS_RSH
 % cvs -d :ext:_anoncvs@anoncvs.mirbsd.org:/cvs co -PA mksh
 % cd mksh
 % (sh Build.sh -r && ./test.sh -v) 2>&1 | tee log

and possibly send me the log. See an earlier post for more information about testing mksh(1) snapshots, as well.

Tests for better standards compliance and bugs, especially in corner cases, are also welcome. Known are: all but the first command of a pipeline are run in subprocesses (made to be an mksh feature, not a bug); the lexer is not recursive (issues with parenthesēs and comments in COMSUBs); some IFS/whitespace issues. Fixes for these bugs, and maybe for the regression tests (they may or may not be correct) are desirable… as well as development related communication.

On an unrelated side note, aptituz told me that Debian etch already had Debhelper v5, and as such, the mksh package has been converted over from Debhelper v4 (yay, RCS IDs in debhelper configuration files! but what about changelog (no-no) and menu? dunno…).

mksh(1)'s "set -u" handling will change RSN to match what POSIX will mandate in the next version, matching similar changes in GNU bash 4 and AT&T ksh93.

Most of the thread can be seen on the miros-discuss@ mailing list archives (although both MARC and GMane seem to not have all related eMails... weird).

RFH: mksh development

07.07.2009 by tg@
Tags: mksh

mksh development is mostly done by a single person, "the mksh team" (as seen on Debian derivate from Canonical that cannot be named forums, out of all places!), a.k.a. me, myself and I. Sometimes, actual users report bugs or even send in patches. I keep track of oksh's development as well, of course. But there are times I would like to get feedback on issues from other people working on pdksh or its descendants. I mailed, for that specific issue in question, the Debian developer who created the original patch which addressed the scenario except for a corner case (interestingly, as the world is small, discovered in a Debian(!) init script from a package maintained by aforementioned formorer, on a DomU running Lenny - don't underestimate the effect of synergy) but would really like to talk to some of the OpenBSD devs about it; they mostly know what they're doing, even if I worked on ksh for longer than most of them, many eyes do help, and most of the time I do not know what I'm doing XD

[Update] There's also the issue of inter-(POSIX-compatible)-shell discussion. For instance, "set -u" vs "$@", which has come up in Debian #522255 because GNU bash4 decided to switch to the behaviour used by the Bourne shell (from V7 to SVR4.2), all Korn shells, ash and its descendents (like posh) except dash, to not treat it specially. (Funny too how they suggest 「${@:-}」 or 「${@:+}」 instead of 「${1+"$@"}」 (from the GNU(!) autoconf texinfo documentation) as replacements.) Oh well, zsh also behaves like bash2/3 and dash, but then, it's not even a POSIX compatible shell. *sigh* Now I wonder what AT&T ksh93 will do and a confirmation if it's indeed being forcibly changed by POSIX (IMHO, they could at least "agree to disagree", like they usually do, and make it vendor defined, so that scripts could not depend on it - "set -u" is something I don't use anyway).

So if you're interested in the further development of MirBSD, The MirOS Project, one of its subprojects, such as The MirPorts Framework, mksh(1), MirMake, even jupp-2.8 or jupp-3.x, please talk to me.

[Update] Do the same for POSIX compatible shell discussion, if you are going to take mksh, its goals, needs and history seriously. (Yes, it also has bugs, like a non-recursive parser troubling COMSUBs, but they may be fixed long-term, especially if people contribute. Ideas, at the very least.)

Thanks in advance.

mksh R38c released

10.06.2009 by tg@
Tags: mksh

The MirBSD Korn Shell R38c has been released. This is one of the everyone-should-upgrade versions because of the fixes for crashes and the likes. Read the online manual page in HTML – mksh(1) – or as PDF for printing.

mksh R38 released

31.05.2009 by tg@
Tags: bug mksh

The MirBSD Korn Shell R38b has been released. It adds portability to QNX 6.4, a built-in base64 decoder and encoder written in mksh itself, and most importantly fixes a regression introduced in R38 causing memory corruption.

This – and a lot more bugs – were discovered while porting Git (resp. running its test suite) for Michael Gebetsroither (grml).

With another mksh release out, and the first feedback (actually a patch with explanation – naaina ported it to the newly released QNX 6.4, 10x) already in, I would like to request user feedback if mksh compiles okay for them, the regression test suite results, and if it does its job – especially on the more obscure platforms. Current plans for R36b are mere portability and bug fixes, and maybe some more of the Syllable, Plan 9 or Haiku issues touched if someone does it.

On the other hand, I'm really glad I get feedback, even patches from people I've never heard of beforehand, which even touch documentation as it should be. One had the luck of adding a feature that had been, independently, requested in IRC mere days beforehand. You're welcome ;)

Let me plug a link to the fine manual page mksh(1) or its PDF version.

Thanks to all users as well, we cracked the 100 in Debian popcon a few days ago (102), even though it's down a tad right now, to 94.

Plans are to get promoted to Arch Linux Community from AUR, included into Mac OSX, Minix, pkgsrc® (as bootstrap shell) and QNX by default, and the usual world domination in general. Hey, I'm fixing dietlibc bugs on Launchpad now, so low I've sunk, so gimme some rest.

mksh R38 released

27.05.2009 by tg@
Tags: mksh

The MirBSD Korn Shell R38 has been released. Most prominently, our developer wbx@ (Waldemar Brodkorb) has suggested to allow for expansion of “!string” style lines, and several things (string lengths, substring expansion) have been made more aware of UTF-8. Grab it as long as it’s hot!

mksh semantics for evaluating substring expansion ${strvar:pos:len} and string length ${#strvar} expressions has changed today. These operations now work on characters, not on bytes. Characters are octets in non-UTFMODE (which is pretty much the same as bytes, because mksh(1) is a BSD application and, as per style(9), allowed to assume certain things about the environment) and MirOS OPTU-8 multibyte character sequences in utf8-mode.

This means things like typeset -Uui16 -Z7 wc=1#${str::1} now do the right thing (getting you the MirOS OPTU-16 wide character value of the first character in the str).

mksh R38 will thusly be released RSN.

Solución al reto del script

16.05.2009 by tg@
Tags: mksh

asarch ha escrito un artículo acerca de cómo se usa la función isatty(3) de para verificar si un script tiene datos en stdin, quizas en "$@", o para imprimir su uso.

Para el soporte oficial del mksh eres bienvenido en su canal (en inglés, ya que no todos hablan el castellano) en #!/bin/mksh (sí, sí es un nombre válido XD) del Freenode PDPC (irc.freenode.net:6667).

Update: asarch ha corregido mi español... ¡gracias!

mksh $(…) evaluation bug

22.04.2009 by tg@
Tags: bug mksh

RedHat BZ#496791 is another example of a bug I documented better in the commitids 10049EF448F5F89A278 and 10049EF493039137B14 in mksh(1).

The gist is: $(…) are not parsed recursively but by a lexer hack, namely merely looking at matching parenthesēs; this needs to go away. Until then, this bug cannot be fixed.

And while at it, ((foo); bar) subshells need to be fixed so that they are not parsed as ((…)) arithmetic expression with a failure upon encountering a sole closing parenthesis.

The mksh plans list this.

Not an mksh bug

08.04.2009 by tg@
Tags: bug debian mksh

When R37c was brought out, I fixed a bug on (among others) IA64. The simple memory allocator added a pointer (or two, in Espie's) to the storage, placed before what the user got. Of course, gcc wanted to align the struct not taking this into account, failing evilly. Luckily, another FTBFS was not my fault, but sigsetjmp(3) was merely broken on S/390 with dietlibc; waldi fixed it in the meanwhile, but I uploaded another version of mksh to Debian for now whose mksh-static binary links against glibc instead and added me a TODO bug.

All the testsuite failures are certainly interesting though; the hppa one looks like a bug in ed(1) there; as to the others, either Perl, or binfmt_misc was configured to accept or drop (but not reject) shebangs præfixed with a BOM. Whatever.

Maybe I can now finally go back to working on MirBSD instead? :D
After all, we want a new snapshot (for NetInstall, at least).

The MirBSD Korn Shell R37c has been released. You do not need to update if you are already running R37b, unless one of the following items affects you:

  • Fix a file descriptor bug on Minix 3 (all users are affected, unless their Minix 3 version supports >64 fds already)
  • Support ACK on Minix 3 by means of a workaround
  • Change structure alignment and padding (affects people getting a SIGBUS or sometimes SIGSEGV on strict-alignment arches, e.g. IA64)

Since bsiegert@ has not indicated willingness to take over Freshmeat announcements of mksh releases, and I am almost unable and certainly unwilling to, this ceases our use of said site's services. Please use https://www.mirbsd.org/tag_mksh.rss to keep informed about things related to the MirBSD Korn Shell.

The MirBSD Korn Shell R37b has been released, incorporating a plethora of changes from R37, which was only available from CVS, and a few news by itself.

The new ‘-combine’ Build.sh option (gcc4 only) is about as noteworthy as the Minix 3 support. Users of Emdebian, FreeWRT and Beastiebox however will find that the new memory allocator has less overhead, i.e. the shell shrank in size. On the other hand, it is now much more standards compliant, especially wrt. “set -e”.

As such, mksh R37b is a strongly suggested upgrade.

Note: Freshmeat has a new website. I am utterly confused from it, both in Lynx and Opera, so I don’t know if this announcement has made it to there. I probably will cease to use Freshmeat, similarily as Slashdot, when it became unusable in Lynx. Damn “Web 2.0”!

mksh at Apple again

05.04.2009 by tg@
Tags: mksh

laffer1 remembered me of Benny’s Apple bug report about mksh inclusion into their base system. He also did it back then, as requested, and the bug #6735470 would be the result. Unfortunately, one still cannot get a publicly accessible version of them.

By the way, the R37b version is out.

[rant] FreeBSD® sucks!

29.03.2009 by tg@
Tags: bug mksh snippet

How about this:

	tg@herc:~ $ /bin/sh -c 'echo "0=($0) 1=($1) 2=($2) 3=($3)"' -- a "b c"
	0=(--) 1=(a) 2=(b c) 3=()
	tg@herc:~ $ nbsh -c 'echo "0=($0) 1=($1) 2=($2) 3=($3)"' -- a "b c"
	0=(--) 1=(a) 2=(b c) 3=()

Heck, not just NetBSD® ash, and almost any shell Debian can dig out (bash pdksh ksh93 zsh dash posh) behaves the same as mksh.

MidnightBSD (DragonFly and FreeBSD too):

	tg@mnbsd:~ $ /bin/sh -c 'echo "0=($0) 1=($1) 2=($2) 3=($3)"' -- a "b c"
	0=(a) 1=(b c) 2=() 3=()

And the best thing of all: I am to "fix" this in mksh, as the old FreeBSD® ash is documented to have that weird behaviour. (Well, there's always -DMKSH_BINSHREDUCED...)

This is not a fault of MidnightBSD (except they could fix mports to behave more POSIXish; after all, I had to do that to MirBSD and MirPorts often enough), don't get me wrong. It just surprised me, and since I'm glad they switched to mksh as /bin/sh I basically had to debug it.

I don't remember any more what NetBSD® 1.5 did that its /bin/sh could not be replaced by mksh, but they fixed it in 1.6 – K?buntu (the LTS version) fscks up similarily...

mksh, the MirBSD™ Korn Shell, is not only the MidnightBSD /bin/ksh, it is now their default /bin/sh as well! See the commit message for details.

laffer1@mnbsd and tg@mbsd worked together to create compatibility code for the FreeBSD® ash extensions; the version R37 of mksh(1) used already has been proven as /bin/sh in MirBSD and FreeWRT for some time, and Debian as well, which provided valuable corner case tests leading to eventual bugfixes.

We can only suggest others follow ☺ Apple, pkgsrc®, where are you?

mksh improvements

22.03.2009 by tg@
Tags: mksh

Today's commits should fix Debian #518359 (not tried yet, as looking at a 14" Belinea 10 40 40 CRT makes my eyes and head hurt), as well as improve a lot of more things.

The mksh(1) echo builtin now behaves more POSIXly in "set -o posix" mode (or when called as "sh", if that feature was set at compile time). The exact feature set is negotiable, if certain parties, say pkgsrc®, are interested.

Thanks to the OpenBSD developers for improving their ksh as well.

The ;& and ;;& features are delayed for now though, sorry.

We're in a feature freeze; people please test mksh-current on your system to make R37 rock!

The MirOS Project News in the upcoming February issue of the BSD magazine contains a mention of mksh R37 with a new memory allocator. This will, however, be postponed, because I could not get it right in time (even though I learned from the first tries – and wish I could write it in i386 assembly) and lost interest for now, as the current one still does work. I may revisit that at a later date.

Fixing the PS1 bug, maybe changing the echo builtin to SUSv3 (and nothing more), maybe porting, are on the list, besides continuing the Coverity Scan fixes.

more bugs – bootloader and mksh

01.02.2009 by tg@
Tags: bug event grml mksh snapshot

Again. I almost have a MirGRML+bsd4grml ISO ready (exactly 72 MiB), just for the fun of it, and so that people can toy with it – and test the integration. But nooo, I even hacked a disklabel, yet it wouldn’t access the filesystem. The “machine label” command shows what’s wrong (and hints how to fix it), and I missed a corner case in disc I/O due to adding two not-so-independent-from-each-other scenarios during the El Torito merge.

mksh has funny behaviour with ${foo/@(%)/x} failing – only in Unicode mode though.

more bootloaders *grml*

31.01.2009 by tg@
Tags: bug event mksh snapshot unicode

To counter the bug in Parallels Desktop, I rewrote all of our Master Boot Record (mbrldr, mbrmgr), Partition Boot Record (bootxx), and BIOS disc detection code in boot(8/i386). I've also changed the magics, API etc. a little between these, rewriting or removing quite some parts of both installboot(8/i386) and bootxx.sh a.k.a. self-installing bootblocks. Oh, and MBR and PBR are smaller, or rather, have more room for informatory texts and data block storage now.

Lucas “laffer1” Holt from that cat's BSD helped me testing, qemu and bare metal IBM X40 works, so I suppose it's good. Can't test on VMware Server 2 right now.

On the other front, I'm writing this wlog entry on MirGRML, which is finally sort-of finished (which is why it has much less Unicode than a regular posting from my laptop). lynx-cur in sid is broken though, due to it using GnuTLS. Gah! Anyway, I see a sid “wtf” repo coming, and it was workarounded; next time I'll just do things differently.

Lukas “smultron” , the graphician of our partner project mnbsd, helped a little with the label (while I'm not versed in that field, if I had gotten the right material to work with I could've managed it all alone this time), and all that's left now is the Live-CD part and baking (and testing! even on sparc...) the ISO 9660 image.

There will be 297 MirBSD Triforce™ DuaLive™-CDs (two will go to Kiwi land to swishy, one is already reserved for XTaran) and 50 MidnightBSD Live DVDs at FOSDEM.

gecko2@ and bsiegert@ were not too helpful today, although I have to admit that real life often has precedence. I merely read some and went to sleep early yesterday too (and disabled both POTS and cell phone so that I couldn't be waked, haha!) instead of hacking late. Luckily, the deadline is not that dead, and I'll almost certainly make it tomorrow, in time. Thanks again, Daniel Seuffert from AllBSD.de, for all support you and others give to the smaller BSDs. Heh, and laffer1 is not quite done yet with his ISO (or UDF?) either.

positive feedback

15.01.2009 by tg@
Tags: mksh

iMil of Beastiebox has apparently lauded me for mksh – someone came into our IRC channel #!/bin/mksh by means of it. Thanks, it is not often that people give feedback on things. According to them, code quality is very good. While many things are inherited this shows that the cleanup both OpenBSD and I did did pay off.

Tonnerre thanked for a script of mine he put to good production use: svn2cvs, which I already talked about. Glad to be helpful!

Sadly, at work we’d probably need cvs2svn. Not going to do. Besides, it won’t work that easily – their CVS doesn’t use commit IDs, and svn has no tools like rcs(1), ci(1), co(1) which are immensely useful.

New allegiances for mksh & Co.

08.01.2009 by tg@
Tags: grml mksh snapshot

mksh formed new allegiances: the Beastiebox Project (10x hubertf@TNF for mentioning) has added mksh(1) today, and the latest project of CcSsNET, CcSsLIVE (broken link removed 20100916), will include it soonish as well. Nicely, he already links to MirBSD, for he sits in our IRC channel usually and is one of the sparc users.

The next version of grml GNU/Linux might very well come not just with mksh(1) (except grml-small), but also with bootbsd.com née boot(8/i386) and bsd.rd (a slightly “tuned” version with added e3 editor and sans the Install/Upgrade/Shell prompt). On the contrary, I’m toying with the idea of adding a ports/sysutils/pxegrub for local boot along with a slightly tuned (add mksh at the very least) on the DuaLive ISOs if I find we have the space. We’ll have to work out something, licence and GPL-source-requirement wise, but we’re positive this would work out well. Considering just how good of a rescue system our bsd.rd kernel is, already, and what added benefit a minimal pretty standard Live Linux may provide.

Best bash tip ever! Use mksh!

31.12.2008 by tg@
Tags: mksh

Best Bash tip ever! is a little… interesting. Of course, mksh(1) can do it as well:

tg@bleu:~ $ head -2 /var/run/dmesg.boot
MirBSD#10uA4 (GENERIC) #1161: Fri Dec 26 21:05:59 UTC 2008
        tg@bleu.mirbsd.org:/usr/src/distrib/generic/obj/build/GENERIC
tg@bleu:~ $ r 2=3
head -3 /var/run/dmesg.boot
MirBSD#10uA4 (GENERIC) #1161: Fri Dec 26 21:05:59 UTC 2008
        tg@bleu.mirbsd.org:/usr/src/distrib/generic/obj/build/GENERIC
cpu0: Intel(R) Pentium(R) M processor 1.40GHz ("GenuineIntel" 686-class) 598 MHz
tg@bleu:~ $ fc -l
1       head -2 /var/run/dmesg.boot
2       head -3 /var/run/dmesg.boot

What’s best, the modified commands are written into the history, not the modificator itself.

Some of the commentaries are rather clueless too, not $! but $_ is the last word of the last command, in this case:

tg@bleu:~ $ head -2 /var/run/dmesg.boot
MirBSD#10uA4 (GENERIC) #1161: Fri Dec 26 21:05:59 UTC 2008
        tg@bleu.mirbsd.org:/usr/src/distrib/generic/obj/build/GENERIC
tg@bleu:~ $ print $_
/var/run/dmesg.boot

Instead of “^-s” you would use “r -- -s=” (the two dashes are needed as the “r” built-in alias parses its arguments).


More on Planet Debian (read via Planet Symlink): how many times do I have to tell you it’s “CAs” not “CA’s” again? Please do the world a favour and read Apostrophen und andere Katastrophen with rules for German and English: never in German except the word ends with s or similar: „Jens’, Max’ und Joes CDs“ and for genitives only in both languages, but with apostrophe in English: “Jens’, Max’ and Joe’s CDs”

ciruZ now has a blog too… with two ruby scripts. I prefer mine in mksh very much, thank you :þ

An mksh-current cygwin snapshot I just built for smultron, our graphical artist friend from MidnightBSD (which, by the way, also packages mksh(1)). He just did this:

mksh Logo

We’ll see what we do with that “m”…

svn2cvs.sh is an mksh(1) shell script able to convert from Subversion repositories (remote access via URLs, needs installed svn client) to RCS/CVS repositories (local access via pathnames, needs installed rcs(1) ci(1) co(1) GNU RCS, as well as cvs(1) GNU CVS, for rcsfile(5) handling). For more information, see the included help as well as the commit message.

If you have any improvements or requests regarding this script, please contact me or the miros-discuss@ mailing list. There is a discussion thread for it.

In case you didn’t, read cvs(GNU) a.k.a. The Cederqvist! Another source of CVS tricks is courtesy of «ThunderChicken:#cvs» and includes some from myself.

The 13.rcs(PSD) documentation and rcsintro(1) are online as well!

mksh R36b

14.12.2008 by tg@
Tags: mksh

mksh R36b is out, grab it while it’s still hot ☺ There’s an impressive change list for a mere bug-fix version, and this is the first one where ahoka@ has contributed directly. Upgrade is recommended for all users. The new memory allocator has been backed out for stability.

MirOS-current has mksh R36b plus some changes from mksh-current, mostly renaming set -o utf8-hack to set -o utf8-mode denoting said technology is solid after ages of testing. MirOS-stable has R36b.

galloc and OPTU

22.11.2008 by tg@
Tags: unicode mksh

aalloc was a failure and prone to mis-type; I’ve started on galloc, which is supposed to be even more type-safe and flexible, yet still segfaults on me as well.

In the meanwhile, to not stay bored, I enhanced the Unicode (MirOS OPTU-8/OPTU-16) function suite in MirBSD and hacked an implementation of Plan 9’s Rune functions – dubbed p9¾_*.c due to usage of our “internationalisation like Plan 9, just on the next layer, and within the confines of UNIX®” approach to internationalisation. These (as well as some well-known ones like wcsrtombs(3)) should be OPTU safe, and we now have a macro telling us if a wide character is part of the OPTU Raw Octet codepoint range (in the CSUR PUA assignment).

I’ve contacted Bruno Haible again because his libutf8 misses the Unicode Title case (complementing Upper and Lower case)… I’ve got an idea how to implement it but would like to double-check with him to ensure nothing breaks. The Plan 9 functions need it (these two are currently implemented as stubs that just throw ENOCOFFEE).

The number of manual pages also raised…

I built XFree86® with a (slightly beefed up) Reiser CCCP in use as sole C Præprocessor now… and have yet to notice failures in operation. Now we’ll have to find out what else stuff is there depending on the existence of /usr/bin/cpp, possibly switching it, like the <sys.mk> mirmake(1) file, to ${CC} -E - (which is not 100% compatible, because cpp can also read from stdin without the dash, whereas, with the dash, neither can read from only a file). X11 etc. can be switched to /usr/libexec/cpp then.

aalloc and COFF

17.11.2008 by tg@
Tags: mksh

I got (E)COFF executables to work, even though I cannot produce them except with a hand-crafted binary yet. Maybe these produced by kencc work now.

I didn’t get the time or nerve to continue hacking on aalloc today… the whole pointers thing in C is so useless. If I could use assembly, it were so trivial. Anyway, the TPtr data type will probably have to get lost, and I plan on using more temporary local variables for overview.

Besides a beer discussion, IRC channel today featured plans on getting a temporary setup of fast enough machines for crossbuilding OpenJDK; replaced confirmed that someone had success in doing what I planned, although the nōn-free JDK 1.5 only. We’ll see…

mksh R36 released!

25.10.2008 by tg@
Tags: mksh

The MirBSD Korn Shell R36 has been released yesterday; the Changelog contains more details as usual.

Major changes:

  • Digital UNIX (DEC OSF/1) V2.0 (MIPS) is now supported using the DEC C and ucode toolchain. (We could not run the testsuite, though.)
  • The ‘-Q’ option to Build.sh is gone without replacement.
  • A new option can be enabled by adding -DMKSH_BINSHREDUCED to $CPPFLAGS, intended mostly for Debian. This option adds extra code to determine whether mksh(1) is called as sh or -sh and switch to “set -o posix” mode if so, disabling brace expansion and some security features, because legacy scripts, such as debconf, depend on such reduced functionality /bin/shs.
  • Input lines beginning with an IFS character of IFS white space are no longer pushed into the history. Some other, less surprising, changes were also made to both Emacs and Vi command line editing modes.
  • The arc4random.c contributed file is now stored with the RCS version number appended, and has been updated.

This is a major update with improved stability and compatibility, and as such recommended in general. It has undergone more extensive testing than mksh R35; if there still be bugs, please report to us. Note that some sorts of behaviour are not considered bugs but expected, sometimes due to historical, sometimes compatibility reasons.

Request for Assistance: We are still looking for an OpenBSD committer to import mksh into the ports tree. Furthermore, a Slackware GNU/Linux package is still missing, as well as a PC-BSD PBI, and Macports (former Darwinports) do not carry mksh either. My earlier Minix 3 problem persists, as do the Plan 9 and Syllable Desktop ones.

mksh porting

21.10.2008 by tg@
Tags: mksh

If you read the comp.os.minix newsgroup, please have a look at Message-Id <Pine.BSM.4.64L.0810202029070.28869@herc.mirbsd.org> and help us porting mksh to Minix 3. (Yes I know I replied to a different thread, but this is just because Pine did not want to let me post using a simple ‘C’ompose in the gabeln.)

If you have an idea how I get the OSF/1 V2.0 cc(1) to correctly fail if it cannot resolve functions, or how we can fix Plan 9 or even Syllable Desktop (which is said to have fixed their signal handling in the kernel, however an actual user would have to test it), also feel free to contact me, you can probably help.

Hi everyone, mksh’s current development version should be tested on as many platforms as possible before a new release. This mostly means:

 $ CVS_RSH=ssh; export CVS_RSH
 $ cvs -d :ext:_anoncvs@anoncvs.mirbsd.org:/cvs co -PA mksh
 $ cd mksh
 $ (sh Build.sh -r && ./test.sh -v) 2>&1 | tee build.log

Afterwards, please mail me (tg@) the build.log file, play around a little with mksh(1), see if it works. You can use a relative path to the source directory for repeated builds and also pass different compilers or flags as shown below, but please remember that passing CFLAGS or LIBS overrides the defaults, which you thusly should include:

  • CC=gcc ksh93 ../mksh/Build.sh -r && ./test.sh -v
  • CFLAGS="-q64 -O3 -qstrict" bash ../mksh/Build.sh -r && ./test.sh -v
  • CFLAGS="-xO2 -m64" /usr/xpg4/bin/sh Build.sh -r && ./test.sh -v

You are, of course, invited to join the mailing list discussions regarding features, bugs, and behaviour, or even send in patches for missing items or architectures. I’d especially be happy if someone made Minix 3 work… pdksh runs there, mksh didn’t last time I tried (with gcc, but once that’s up ACK should be easy). Or Plan 9.

If you’re on Ohloh, please add mksh to one of your stacks.

Six Years of MirBSD

29.08.2008 by tg@
Tags: release mksh

Six years ago, I started collecting patches against OpenBSD, a bunch of 4.4 elitist snobs since the 1980s (or so they say). Said patchkit should eventually become MirOS BSD and the MirPorts Framework, along with a couple of more or less well-known and successful subprojects. Time to celebrate (TGIF)!

IOW, I hacked a Debian package of mirmake, depending on latest mksh of course. It needs more upstream work to be useful and pretty, though.

Let me plug here an animated GIF found somewhere in the net, which shows my opinion about Netscape/Mozilla/Fire…tapir quite well:
firesomething

And, even more unrelated, MirOS is (of course) continuously improving. We now use Fedora Core 4 libraries for the linuxulator, by suggestion of the MidnightBSD people, which should improve some things (it already has opened the door for Opera-linux plugins such as AcroRead-linux).

(The best browser of all is still lynx(1) though.)

mksh – The iShell

Try this:

	$ cvs -d _anoncvs@anoncvs.mirbsd.org:/cvs co -PA mksh
	$ cd mksh
	$ cat >iBuild.sh <<-'EOF'
		DEV=/Developer/Platforms/iPhoneOS.platform/Developer
		SDK=${DEV}/SDKs/iPhoneOS2.0.sdk
		CC="${DEV}/usr/bin/gcc-4.0 -arch arm"
		CPPFLAGS="-I${SDK}/usr/include"
		CPPFLAGS="$CPPFLAGS -I${DEV}/usr/lib/gcc/arm-apple-darwin9/4.0.1/include"
		CPPFLAGS="$CPPFLAGS -F${SDK}/System/library/Frameworks"
		CPPFLAGS="$CPPFLAGS -F${SDK}/System/library/PrivateFrameworks"
		CPPFLAGS="$CPPFLAGS -F/System/library/Frameworks"
		LDFLAGS="-L${SDK}/usr/lib -L${DEV}/usr/lib/gcc/arm-apple-darwin9/4.0.1/lib"

		export CC CPPFLAGS LDFLAGS
		exec "$@"
	EOF
	$ mksh iBuild.sh mksh Build.sh -r
	$ file mksh

Mach-O executable arm

Of course, you need the iPhone 2.0 SDK on your Mac for this (and, presumably, an installed native mksh… otherwise just change the above instructions a little.

We haven't tested it yet, but if gecko2@ can figure out how to run mobileterminal.app inside the emulator, he will… and possibly jailbreak his iPhone G3.

Maybe this fact (availability of mksh) strengthens our point.

In the meanwhile, gecko2@ has found the geocaches I placed.

Update 18.07.2008 12:42 – we have a fat binary with five architectures (amd64 arm i386 powerpc ppc64) now. This is the iShell, indeed.

The MirBSD Korn Shell R35 has just been released; as per the Changelog this is a major update with some bugfixes, a lot of new features, and licence simplification (the advertising clause is gone). This version was not tested on AIX, BSD/OS, Interix, IRIX, GNU/kFreeBSD, UWIN, the Intel compiler, but we expect no regressions on these platforms either. New supported platforms include dietlibc, LLVM. Platforms already working continue to be MirOS BSD, DragonFly BSD, FreeBSD, MidnightBSD, NetBSD, OpenBSD, DEC ULTRIX, Mac OSX, HP Tru64, HP-UX, Solaris, Debian GNU/HURD, Cygwin, and various GNU/Linux systems; using gcc, pcc, SUNWcc, llvm-gcc, Compaq C, HP aCC, TenDRA; on a variety of hardware architectures.

Online manual: HTML, PDF (ISO A4 paper, we don’t support Imperial units, as even the USA has converted to Metric)

Update 18.07.2008 – mksh R35b is out, with major bug fixes, read the changelog.

Back home again

20.04.2008 by tg@
Tags: hardware mksh

Back home… there just ain’t such place as [::1]… (that’s localhost for all of you who don’t use BSD). Swinging on the bike and going to the ice dealer, the best of them all. It was kind of nice in Switzerland and it’ll be a hard decision for me whether I’m going to move there or not. But after arriving at home, past the bike tour, I fell into sleep pretty soon. Travelling may be interesting, but it sure is tiring.

Too bad I couldn’t find the two geocaches I looked for on the way back.

I hacked some mksh on the train, until I had no power left in the batteries… the laptop literally just went off all suddenly… and continued that until now. We have some quite interesting new features in now, only sad point is that we still can’t hexdump NUL.

I should definitively get my new server (tear) running now, for which the only dependency left ought to be the updated vnd(4) crypto stuff. This will take a while, as I’ll design a new on-disc format as well for improved security (think of keys, IVs, and so on). After that’s done I’ll give y’all a snapshot of MirOS-current, and update a lot of ports.

Maybe I should work on bringing a regular sparc boot floppy into the tree as well – last time, I had to hand-craft one. But it will be lacking.

There’s so much interesting stuff to do. Working on the Zaurus, ALIX, my SPARcstations (still no big monitor yet, so I couldn’t test Miod’s patch to make tvtwo(4/sparc) work yet), more FreeWRT devices… but I can’t neglect my dayjobs either. And I ought to learn to read and fix Perl *sigh*

This sucks: I have network (internet) outages since last night. Sometimes, ppp(8) + pppoe(8) still work when pppoe(4) doesn’t, but most of these times, both are unusable. The rest of the time, I sometimes have huge lags. My ISP (which unfortunately is a telco, but they aren’t completely clueless either) wanted to upgrade me from a 4 Mbit/s connection to a 6 Mbit/s connection as the old product doesn’t even exist any more (and I’ll save 10 €/month now), and the cable length (230m) isn’t an issue either. Testing today (as per the salesman I should have it May 1st, per the acknowledgement mail April 15th) I’ve got about 6 Mbit/s down, but my upstream speed is even reduced! WTF?

Argh! Later on this night, my network connection is so flakey…

mksh, mirmake, pkgtools upgraded

12.04.2008 by tg@
Tags: mksh mirmake

The MirBSD Korn Shell R33d has been released, mostly containing a security fix for the -T option, see the Changelog for details.

A new MirMake version has been issued, to fix problems compiling C++ code with strange file extensions (.cpp as on Microsoft® Windows®, and .C as BOINC uses). MirMake is a sort-of-portable version of the MirOS make(1) utility, a few header files and a libmirmake.a containing helper functions, and some supporting utilities: install(1) lndir(1) lorder(1) mkdep(1) readlink(1) tsort(1)

The MirPorts Package Tools have been updated as well, as Lucas “laffer1” Holt, the MidnightBSD founder, has implemented ldconfig(8) -U too, and the use of this option by pkg_create(1) is now required for a lot of packages (since GNU libiconv, expat and libpng use their own fake-pkgview subdirectories.

Hashes of our current distfiles:

  • RMD160 (gzsigkey.cer) = 0a24e416b26e4753eee4cbfe8f1dc74f2486eb6e gzsig(1) distribution sign key
  • RMD160 (fwcf-1.05b.tar.gz) = 355e1a46ef384b4ca26cccb7892a2f60eff2ffe2 for FreeWRT
  • RMD160 (libnointl-20070726.tar.gz) = 3b97197a9bd8f693bebd8b528dd0d5c942e83e64
  • RMD160 (arc4random.c) = ad0ef189b7ffe80b9224b1f3886377847341bac5
  • RMD160 (joe-3.1jupp8.cpio.gz) = bbcc42ff82fd7a47feb51579864a487f31ca3a91
  • RMD160 (cksum-20080314.cgz) = 7566313c6db3182a99437eb6543b0da6abbed5bd
  • RMD160 (paxmirabilis-20080314.cpio.gz) = 3966925c2d87838b0b55e2a0c715f027df3d965b
  • RMD160 (ed-20060924.tar.gz) = 7d41e76e169caa73c8bbd043ad40b2b84339761a
  • RMD160 (mirmake-20080411.cpio.gz) = e18bd36bc6c260332ab5df008f30b57a9849bc50
  • RMD160 (mksh-R33d.cpio.gz) = d810a705b01e146f9addd184c168798a22a36d38
  • RMD160 (mpczar-20061119.cgz) = cccf88ae27225c808f327ae7d56dbd9f2c053249
  • RMD160 (mirmtree-20050912.cpio.gz) = b4de12ddfd022538732b2c14dad60c2351b3011c
  • RMD160 (mirnroff-20050912.cpio.gz) = 5063466a267a64b3f088c86a29e011b5ad712c4a for Interix ONLY
  • RMD160 (sitetmac-20071023.tar.gz) = 107e5b64ecbf1214e19681f6fc0add6599fef25d temporary
  • RMD160 (tmacmbsd-20071003.tar.gz) = 223c38916e489469703627b189f6fc16b9a36e27 temporary
  • RMD160 (patch-20051110.cpio.gz) = ac32b2b0a1e7203256ec4bd255c7525053afb3ed for Interix
  • RMD160 (pkgtools-20080411.cgz) = edf320a3798aa40c015e57c579310df754f28820
  • RMD160 (rdate-20070817.tar.gz) = 3501ae05c05d7c4f39ccb1c76dd51663c2b91e34
  • RMD160 (mirsort-20050912.cpio.gz) = 69a1a632c7343c09ca7bab39f6359667bb461c1c

mksh R33b, no, R33c released

28.03.2008 by tg@
Tags: mksh

The MirBSD Korn Shell R33b has just been released; as per the Changelog this is a minor update: portability fixes, support for dæmonising scripts, and manual page improvements.

Online manual: HTML, PDF (ISO A4 paper, we don’t support Imperial units, as even the USA has converted to Metric)

UPDATE 02.04.2008: mksh R33c is out, with a bugfix and a new fully supported OS (ULTRIX 4.5).

mksh R33 released

01.03.2008
Tags: mksh

mksh R33 has been released; read the ChangeLog for details. (Happy Birthday to everyone who has on this day.)

The news in short and human-readable:

  • Improved build system, once again; better error tracking; better portability with regards to tools in /usr/bin at build and compilers
  • Integrate all changes from OpenBSD ksh
  • Integrate some changes from Debian: the “dot” command needs an argument; prevent namespace conflict between (built-in) aliases and (POSIX, not Korn) shell functions
  • Add \uXXXX and \xXX escapes to the print builtin
  • No longer check if we are called as “sh” for special handling
  • Implement here strings: you can now replace print -r -- "$foo" | command with command <<<"$foo" – like ksh93, zsh; GNU bash behaves differently if and only if $foo is not quoted
  • Implement string replacement ${var/pattern/string}

Especially cool feature: replace x=$(print -r -- "$x" | sed -e 's/foo.*bar/baz/g') with x=${x//foo*bar/baz}

Note: the distfile is not here yet, it’ll take a while to be released. Once it’s uploaded, the mksh homepage will be updated with appropriate information. UPDATE: It’s out now.

All 1 2 3 4 5 6 7 8

MirOS Logo