mksh tag cloud

Sponsored by
HostEurope Logo

mksh tag cloud

All 1 2 3 4 5 6 7 8

carstenh asked in IRC how to make a shebang for mksh(1) scripts that works on both regular Unix and Android.

This is not as easy as it looks, though. Most Unicēs will have mksh installed, either manually or by means of the native package system, as /bin/mksh. Some put it into package manager-specific directories; I saw /sw/bin/mksh, /usr/local/bin/mksh and /usr/pkg/bin/mksh so far. Some systems have it as /usr/bin/mksh but these are usually those who got poettering’d and have /bin a symlink anyway. Most of these systems also have env(1) as /usr/bin/env.

Android, on the contrary, ships with precisely one shell. This has been mksh for a while, thankfully. There is, however, neither a /bin nor a /usr directory. mksh usually lives as /system/bin/mksh, with /system/bin/sh a symlink(7) to the former location. Some broken Android versions ship the binary in the latter location instead and do not ship anything that matches mksh on the $PATH, but I hope they merge my AOSP patch to revert this bad change (especially as some third-party Android toolkits overwrite /system/bin/sh with busybox sh or GNU bash and you’d lose mksh in the progress). However, on all official Android systems, mksh is the system shell. This will be important later.

The obvious and correct fix is, of course, to chmod -x the scripts and call them explicitly as mksh scriptname. This is not always possible or desirable; sometimes, people will wish it to be in the $PATH and executable, so we need a different solution.

There’s a neat trick with shebangs – the absence of one is handled specifically by most systems in various ways. I remember reading about it, but don’t remember where; I can’t find it on Sven Mascheck’s excellent pages… but: the C shell variants run a script with the Bourne Shell if its first line is a sole colon (‘:’), the Bourne family shells run it with themselves or ${EXECSHELL:-/bin/sh} in those cases, and the kernel with the system shell, AFAIK. So we have a way to get most things that could call the script to interpret it as Bourne/POSIX shell script on most systems. Then we just have to add a Bourne shell scriptlet that switches to mksh iff the current shell isn’t it (lksh, or something totally different). On Android, there is only ever one shell (or the toolkit installer better preserve mksh as mksh), so this doesn’t do anything (I hope – but did not test – that the kernel invokes the system shell correctly despite it not lying under /bin/sh) nor does it need to.

This leaves us with the following “shebang”:

	:
	case ${KSH_VERSION-} in
	*MIRBSD\ KSH*) ;;
	*)	# re-run with The MirBSD Korn Shell, this is an mksh-specific script
		test "${ZSH_VERSION+set}" = set && alias -g '${1+"$@"}'='"$@"'
		exec mksh "$0" ${1+"$@"}
		echo >&2 E: mksh re-exec failed, should not happen
		exit 127 ;;
	esac
 

The case argument not only does not need to, but actually should not be quoted; the expansion is a set -u guard; the entire scriptlet is set -e safe as well; comments and expansions are safe. exec shall not return, but if it does (GNU bash violates POSIX that way, for example), we use POSIX’ appropriate errorlevel. zsh is funny with the Bourne shell’s way of using "$@" properly. But this should really be portable. The snippet is both too short and too obvious (“only way to do it”) to be protected by copyright law.

Thanks to carstenh and Ypnose for discussing things like this with us in IRC, sending in bugfixes (and changes we decline, with reason), etc. – it feels like we have a real community, not just consuments ☺

mksh R50f coming soon

11.04.2015 by tg@
Tags: mksh pcli

Please test mksh-current from CVS (or the inofficial git mirror)! There are security-related fixes I’ll MFC in a few days, for which I’d prefer for them (and the other changes) to not introduce any regressions. Thanks!

exciting news, or so

07.04.2015 by tg@
Tags: debian event fun geocache mksh news personal pkgsrc plan rant security work

I implemented <? support (including <?php…) script embedding support for *.inc in MirWebseite today; the specific syntax was explicitely requested by Natureshadow. Ugh.

My own hacking activities are progressing, even if slowly. I do some other interesting, funny, social, beneficial, etc. stuff in between, though. I’ll even have to get some of my DD buddies to sponsor me some QA uploads of packages I formerly maintained, whereever changes are queued up… such as better old-format repo compatibility in cvs(GNU) ☺ Though some of the stuff I do at work is currently done only there… sorry.

Also: prepare to be fully enlightened about just what evil (nice picture) Docker is. I especially liked the comparison of containers to a herd of cattle, mere numbers, replaceable, whereas VMs are cats, each with their individual name, lovely petted each day, etc.

ObHint: Some may have noticed I do have a Twitter account now. I do not really use it much. I got it because I wanted to rant at someone who only gave Twitter as means to contact them (a European company running a lottery for USA citizens only). But I found one nice thing: @HourlyCats (though @FacesPics and @BahnAnsagen are funny too, and the Postillon anyway). The internet is there for cat content, anyway.
Ahem. Do not contact me there, use IRC, more specifically, the Freenode network, and possibly memoserv to mirabilos instead, I can’t fit things into 140 chars, that’s just ridiculous. Also, don’t follow me. It may contain rants, it’s NSFW, and I’m not censoring there. As I said: I do not use it. So should you. (But kudos for having a mostly functional “fallback” site (the “mobile” one), which even works in PocketIE (Windows Mobile) and Opera 9, though not so much lynx(1)…)

odc (from #!/bin/mksh on IRC) is hacking support to use mksh instead of GNU bash for bootstrapping pkgsrc® (e.g. on Solaris). Nice! Good luck!

… à propos mksh(1), dear Debian armel and armhf buildd maintainer colleagues, pretty please with strawberries and chocolate ice on top (I just had that on waffles at my favourite ice salon, so I may be biased), do like s390x and update your chroots and wanna-build give-back mksh, as we requested, so the privacy fix makes it into jessie. Thanks in advance!

Oh, and Y_Plentyn and I both have been putting more and updated packages into my APT repository. XTaran held a talk at CLT 2015 mentioning it… maybe I should write up some docs about how to use it for which purposes (e.g. how to avoid systemd but not get the other packages from it, or how to use it with systemd (trivial but has to be stated, it’s freedom of choice after all), etc.)?

Besides decent fanfiction (the stories in the Uzumaki Naruto universe seem, on average, to be much longer than those in the Harry Potter one), the weather is becoming good, so I’ve already been enjoying going out for some geocaching and will have the bike fixed at the shop RSN (it suffers a bit each winter, as it stands outside, since our basement is mouldy, which is worse than a bit of rust IMHO) to get more activity in. Also planning to head to the GPS Maze in Mainz and, besides what time FrOSCon (including preparation) allows, heading to DebConf for a while.

mirabilos’ Waypoints

… to my shame I must admit I fucked up, and we still do not have support in libssl for SHA2-signed X.509 certificates. Also, StartSSL fucked up, so currently https for www.mirbsd.org is toast.

Also more on the rant side, services offered by web-based platforms, be they web (e.g. Groundspeak’s GC.COM) or not (Googlemail, which $orkplace switched to against my express veto some time ago) are getting worse and worse over time. I had hoped they realise that and improve, especially when seeing small signs (such as GC.COM pages shrinking to 20% of the formerly served bloat) but… no.

Bernhard’s article on Plänet Debian about the “colon” command in the shell could use a clarification and a security-relevant correcture.

There is, indeed, no difference between the : and true built-in commands.

Stéphane Chazelas points out that writing : ${VARNAME:=default} is bad, : "${VARNAME:=default}" is correct. Reason: someone could preset $VARNAME with, for example, /*/*/*/*/../../../../*/*/*/*/../../../../*/*/*/* which will exhaust during globbing.

Besides that, the article is good. Thanks Bernhard for posting it!

PS: I sometimes use the colon as comment leader in the last line of a script or function, because it, unlike the octothorpe, sets $? to 0, which can be useful.

Update: As jilles pointed out in IRC, “colon” (‘:’) is a POSIX special built-in (most importantly, it keeps assignments), whereas “true” is a regular built-in utility.

mksh R50d released

07.10.2014 by tg@
Tags: bug debian mksh news pcli

The last MirBSD Korn Shell update broke update-initramfs because I accidentally introduced a regression in field splitting while fixing other bugs – sorry!

mksh R50d was just released to fix that, and a small NULL pointer dereference found by Goodbox on IRC. Thanks to my employer tarent for a bit of time to work on it.

mksh R50c released, security fix

03.10.2014 by tg@
Tags: android bug debian mksh news pcli release security

The MirBSD Korn Shell has got a new security and maintenance release.

This release fixes one mksh(1)-specific issue when importing values from the environment. The issue has been detected by the main developer during careful code review, looking at whether the shell is affected by the recent “shellshock” bugs in GNU bash, many of which also affect AT&T ksh93. (The answer is: no, none of these bugs affects mksh.) Stephane Chanzelas kindly provided me with an in-depth look at how this can be exploited. The issue has not got a CVE identifier because it was identified as low-risk. The problem here is that the environment import filter mistakenly accepted variables named “FOO+” (for any FOO), which are, by general environ(7) syntax, distinct from “FOO”, and treated them as appending to the value of “FOO”. An attacker who already had access to the environment could so append values to parameters passed through programs (including sudo(8) or setuid) to shell scripts, including indirectly, after those programs intended to sanitise the environment, e.g. invalidating the last $PATH component. It could also be used to circumvent sudo’s environment filter which protected against the vulnerability of an unpatched GNU bash being exploited.

tl;dr: mksh not affected by any shellshock bugs, but we found a bug of our own, with low impact, which does not affect any other shell, during careful code review. Please do update to mksh R50c quickly.

mksh R50b released

03.09.2014 by tg@
Tags: mksh news pcli

The MirBSD Korn Shell has got a new bugfix release. Thought you’d want to know ☺

mksh R50, jupp 27 released

29.06.2014 by tg@
Tags: jupp mksh news pcli

Both the MirBSD Korn Shell and jupp – the editor which sucks less have seen new releases today. Please test them, report all bugs, and otherwise enjoy all the bugfixes.

Other subprojects will also have new releases… once I get around doing so after hacking them…

Update 03.07.2014: New release for MirCPIO, that is, cpio(1) and pax(1) and tar(1) in a somewhat portable package.

-r--r--r-- 4 tg miros-cvssrc 141973 Jul 3 19:56 /MirOS/dist/mir/cpio/paxmirabilis-20140703.cpio.gz

mksh build problem workaround

26.01.2014 by tg@
Tags: mksh

As a workaround to the build problem of mksh R49 with some host shells, you can try removing every sequence of backslash + newline in rlimits.opt and sh_flags.opt, for example with the following sequence:

	cd /path/to/mksh
	for f in *.opt; do
		tr '\n' '`' <"$f" | \
		    sed 's/\\`//g' | \
		    tr '`' '\n' >"$f.out"
		mv "$f.out" "$f"
	done
 

This will apply to every upcoming mksh(1) release until such time as this code has been rewritten in (host) C. Another thing that could be done is to add -r to both IFS= read line occurences in Build.sh, function do_genopt.

FOSDEM preparations… done.

20.01.2014 by tg@
Tags: debian event fun grml mksh twitxr work

I’ve produced several pin-on buttons to take with me to FOSDEM for giving away (as long as there are any left):

Several pin-on buttons I made

First row (nice projects), from left to right: MidnightBSD; Glenda, the Plan 9 bunny; Teckids e.V.

Second row (The MirOS Project): mksh; the Shilouette Dæmon; the “Triforce” (Live+Install CDs for i386 and sparc, with MirGrml); “the m” (alternative logo, vector)

Third row (things originating from tarent): Freedroidz (now a Teckids project); OSIAM (Identity and Access Management); tarent (tarent AG, tarent GmbH), who sponsored production of these buttons

Hm… jupp needs a button’able logo!


FOSDEM meetup

mksh-current version "R48 2013/11/17" (only this version, no others) suffer from a now-known regression which can make your MirBSD system unbootable. Please immediately either revert to an earlier version (such as a stable release) or update to "R48 2013/11/29" from AnonCVS.

This reverts the fix for “x="X 1 2"; showargs ${x#X}”, which nevertheless is a genuine bug. Contributions fixing it without introducing any such regressions welcome…

FrOSCon is approaching, and all MirBSD developers will attend… but why’s there no MirBSD exhibit? The answer to that is a bit complex. First let’s state that of course we will participate in the event as well as the Open Source world. We’ll also be geocaching around the campus with other interested (mostly OSS) people (including those we won for this sport) and helping out other OSS projects we’ve become attached to.

MirOS BSD, the operating system, is a niche system. The conference on the other hand got “younger” and more mainstream. This means that almost all conference visitors do not belong to the target group of MirOS BSD which somewhat is an “ancient solution”: the most classical BSD around (NetBSD® loses because they have rc.d and PAM and lack sendmail(8), sorry guys, your attempt at being not reformable doesn’t count) and running on restricted hardware (such as my 486SLC with 12 MiB RAM) and exots (SPARCstation). It’s viable even as developer workstation (if your hardware is supported… otherwise just virtualise it) but its strength lies with SPARC support and “embedded x86”. And being run as virtual machine: we’re reportedly more stable and more performant than OpenBSD. MirBSD is not cut off from modern development and occasionally takes a questionable but justified choice (such as using 16-bit Unicode internally) or a weird-looking but beneficial one (such as OPTU encoding saving us locale(1) hassles) or even acts as technological pioneer (64-bit time_t on ILP32 platforms) or, at least, is faster than OpenBSD (newer GNU toolchain, things like that), but usually more conservatively, and yes, this is by design, not by lack of manpower, most of the time.

The MirPorts Framework, while technically superiour in enough places, is something that just cannot happen without manpower. I (tg@) am still using it exclusively, continuing to update ports I use and occasionally creating new ones (mupdf is in the works!), but it’s not something I’d recommend someone (other than an Mac OSX user) to use on a nōn-MirBSD system (Interix is not exactly thriving either, and the Interix support was only begun; other OSes are not widely tested).

The MirBSD Korn Shell is probably the one thing I will be remembered for. But I have absolutely no idea how one would present it on a booth at such an exhibition. A talk is much more likely. So no on that front too.

jupp, the editor which sucks less, is probably something that does deserve mainstream interest (especially considering Natureshadow is using it while teaching computing to kids) but probably more in a workshop setting. And booth space is precious enough in the FH so I think that’d be unfair.

All the other subprojects and side projects Benny and I have, such as mirₘᵢₙcⒺ, josef stalin, FreeWRT, Lunix Ewe, Shellsnippets, the fonts, etc. are interesting but share few, if any, common ground. Again, this does not match the vast majority of visitors. While we probably should push a number of these more, but a booth isn’t “it” here, either.

MirOS Linux (“MirLinux”) and MirOS Windows are, despite otherwise-saying rumours called W*k*p*d*a, only premature ideas that will not really be worked on (though MirLinux concepts are found in mirₘᵢₙcⒺ and stalin).

As you can see, despite all developers having full-time dayjobs, The MirOS Project is far from being obsolete. We hope that our website visitors understand our reasons to not have an exhibition booth of our own (even if the SPARCstation makes for a way cool one, it’s too heavy to lift all the time), and would like to point out that there are several other booths (commercial ones, as well as OSS ones such as AllBSD, Debian and (talking to) others) and other itineries we participate in. This year both Benny and I have been roped into helping out the conference itself, too (not exactly unvoluntarily though).

The best way to talk to us is IRC during regular European “geek” hours (i.e. until way too late into the night – which Americans should benefit from), semi-synchronously, or mailing lists. We sort of expect you to not be afraid to RTFM and look up acronyms you don’t understand; The MirOS Project is not unfriendly but definitely not suited for your proverbial Aunt Tilly, newbies, “desktop” users, and people who aren’t at least somewhat capable of using written English (this is by design).

All 1 2 3 4 5 6 7 8

MirOS Logo