mksh tag cloud

Sponsored by
HostEurope Logo

mksh tag cloud

All 1 2 3 4 5 6 7 8

The GNU Project is famous for its coding style – Linus Torvalds even suggests to print them out, but not read them; burning them is a great symbolic gesture. Legibility issues aside (Linus’ own are interestingly similar to style(9) aka KNF, the BSD coding styles), why is that so?

mksh-current (R40+) now supports pathnames in arbitrary lengths on Debian GNU/Hurd (I think; I only could test on gnubber.bddebian.org that it compiles, seeing that all existing installations set sysconf(_PC_PATH_MAX) to 1024 to be consistent with POSIX PATH_MAX) by using some glibc-only functions. This is because Hurd does not have PATH_MAX (some older systems also don’t have it, but there, we just define it to POSIX 1024 and good is). Now, why? Simple: because the GNU coding style says to have “no arbitrary limits” in your code.

I would like to call that ridiculous, but it’s actually dangerous: if you are on a 32-bit machine and have a pathname of 512 MiB, you’re in danger of freezing your system or at least crashing mksh, even if you have the full 4 GiB worth of RAM, due to the amount of copying and carrying around pathnames. This is a security relevant issue, in my eyes.
Now on to the ridicule part: This is Open Source, people! Change the limit (as it’s a – only one – constant in a header file) and recompile everything! Simple as that! The BSDs do it all the time! In GNU, it’s even simpler because you force developers, redistributors and some users to actually give up freedom and require them to put the source code alongside. Now, why doesn’t anybody see this? I can’t be the only one, can I? (I actually think that changeable limits would suffice the horrid GNU coding style, but find myself reluctant to read it again due to its sheer size – similar to their licences…)

On an unrelated note, I hope to have a bi-arch ISO format snapshot on BitTorrent by end of the month. Mika is also trying to put a new Grml release out by then, of course with an up-to-date MirOS bsd4grml

New MirMake (mandatory update)

20.10.2009 by tg@
Tags: bug mksh

Due to a bug mksh fixed after inheriting it from pdksh via OpenBSD ksh (oksh), which probably got it from AT&T ksh88 (ksh93 exhibits the correct behaviour, as does posh), coupled with the unfortunate lines

	CC=${CC:Q} ${MKDEP_SH} -a ${MKDEP} ${CFLAGS:M-[ID]*} \
	    ${CPPFLAGS} $$files; \

in <bsd.dep.mk, where ${MKDEP} can be the mkdep(1) option -p as well as additional CPPFLAGS like -I... (which I actually found in our tree), you absolutely must upgrade your MirMake package, as well as mkdep(1) in the base system, before upgrading to mksh-20091015 or newer. (Note that R40, which will carry the breaking fix, has not been released yet, but FreeWRT uses an mksh-current snapshot bearing it with still major 39 enacted.) It is actually pretty hard to work around, see the mkdep source code for details. There are basically two things to take care of:

  • For each x in getopts "...x..." c, make sure you not only case (x) ... ;; but also either case (+x) ... ;; or have a (*) ... ;; default trailing block, because mksh(1) getopts will also catch foo -x +y -z sanely. This is, in contrast to oksh, not disablable with a shell option.
  • If you case (\?) cmds ;; (either explicitly or via a default block), special rules apply: if you do anything other than exiting from there (e.g. via usage), $OPTIND will differ: newer shells count this option, olders don't.

The new distfile {RMD160 (/MirOS/dist/mir/make/mirmake-20091020.cpio.gz) = b9ac1258bc66b3d0d63537cc82d02c91408d1ba8} has been uploaded for your convenience already and will be integrated (after testing) into both The MirPorts Framework and FreeWRT as soon as we get to it, probably tomorrow.

mksh’s Build.sh can now generate Makefrag.inc snippets that reflect its environment and can be used like Rebuild.sh or integrated in, for example, the Android.mk files.

mksh(1) also is a great compiler testsuite: it regularily(!) points out bugs in gcc’s -fwhole-program --combine and helped to find (and fix) bugs in huge things like SunStudio, old things made modern like pcc, as well as one-man projects such as nwcc
I wish compiler authors would just use mksh as testsuite regularily.

I released a new version of jupp for Unix® today: joe-3.1jupp12 (including a backported hex editing mode, as a late birthday present for waga (from IRC) who asked for it, a plethora of bug fixes, raw octet support in UTF-8 mode, UTF-8 support in the 8-bit modes, and more changes. If you already use it, update please. If not, give it a try! It’s included in at least FreeWRT already, and I build Debian packages (link on the jupp website) in my “play repository”, as well as packages for Univention Corporate Server at work (from these packages). I’m also using it on Solaris 8, Solaris 10 and AIX 5.3 ☻ (and many more)

There are currently quite a lot more things I would like to do but can’t seem to find the time for. I had originally planned a new MirBSD snapshot, including ISO, for mid→end of September – now, end of October seems more likely (if I can get a few days off work and some RAM for the SPARCstation 5). Several things, like the mksh prompt wlog entry and webpages for a few subprojects (BSD::arc4random; arc4random.c; the RANDEX protocol, plugins, implementations, proxies; kwalletcli; keystash) are lacking too, and the Grml project is also expecting code from me. Sorry. I am, after all, human too…

As sent to the mailing list I had a chance to compile mksh for Android today, thanks to Waldemar confirming that it works (with OpenADK's eglibc linked in statically) and CeKay's help in mastering the SDK and NDK. The posting contains all relevant information; it is virtually impossible to write an Android.mk file though.

On the other hand, I did submit a patch for Bionic (the libc) to have sys_signame[] like real BSDs, so that we could simply hard-code the appropriate CFLAGS and CPPFLAGS and let the NDK compile the mksh source *.c files (signames.inc would no longer be needed).

Maybe we'll submit mksh for inclusion somewhere, as this would be the first really usable shell. For this reason, I have uploaded a binary (gunzip(1) first) at ports/mksh-39.9.20090929-android15.tgz (MD5 64ee103453d65e947f2beb1aeb6450d2) which you can place in, for instance, /data/mksh then put a (modified, as the ls(1) and id(1) and possibly other utilities are more than reduced) dot.mkshrc from the source (CVSweb) as /data/.mkshrc and run it from adb (which, by the way, is the one responsible for some cursor keys etc. not working, as Android and mksh(1) do both fine in that regard) with # HOME=/data /data/mksh in the shell. I may post an Android specific patch for dot.mkshrc some day.

Update 10.10. – .gz → .tgz and Cygwin binary now too!

smultron, MidnightBSD developer and project member, and graphician / art master for the MidnightBSD Korn Shell (among other things), has pointed me to a blog article: 8 Useful and Interesting Bash Prompts and asked for an mksh(1) translation. These will not be minimalistic (I can do quite some things), but focus on containing no control characters or other things not easily cut-and-past'able. Made more readable, too.

Small note: all mksh examples assume the following "setup code" in front of the PS1=... line. Yes, PS1 is set twice (makes the code more readable and perform better). Most of this is from the mksh(1) manpage or dot.mkshrc and relatively portable, which is why some things seem more complicated than needed. They do, however, use some rather recent mksh features; if there's real interest in making it work with, say, Debian etch mksh R28, eMail me. Note: that → character is a literal Tab (the "[→·]" Tab-Space occurs often).

	ca=$(print -n '\001'); ce=$(print -n '\033'); cm=$(print -n '\r')
	: ${HOSTNAME:=$(ulimit -c 0; hostname -s 2>&-)}
	[[ $HOSTNAME = @(localhost|*([→·])) ]] && \
	    HOSTNAME=$(ulimit -c 0; hostname 2>&-)
	: ${USER:=$(ulimit -c 0; id -un 2>&- || print \?)}
	if (( USER_ID )); then PS1='$'; else PS1='#'; fi

	function twd {
		typeset d=${PWD:-?} p=~

		[[ $p = ?(*/) ]] || d=${d/#$p/~}
		print -nr -- "$d"
	}

If you do not need exactly the effect of "\w" (tilde unexpansion), use $PWD ipv $(twd) below. Note that $USER and $HOSTNAME are usually evaluated only once; escape them if you need it evaluated every time.

  1. Show Happy face upon successful execution
    	bash$ PS1="\`if [ \$? = 0 ]; then
    		echo \[\e[33m\]^_^\[\e[0m\]
    	else
    		echo \[\e[31m\]O_O\[\e[0m\]
    	fi\`[\u@\h:\w]\$ "
    
    	mksh$ PS1="$ca$cm"'$(if (( $? )); then
    		print "'"$ca$ce[31m$caO_O$ca$ce[0m$ca"'"
    	else
    		print "'"$ca$ce[33m$ca^_^$ca$ce[0m$ca"'"
    	fi)'"[$USER@$HOSTNAME:\$(twd)]$PS1 "
    

    I do think Unicode makes things more fun though:
    	mksh$ PS1="$ca$cm$ca$ce["'$(if (( $? )); then
    		print "'"31m$ca☹"'"
    	else
    		print "'"33m$ca☺"'"
    	fi)'"$ca$ce[0m$ca[$USER@$HOSTNAME:\$(twd)]$PS1 "
    
  2. Change color on bad command
    Coming soon... colour changes are okay, but I wonder why anyone would want the history number. I've got to discuss this a little, and think about ways to use such information with mksh, and if I really want to transcribe the prompts 1:1 (it's possible by just incrementing a counter, but, hey, you don't want that). This is actually a csh feature, and who'd really use csh? I'm on my 80486DLC notebook right now, so my debugging methods are a little limited, but I promise a larger article later. I just needed to get started.

This article will be updated in-place.

mksh feature weeks

07.09.2009 by tg@
Tags: mksh

It's feature weeks for mksh(1). You almost certainly want to follow the mksh-current changelog. Please test all the new features, as well as make sure that nothing breaks nor I introduced other regressions, such as making resource usage worse (disc, CPU, memory). Please be reminded that many more deep-reaching changes will follow in the next few hacking days.

Thanks to David Korn (the Korn in AT&T Korn Shell, ksh88/ksh93) for discussing features and helping with finding syntax and semantic of some of the more complicated language pecularities.

On an unrelated note, cvs(1), lynx(1), ncurses(3), some other things, and a lot of ports have also been worked on. The next snapshot is planned for mid to end of September, bi-arch. Stay tuned!

I took the weekend to release a couple of things… MirBSD™ base code stuff and base releases (printf.c for mksh in Debian; arc4random.c for Win32 and other non-BSD OSes, now with a HKCU key used in addition to the HKLM seed key which may not be writable; MirMake; MirPorts Package Tools; mirdate – rdate(8)), jupp, mksh including a new PDF manpage, KWalletCLI, and the RANDEX plugin for XChat (Win32, BSD and *nix). I also prepared for the inclusion of more Debian source packages in my CVS “home subtree” and creation of SRPMs for more software (not in CVS though). I cleaned up the mess that were the X11 dist sets in base, cleaned up compiler warnings on half a dozen or more platforms in several parts of the code, fixed bugs in a lot of subprojects, integrated things better, updated the BSD::arc4random MirPort as well as TinyIRC, MirSirc, the irssi and XChat RANDEX plugins to include better version reporting and, for XChat, seedfile support and better responsibility. Now all I need to do is build more binaries and ports (DEB RPM OpenBSD FreeWRT etc.) of the subprojects, update Lynx in base and ports (there is a new major release out, even), update MirGRML, fix the HDD in my sparc, compile stuff, … you see I’m busy.

Here’s a “checksum and link collection” for today’s finest:

  • Simple CVS file drops
  • RMD160 (printf.c.1.10) = 8e8b88401a04474db973be07540a79b129919ff5
  • TIGER (printf.c.1.10) = 3cec4bc24074e88c7889143d19f7659ced17482115ea5afb
  • 3098389975 10575 /MirOS/dist/hosted/other/printf.c.1.10
  • MD5 (printf.c.1.10) = d09ae97aebac104f834d3d3ddd1702ca
  • RMD160 (arc4random.c.1.16) = b0caa3509d2cade6d86cb2c13e6b8817ced2d9a9
  • TIGER (arc4random.c.1.16) = ef6d7a281d451e28434b0e003990eebb47edd0cd4d899fd1
  • 2199066621 12558 /MirOS/dist/hosted/other/arc4random.c.1.16
  • MD5 (arc4random.c.1.16) = e8376a9b51c0ce08f5ed20722b05cad3
  • Simple subproject checkouts
  • RMD160 (mirmake-20090801.cpio.gz) = 79e0d15aab4c7a05690e66769c12dbeb3d99daa1
  • TIGER (mirmake-20090801.cpio.gz) = 2c6642b9515f38e736386945e72c06f402134ebf898613de
  • 788720631 372063 /MirOS/dist/mir/make/mirmake-20090801.cpio.gz
  • MD5 (mirmake-20090801.cpio.gz) = 47c63503210054d86db80040474f1f71
  • RMD160 (pkgtools-20090801.mcz) = 482dcf4b915a10bb6b76859f0c1755b67d6343bb
  • TIGER (pkgtools-20090801.mcz) = 3a622ac3c895c4af9df719dd30cfd3fe45e6d719cc34db5e
  • 2864495035 180188 /MirOS/dist/mir/pkgtools/pkgtools-20090801.mcz
  • MD5 (pkgtools-20090801.mcz) = 87378c95bde1c219d4a09e6bb8ccb897
  • RMD160 (rdate-20090802.tar.gz) = abac9ae8a08ac566d6c0396d39cd5d2cd724f7b0
  • TIGER (rdate-20090802.tar.gz) = f38a164e9d77412203349f79e8033c413335dd6f43a5cbf5
  • 3840714105 11987 /MirOS/dist/mir/rdate/rdate-20090802.tar.gz
  • MD5 (rdate-20090802.tar.gz) = a8fa4550b5a77cff6db1ed0a9d8aa357
  • JUPP (including Win32 binary)
  • PDF manpage
  • HTML manpage
  • RMD160 (joe-3.1jupp11.cpio.gz) = 7ade55cb8511600b3a9d77f37bc581b2d09ab2aa
  • TIGER (joe-3.1jupp11.cpio.gz) = b7bb4aa464b705e697ab2a52ad75fc8755a5817bfb83e09a
  • 805235529 419484 /MirOS/dist/jupp/joe-3.1jupp11.cpio.gz
  • MD5 (joe-3.1jupp11.cpio.gz) = 1e2f21a6fdebe678b125e96806267f33
  • RMD160 (JWIN31B.EXE) = f9eb9f6b3bd2a1bb5874e36d2dcc6dbdaabf75cc
  • TIGER (JWIN31B.EXE) = 771461b752114978ed64f67c01e3ef22a9a9cdf76fda6b11
  • 674256238 948176 /MirOS/dist/jupp/JWIN31B.EXE
  • MD5 (JWIN31B.EXE) = b2d3f1044221fdea76f15621e94e1ae4
  • mksh (including Cygwin package)
  • PDF manpage
  • RMD160 (mksh-R39.cpio.gz) = 5a5bcbe288e722f9772e27d2fdc36eee174bbb7b
  • TIGER (mksh-R39.cpio.gz) = 2a2c08ccf5e27365aa652663629789ade93b3d30c0d1d51f
  • 4103085544 278476 /MirOS/dist/mir/mksh/mksh-R39.cpio.gz
  • MD5 (mksh-R39.cpio.gz) = b2eeb4fe4ccac2704e1440e53cd2672c
  • RMD160 (mksh-39.1-cygwin.tgz) = 0cecd4ffb72f2d51a5c935da58e67350fab10e81
  • TIGER (mksh-39.1-cygwin.tgz) = 3157abadc40696bcb8df1d3574df571b728bef3d4d2ac2f2
  • 2818578374 144625 mksh R39 for Cygwin
  • MD5 (mksh-39.1-cygwin.tgz) = ca949841e39721be666e6a82803e7769
  • KWalletCLI
  • RMD160 (kwalletcli-1.00.tar.gz) = f04ebd39e9714212a915b6d7d4524c8cc2daaee7
  • TIGER (kwalletcli-1.00.tar.gz) = 0fc673c0c813608f0f0d863dfd924a6d62a8507c7bdf361b
  • 2355082724 11524 /MirOS/dist/hosted/kwalletcli/kwalletcli-1.00.tar.gz
  • MD5 (kwalletcli-1.00.tar.gz) = 76ef3c1d611a11ea13dc805d67d82208
  • RANDEX plugin for XChar (including Win32 binary)
  • RMD160 (xchat-randex-1.10.tar.gz) = fd61babbf4e5189f69dae8eb664ee2780433bf4b
  • TIGER (xchat-randex-1.10.tar.gz) = 6bd888b157fcd931e54b71e9778950cbfa675ae6b784ddd5
  • 2651117045 8702 /MirOS/dist/hosted/xchat-randex/xchat-randex-1.10.tar.gz
  • MD5 (xchat-randex-1.10.tar.gz) = d1585c5fae3ee531deeffc8314910553
  • RMD160 (randex.dll.gz.1.10) = a4aaa67cfdad1f9a1bcdc3eea797aff3a30703c4
  • TIGER (randex.dll.gz.1.10) = 55b2dcd7d790d28944d7424121cf5c6d4d386a99751fb556
  • 972086546 23998 /MirOS/dist/hosted/xchat-randex/randex.dll.gz.1.10
  • MD5 (randex.dll.gz.1.10) = 793ce548256efc6a23f7a37dde9215a2

An observation… mksh “print $RANDOM” on Cygwin is very slow, and the HKCU seed changes each time. This should be debugged, it shan’t unless RANDOM is being written to or 400k calls are done.

mksh has just been adjusted to behave as future POSIX will demand, after a lengthy discussion (on the bug-bash and miros-discuss mailing lists as well as the Debian bug tracker), for “set -u” (-o nounset). This, as well as the “stop () {…}” fix, must be tested extensively.

Therefore I urge all of you to

 % CVS_RSH=ssh; export CVS_RSH
 % cvs -d :ext:_anoncvs@anoncvs.mirbsd.org:/cvs co -PA mksh
 % cd mksh
 % (sh Build.sh -r && ./test.sh -v) 2>&1 | tee log

and possibly send me the log. See an earlier post for more information about testing mksh(1) snapshots, as well.

Tests for better standards compliance and bugs, especially in corner cases, are also welcome. Known are: all but the first command of a pipeline are run in subprocesses (made to be an mksh feature, not a bug); the lexer is not recursive (issues with parenthesēs and comments in COMSUBs); some IFS/whitespace issues. Fixes for these bugs, and maybe for the regression tests (they may or may not be correct) are desirable… as well as development related communication.

On an unrelated side note, aptituz told me that Debian etch already had Debhelper v5, and as such, the mksh package has been converted over from Debhelper v4 (yay, RCS IDs in debhelper configuration files! but what about changelog (no-no) and menu? dunno…).

mksh(1)'s "set -u" handling will change RSN to match what POSIX will mandate in the next version, matching similar changes in GNU bash 4 and AT&T ksh93.

Most of the thread can be seen on the miros-discuss@ mailing list archives (although both MARC and GMane seem to not have all related eMails... weird).

RFH: mksh development

07.07.2009 by tg@
Tags: mksh

mksh development is mostly done by a single person, "the mksh team" (as seen on Debian derivate from Canonical that cannot be named forums, out of all places!), a.k.a. me, myself and I. Sometimes, actual users report bugs or even send in patches. I keep track of oksh's development as well, of course. But there are times I would like to get feedback on issues from other people working on pdksh or its descendants. I mailed, for that specific issue in question, the Debian developer who created the original patch which addressed the scenario except for a corner case (interestingly, as the world is small, discovered in a Debian(!) init script from a package maintained by aforementioned formorer, on a DomU running Lenny - don't underestimate the effect of synergy) but would really like to talk to some of the OpenBSD devs about it; they mostly know what they're doing, even if I worked on ksh for longer than most of them, many eyes do help, and most of the time I do not know what I'm doing XD

[Update] There's also the issue of inter-(POSIX-compatible)-shell discussion. For instance, "set -u" vs "$@", which has come up in Debian #522255 because GNU bash4 decided to switch to the behaviour used by the Bourne shell (from V7 to SVR4.2), all Korn shells, ash and its descendents (like posh) except dash, to not treat it specially. (Funny too how they suggest 「${@:-}」 or 「${@:+}」 instead of 「${1+"$@"}」 (from the GNU(!) autoconf texinfo documentation) as replacements.) Oh well, zsh also behaves like bash2/3 and dash, but then, it's not even a POSIX compatible shell. *sigh* Now I wonder what AT&T ksh93 will do and a confirmation if it's indeed being forcibly changed by POSIX (IMHO, they could at least "agree to disagree", like they usually do, and make it vendor defined, so that scripts could not depend on it - "set -u" is something I don't use anyway).

So if you're interested in the further development of MirBSD, The MirOS Project, one of its subprojects, such as The MirPorts Framework, mksh(1), MirMake, even jupp-2.8 or jupp-3.x, please talk to me.

[Update] Do the same for POSIX compatible shell discussion, if you are going to take mksh, its goals, needs and history seriously. (Yes, it also has bugs, like a non-recursive parser troubling COMSUBs, but they may be fixed long-term, especially if people contribute. Ideas, at the very least.)

Thanks in advance.

mksh R38c released

10.06.2009 by tg@
Tags: mksh

The MirBSD Korn Shell R38c has been released. This is one of the everyone-should-upgrade versions because of the fixes for crashes and the likes. Read the online manual page in HTML – mksh(1) – or as PDF for printing.

mksh R38 released

31.05.2009 by tg@
Tags: bug mksh

The MirBSD Korn Shell R38b has been released. It adds portability to QNX 6.4, a built-in base64 decoder and encoder written in mksh itself, and most importantly fixes a regression introduced in R38 causing memory corruption.

This – and a lot more bugs – were discovered while porting Git (resp. running its test suite) for Michael Gebetsroither (grml).

All 1 2 3 4 5 6 7 8

MirOS Logo