mksh tag cloud

Sponsored by
HostEurope Logo

mksh tag cloud

All 1 2 3 4 5 6 7 8

This is both a release announcement for the next installment of The MirBSD Korn Shell, mksh R40b, and a follow-up to Sune’s article about small tools of various degrees of usefulness.

I hope I don’t need to say too much about the first part; mksh(1) is packaged in a gazillion of operating environments (dear Planet readers, that of course includes Debian, which occasionally gets a development snapshot; I’ll wait uploading R40c until that two month fixed gcc bug will finally find its way into the packages for armel and armhf). Ah, we’re getting Arch Linux (after years) to include mksh now. (Probably because they couldn’t stand the teasing that Arch Hurd included it one day after having been told about its existence, wondering why it built without needing patches on Hurd…) MSYS is a supposedly supported target now, people are working on WinAPI and DJGPP in their spare time, and Cygwin and Debian packagers have deprecated pdksh in favour of mksh (thanks!). So, everything looking well on that front.

I’ve started a collection of shell snippets some time ago, where most of “those small things” of mine ends up. Even stuff I write at work – we’re an Open Source company and can generally publish under (currently) AGPLv3 or (if extending existing code) that code’s licence. I chose git as SCM in that FusionForge instance so that people would hopefully use it and contribute to it without fear, as it’s hosted on my current money source’s servers. (Can just clone it.) Feel free to register and ask for membership, to extend it (only if your shell-fu is up to the task, KNOPPIX-style scripts would be a bad style(9) example as the primary goal of the project is to give good examples to people who learn shell coding by looking at other peoples’ code).

Maybe you like my editor, too? At OpenRheinRuhr, the Atari people sure liked it as it uses WordStar® like key combinations, standardised across a lot of platforms and vendors (DR DOS Editor, Turbo Pascal, Borland C++ for Windows, …)

ObPromise: a posting to raise the level of ferrophility on the Planet aggregators this wlog reaches (got pix)

mksh R40b (nowadays with filled in user’s caveats (for R40, too!) and packager’s upgrade hints) has just been released. This is a should-have upgrade, fixing a number of – admittedly some obscure – bugs, changing things begun in R40, improving upon others. Thanks to the PLD Linux guys for spotting all these errors; thanks to them and phpnet.org both for adopting mksh so well.

I have also fixed a bug in nroff(1) which will lead to an even nicer looking HTML manpage mksh(1) (after the next rebuild and upload of a MirBSD snapshot – scheduled RSN).

jupp 3.1.16 took on the task of merging Debian joe changes (aiming at an upload). I also split the jupprc file into three versions (2.8 generic/DOS, 3.1+jupp and 3.7/Unix) because of the differences in the baseline executables making rc files partially mutually incompatible (think Insert key), annoyingly warning (think syntax, hmsg), or less usable (joe’s new menu system).

jupp 2.8.2 is a companion to jupp 3.1.16 – mostly because of the new help window “character map” ☺

Binaries for jupp should be updated RSN too.

Considering Banja Luka is arriving quickly, the “r” in RSN should be taken with a few grains of salt. I’ve also scheduled working on the pcc Debian package for the next future; updating lynx and maybe others like OpenSSH in MirBSD is also due; cvs(1) will receive more of my time, but before the next Upload I’d like to fix LP#12230 once verified.

Builds for Debian/m68k are also still running. I note I did in fact not manage to make a new base image, yet (but 2.6.39 kernels miss a patch, anyway, so waiting for 3.0 is ok). It’s still using gcc-4.4 because nobody tests gcc-4.6 and gcj-4.6 FTBFS due to SIGSEGV, but that’s ok in my books. rsyslog is broken but sysklogd works.

The #ksh|Freenode page finally got a well-deserved link to Planet Commandline. Throw more my way!

Acronyms and translations, too. (Got Norwegian and Rumanian covered in the meantime. No idea whether any RTL languages will work in that beast. But I’m young and need the money)

Since I’m writing a wlog entry anyway… let me thank Gunnar for a nice summary on the current Free Culture discussion; my comments on Nina’s site seem to be eaten, but let me support it fully, although, of course, I normally use a copycenter style licence, which is specifically written for general works of authorship under copyright law, not limited to software. I did in fact have that in mind. Maybe some people will like it (it’s less than one Kibibyte long) either generally or just for their everyday random musings (they can then keep CC-BY-SA for the “big works” if they so desire).

Wouter, grass background makes green headlines illegible. I’ve never liked, and never installed manually, cups either. (Benny tells me that Apple’s new version refuses to talk with a non-Apple cups, kinda defeating the whole idea I think.) Port 9100 is JetDirect (probably with an HP in front and some subset of ©®™ trailing) and just nice. (Being able to talk ESC/P with your printer like print '\033K\x07\0\x3E\x81\x99\xA5\xA5\x81\x3E' >/dev/lpa too rocks though, IMHO. Yes, mine can, and I still can. /dev/lpa is BSD.)

Kai, thanks for your vimrc lines:

	:highlight TrailWhitespace ctermbg=red guibg=red
	:match TrailWhitespace /\s\+$\| \+\ze\t/
 

Automatic removal is harmful, though – I just fell into the trap since jupprc contains needed whitespace at EOL… but manual removal (bound to ^K] in jupp) rocks. And I like that your solution uses such strong a colour – vim users are the single most represented offender group for actually leaving the redundant whitespace at EOL there, and it should hurt their eyes. (Sadly there is some vehement disagreement preventing them from inclusion in grml-etc-core – but that’s why I re-post them here.) Ah, and jupp can of course display whitespace visibly (although it uses ‘·’/‘→’, replacing the arrow with ‘¬’ if no UTF-8, not ‘»’), accessible with ^Ov.

Steve, want to put up a checklist for sites? We can “crowdsource” the… testing… to maybe get some interesting results…

Some other people would get more comments if they were idling in IRC (Freenode) or allow comments on their blog, specifically without too high an entrance barrier – OpenID is ok, but many other things, and ECMAscript, are not; but I can’t really say that loud because our wlog is static HTML compiled from a flat plaintext data source so it doesn’t allow such either. I often forget what I wanted to add if I can’t get it out quickly enough (especially at work). Sowwy…

Me like the cat picture postings (Amayita, Tiago, ¡Gracias!).

New releases

11.07.2011 by tg@
Tags: debian mksh

You might have noticed the release of mksh R40 recently, after more than a year of development. Well, stay tuned for both R40b (with accumulated fixes) and R41 (intent to speed up array handling a lot and prepare for what we postponed to mksh R42 now – associative, multi-dimensional arrays).

You should also upgrade, if you have not yet done so, to kwalletcli 2.11.

Finally, jupp 3.1.15 was left out to the world, including Minix 3 users this time, by special request of one of these on our mailing list. In addition to the MidnightBSD mport – which has been there in like forever – and the MirPort and the FreeWRT package, in December 2011 a user submitted it to FreeBSD® ports, and Benny is going to add it to NetBSD® pkgsrc® soon, he said. (He also updated their mksh source package. Thanks!) I’ve been asked by two people, independent from each other, when I’ll upload it to Debian proper, instead of the private-repo packaging. Maybe I should indeed do that, comments?

mksh R40 released

12.06.2011 by tg@
Tags: mksh

The MirBSD Korn Shell R40 has finally been released. This is a must-upgrade, a major everything version. See the changelog for now, until caveats are filled in, due to the sheer amount of changes. The diff between the last and the new release is more than half of the last release’s size, after all – it’s been 15½ months.

The plan for the next two releases is:

  • mksh R41 will fix all bugs popping up within the next few weeks, as they usually show up shortly after a release only, when things are getting adopted. Furthermore, it removes deprecated code that emits a warning in R40: several Build.sh options, plus set ±o arc4random. Finally, I plan on speed-up for arrays by using hashtables internally.
  • mksh R42 is where we tentatively target associative arrays for, now. Multidimensional arrays should be introduced at the same time.

(First posting to Plänet Commandline! Tag: pcli)

Vutral asked in IRC how to synchronise two shells’ environment while they’re running. As you may know, POSIX systems cannot change a process’ environment vector after it has been started, only the process itself can. Well, the shell can, and we’ll use a variety of things for this.

This trick assumes you have $HISTFILE set to the same pathname in both shells (obviously, they run under the same user). It uses export -p to render the current list of exported variables, then transforms the list from newline-separated to a single big one-line export statement.
Then it transforms all remaining newlines (which will be part of a single-quoted string, since that’s mksh(1)’s export format) into the sequence '$'\n'' which means: terminate current single-quoted string, append $'\n' and open up a new single-quoted string immediately; concatenate these three.
Now, $'\n' is just a fancy way of saying newline, and part of mksh because David Korn (yes, the Korn in Korn Shell) strongly suggested to me that this functionality be included – but, as we can see here, it pays off.
Finally, the so transformed string is prepended by unset \$(export); which, when executed, will cause the shell to unset (and unexport) all currently exported variables. The shell parameters that are not exported, i.e. not in the environment, are not affected by this code (except for $x and $nl, but… whatever).
This string is then passed to read -s (plus -r and clearing IFS to enable raw mode), which means, read into the parameter $REPLY (which we conveniently don’t use – but it’s trashed too, thus) but store into history at the same time.

Ah hah! Now, the persistent history feature comes into effect! After running the below statement in the “source” shell, switch into the terminal running the “destination” shell, press Enter once on the empty line (Ctrl-U to empty it if it wasn’t), then Cursor-Up (↑) to recall… voilà, an insanely large line with the previously created string sorta expanded… and press Enter again to run it. Now your set of exported parameters is the exact same (minus if you exported IFS, nl, x or REPLY) as in the “source” shell.

I’ve added extra spaces and a linewrap below, this is really just one big line:

nl=$'\n'; x=$(export -p); x=${x//${nl}export/}; IFS= read -rs <<<"unset \\\$(export);${x//$nl/\'\$\'\\\\n\'\'}"

Of course, this makes a nice function, for your ~/.mkshrc or somesuch.

mksh-current has just gained an experimental recursive parser for command substitutions, fixing RedHat BZ#496791 and decades-old complaints about the pdksh codebase, compared to AT&T ksh93. (GNU bash could also do the example, but not some other things mksh(1) parses fine now.)

This means that things like the following work now.

	# POSIX, should “always” work
	echo $(case 1 in (1) echo yes;; (2) echo no;; esac)
	# POSIX optional, works now in mksh, works in GNU bash
	echo $(case 1 in 1) echo yes;; 2) echo no;; esac)
	# GNU bash seems to choke on comments ending with backslash
	# a comment with " ' \
	x=$(
	echo yes
	# a comment with " ' \
	)
	# No non-recursive COMSUB parser can pass all of the above
	# tests and these below at the same time (some extensions)
	echo $(typeset -i10 x=16#20; echo $x)
	echo $(typeset -Uui16 x=16#$(id -u)
	) .
	echo $(c=1; d=1
	typeset -Uui16 a=36#foo; c=2
	typeset -Uui16 b=36 #foo; d=2
	echo $a $b $c $d)
	# the ‘#’ is especially tricky, that’s why the above cases
 

Next on my TODO is the complete rewrite of the read built-in command, as well as its documentation. I think that the (reduced) goals for mksh R40 will have been met by then, except porting to LynxOS and MPE, but we’re working on it, and re-testing Syllable and Plan 9). Of course, a release implies testing on a lot of the supposedly supported platforms, so it won’t be out “immediately”. Though, associative arrays have been removed from the R40 goals, so that I can at least get a new release out. Note that Debian and OpenSuSE Buildservice users have been provided with somewhat well-tested mksh-current snapshots for a while already, and Gentoo users can use the “live ebuild”; there’s always compiling from source too…

(Free)BSD vs. Linux

03.03.2011 by tg@
Tags: debian event ill mksh rant

Warning: this is a rant against BSD (specifically FreeBSD®, but don’t let me get started on DragonFly, who think it’s wise to drop all shells except ash from the base system and rely on pkgsrc® – yay let’s compile a dozen packages just to get a shell with tab completion, not to mention boxen with no network access – for the task – although others seem to go into that direction too…; you know, there’s BSD, and then there’s FreeBSD…) – don’t like, don’t read.

If you want to change something in the BSD world, you gotta fork your own BSD – no other way around the thickheads. Ok, back then, I ran into a particularily thick one, but others tend to not be much better. Users share the thickness. If you want to change something in the GNU/Linux world, just make a package, have someone upload it, prod (or pay, Hanno got a Radler) people to do it, or just upload it yourself.

At the BSD booth at FOSDEM, despite me bringing the Windows® Mobile 6 Professional devive, strictly for Geocaching mind you, Macintosh boxen had a share of more than 50% – I didn’t manage to tip the scale. At the Debian booth, almost everyone had a “I want to buy a new laptop some day, but it just keeps on working and doesn’t break” pre-Lenovo IBM laptop. No hyping of Google either. (Last year’s CLT saw BSD people advocating pro-Schily – the guy with the broken encoding in his name – shockingly.)

Honestly, tcsh, FreeBSD® people? Sorry. While I agree that there is merit on having the same script and interactive shell, as someone has pointed out (copy-paste examples into the command line), there’s those zsh users who use mksh or GNU bash for scripting. Or just POSIX shell. And that’s with an interactive shell which can be used for scripting. On the other hand, the C shell (both csh and tcsh) cannot.

And what’s with pretending the accent gravis is non-combining, called “backtick” (such a thing does not exist); and advocating it? Sorry, if your csh/tcsh doesn’t handle the POSIX $(…) you should just drop it. (By the way, there is a convention that example command lines are prefixed with for csh and for sh (or but we write $ sudo  instead, these days). Use it. Or leave it. If you have examples that substitute another process’ output, be specific.) It’s funny to see how one person tries to defuse my arguments against csh by telling me “it’s just an interactive shell”, while the other argues that people copy-paste between them, to which that was my response. Read the thread!

And please, get your facts right. “I would prefer that the standard shell be at least Bourne-compatible.” You don’t want Bourne (“^” instead of “|” for pipes), you want POSIX. That GNU bash is called the Bourne-Again Shell in one of their usual semi-bad puns doesn’t help the global perception of such things any. Also, the root shell and /bin/sh are disjunct.

(Plus, why change the root shell, use sudo(8), plain and simple.)

ObNote: in jupp (should I package that for Debian, btw? rather upload, packages are ready…) the ‘`’ key is used as præfix for Ctrl-X (`X) or to directly enter numerical (decimal, octal, sedecimal/hexadecadic) ASCII, 8-bit or Unicode codepoints. Yay!
And even the FSF has seen the light; for a few releases already, GCC uses “'…'” instead of “`…'” for quoting in messages, even without locales. Great job there! (LC_MESSAGES=en_GB.UTF-8 usually works, too, though.)

ObDisclaimer: I have an (yes, Google…) Alert on the word “mksh”, so I know when it’s being discussed. This obviously includes certain fora. Also, I’m a shell implementer and bound to know a certain amount of details. Plus, mksh’s build script runs with pretty much any Bourne/POSIX/Z Shell which has functions and not too many bugs. I wrote it. Go figure. No lowly trolling.

FWIW, mksh(1) has the cat(1) builtin both because Android has no cat(1), and as speed hack. Almost all other shells have worse speed hacks, like a printf(1) builtin. And recently, builtins have become direct-callable, so this actually reduces the overall system footprint. (Its inclusion also provides for some other possibilities, internally.) And as two final side notes, if you haven’t seen this: determine which shell we are run under (CVS) and I still offer a prompt conversion service (send me any GNU bash or oksh $PS1 and I’ll send that to you in mksh(1) syntax – optionally with adjustments/improvements, like cwd uses only up to 1/3 of screen width).

Eh. Why does mksh built with (a patched: mkstemp(3) added) klibc work suddenly, unexpectedly?

To reproduce, I just uploaded mksh_39.3.20110218-1.dsc and you can run DEB_BUILD_OPTIONS=mksh-static=klibc,dietlibc,eglibc dpkg-buildpackage -rfakeroot to verify it, once you have mkstemp(3). (I will probably send a smaller implementation of that in, later.) I have that and the open fix and the m68k patch applied, nothing else… where did my bug go?

ObQuestion: what’s the legal (copyright/trademark) status of the Atari logo (the one in rainbow colours, with three things going up, right and left “leg” looking like an umbrella stand’s)?

FOSDEM was a blast!

13.02.2011 by tg@

I just need to work more on bilocality. While I did find two geocaches, one at the South/Noon Train Station (taalverwarringen…), one in the buurt of the University, I did manage to miss the AW building completely and utterly. Wow. Except, that Haiku guy came over to talk for a bit (nice). And I drew. An Atari logo with swirl, for that weird stuff I recently have been found doing.

More mksh-current news coming soon, stay tuned. In the meanwhile, I met bonsaikitten IRL (at FOSDEM, yes, too) who kindly made a “live ebuild”, i.e. a source package building -current.

Finally let’s say a big thank you to the person mostly manning our booth, gecko2, and to Benny for talking to people, getting That Other Packaging Thingy working, and pimping the website a bit.

mksh wins

28.01.2011 by tg@
Tags: debian mksh

The Android 3.0 (Honeycomb) Preview [German news] comes with a much more sensible /system/bin/sh than previous released versions… Android-x86 and AOSP have had it for a while, but this sees the general adoption of mksh(1) as the only POSIX/Korn Shell on the Android OS! Thanks to those who helped to make this reality.

I’ve just prepared an ISO for FOSDEM Ⅺ which we might export on BitTorrent soon. Benny has provided an upgraded desktop background image, thanks.

The next mksh release… well I’ll bite the sour apple and will release it without associative arrays but hope I get around to hack a few things (especially the read and sleep builtins) before releasing. One lession learned, don’t brag with oh-so-big plans when you haven’t got a deliverable yet. Real Life will interfere. Oh, and Murphy, of course. (mksh R41 might have the associative arrays then. But mksh-current has ${foo@#} which is hash($foo).

FOSDEM, the Free and Open Source Software Developers' European Meeting

❧ Who’s not? ☺ Same procedure as every year.

(okay, lolando prefers skiïng but…)

Anyway. A cow‐orker told me that Belgium again/still has no gouvernment, and they have been asked to grow out their beards until they do. I found “evidence” on the ’net but won’t link it here, also it’s on German… anyway. Let’s all join in. (Besides, I now have an excuse to not shave, maybe even my grandmother will accept this one…)

RT said on IRC that mksh will probably work on MSYS.

My Debian/m68k stuff is coming around nicely, but I still haven’t gotten around to do everything planned, plus I need to grow a new kernel and eglibc, after the latest uploads, and the 2.6.37 based one panics. Also I’ve got to take care to not overwork myself. (And make a MirBSD ISO for FOSDEM.) But hey, it’s been not working for some time and better now. And slow anyway ☺ yet we’re progressing. Does anyone know how to debug that a C programme only calling res_init(3) segfaults?

Benny is apparently not just working on making NetBSD® pkgsrc® available on MirOS BSD (picking up my work from 4+ years ago) but also replacing The MirPorts Framework with it. Sad, as I got a request for a gajim MirPort over a cocktail just this evening…

All 1 2 3 4 5 6 7 8

MirOS Logo