debian tag cloud

Sponsored by
HostEurope Logo

debian tag cloud

All 1 2 3 4 5 6 7 8 9 10 11 12

Updates to the last two posts

16.03.2017 by tg@
Tags: bug debian grml news pcli rant snippet tip work

Someone from the FSF’s licencing department posted an official-looking thing saying they don’t believe GitHub’s new ToS to be problematic with copyleft. Well, my lawyer (not my personal one, nor for The MirOS Project, but related to another association, informally) does agree with my reading of the new ToS, and I can point out at least a clause in the GPLv1 (I really don’t have time right now) which says contrary (but does this mean the FSF generally waives the restrictions of the GPL for anything on GitHub?). I’ll eMail GitHub Legal directly and will try to continue getting this fixed (as soon as I have enough time for it) as I’ll otherwise be forced to force GitHub to remove stuff from me (but with someone else as original author) under GPL, such as… tinyirc and e3.

My dbconfig-common Debian packaging example got a rather hefty upgrade because dbconfig-common (unlike any other DB schema framework I know of) doesn’t apply the upgrades on a fresh install (and doesn’t automatically put the upgrades into a transaction either) but only upgrades between Debian package versions (which can be funny with backports, but AFAICT that part is handled correctly). I now append the upgrades to the initial-version-as-seen-in-the-source to generate the initial-version-as-shipped-in-the-binary-package (optionally, only if it’s named .in) removing all transaction stuff from the upgrade files and wrapping the whole shit in BEGIN; and COMMIT; after merging. (This should at least not break nōn-PostgreSQL databases and… well, database-like-ish things I cannot test for obvious (SQLite is illegal, at least in Germany, but potentially worldwide, and then PostgreSQL is the only remaining Open Source database left ;) reasons.)

Update: Yes, this does mean that maintainers of databases and webservers should send me patches to make this work with not-PostgreSQL (new install/, upgrade files) and not-Apache-2.2/2.4 (new debian/*/*.conf snippets) to make this packaging example even more generally usable.

Natureshadow already forked this and made a Python/Flask package from it, so I’ll prod him to provide a similarily versatile hello-python-world example package.

Since I use this as base for other PHP packages like SimKolab, I’ve updated my packaging example with:

  • PHP 7 support (untested, as I need libapache2-mod-php5)
  • tons more utility code for you to use
  • a class autoloader, with example (build time, for now)
  • (at build time) running a PHPUnit testsuite (unless nocheck)

The old features (Apache 2.2 and 2.4 support, dbconfig-common, etc.) are, of course, still there. Support for other webservers could be contributed by you, and I could extend the autoloader to work at runtime (using dpkg triggers) to include dependencies as packaged in other Debian packages. See, nobody needs “composer”! ☻

Feel free to check it out, play around with it, install it, test it, send me improvement patches and feature requests, etc. — it’s here with a mirror at GitHub (since I wrote it myself and the licence is permissive enough anyway).

This posting and the code behind it are sponsored by my employer ⮡ tarent.

Please use the correct (perma)link to bookmark this article, not the page listing all wlog entries of the last decade. Thank you.</update>

Some updates inline and at the bottom.

The new Terms of Service of GitHub became effective today, which is quite problematic — there was a review phase, but my reviews pointing out the problems were not answered, and, while the language is somewhat changed from the draft, they became effective immediately.

Now, the new ToS are not so bad that one immediately must stop using their service for disagreement, but it’s important that certain content may no longer legally be pushed to GitHub. I’ll try to explain which is affected, and why.

I’m mostly working my way backwards through section D, as that’s where the problems I identified lie, and because this is from easier to harder.

Note that using a private repository does not help, as the same terms apply.

Anything requiring attribution (e.g. CC-BY, but also BSD, …)

Section D.7 requires the person uploading content to waive any and all attribution rights. Ostensibly “to allow basic functions like search to work”, which I can even believe, but, for a work the uploader did not create completely by themselves, they can’t grant this licence.

The CC licences are notably bad because they don’t permit sublicencing, but even so, anything requiring attribution can, in almost all cases, not “written or otherwise, created or uploaded by our Users”. This is fact, and the exceptions are few.

Anything putting conditions on the right to “use, display and perform” the work and, worse, “reproduce” (all Copyleft)

Section D.5 requires the uploader to grant all other GitHub users…

  • the right to “use, display and perform” the work (with no further restrictions attached to it) — while this (likely — I didn’t check) does not exclude the GPL, many others (I believe CC-*-SA) are affected, and…
  • the right to “reproduce your Content solely on GitHub as permitted through GitHub's functionality”, with no further restructions attached; this is a killer for, I believe, any and all licences falling into the “copyleft” category.

Note that section D.4 is similar, but granting the licence to GitHub (and their successors); while this is worded much more friendly than in the draft, this fact only makes it harder to see if it affects works in a similar way. But that doesn’t matter since D.5 is clear enough. (This doesn’t mean it’s not a problem, just that I don’t want to go there and analyse D.4 as D.5 points out the same problems but is easier.)

This means that any and all content under copyleft licences is also no longer welcome on GitHub.

Anything requiring integrity of the author’s source (e.g. LPPL)

Some licences are famous for requiring people to keep the original intact while permitting patches to be piled on top; this is actually permissible for Open Source, even though annoying, and the most common LaTeX licence is rather close to that. Section D.3 says any (partial) content can be removed — though keeping a PKZIP archive of the original is a likely workaround.

Affected licences

Anything copyleft (GPL, AGPL, LGPL, CC-*-SA) or requiring attribution (CC-BY-*, but also 4-clause BSD, Apache 2 with NOTICE text file, …) are affected. BSD-style licences without advertising clause (MIT/Expat, MirOS, etc.) are probably not affected… if GitHub doesn’t go too far and dissociates excerpts from their context and legal info, but then nobody would be able to distribute it, so that’d be useless.

But what if I just fork something under such a licence?

Only “continuing to use GitHub” constitutes accepting the new terms. This means that repositories from people who last used GitHub before March 2017 are excluded.

Even then, the new terms likely only apply to content uploaded in March 2017 or later (note that git commit dates are unreliable, you have to actually check whether the contribution dates March 2017 or later).

And then, most people are likely unaware of the new terms. If they upload content they themselves don’t have the appropriate rights (waivers to attribution and copyleft/share-alike clauses), it’s plain illegal and also makes your upload of them or a derivate thereof no more legal.

Granted, people who, in full knowledge of the new ToS, share any “User-Generated Content” with GitHub on or after 1ˢᵗ March, 2017, and actually have the appropriate rights to do that, can do that; and if you encounter such a repository, you can fork, modify and upload that iff you also waive attribution and copyleft/share-alike rights for your portion of the upload. But — especially in the beginning — these will be few and far between (even more so taking into account that GitHub is, legally spoken, a mess, and they don’t even care about hosting only OSS / Free works).

Conclusion (Fazit)

I’ll be starting to remove any such content of mine, such as the source code mirrors of jupp, which is under the GNU GPLv1, now and will be requesting people who forked such repositories on GitHub to also remove them. This is not something I like to do but something I am required to do in order to comply with the licence granted to me by my upstream. Anything you’ve found contributed by me in the meantime is up for review; ping me if I forgot something. (mksh is likely safe, even if I hereby remind you that the attribution requirement of the BSD-style licences still applies outside of GitHub.)

(Pet peeve: why can’t I “adopt a licence” with British spelling? They seem to require oversea barbarian spelling.)

The others

Atlassian Bitbucket has similar terms (even worse actually; I looked at them to see whether I could mirror mksh there, and turns out, I can’t if I don’t want to lose most of what few rights I retain when publishing under a permissive licence). Gitlab seems to not have such, but requires you to indemnify them… YMMV. I think I’ll self-host the removed content.

And now?

I’m in contact with someone from GitHub Legal (not explicitly in the official capacity though) and will try to explain the sheer magnitude of the problem and ways to solve this (leaving the technical issues to technical solutions and requiring legal solutions only where strictly necessary), but for now, the ToS are enacted (another point of my criticism of this move) and thus, the aforementioned works must go off GitHub right now.

That’s not to say they may not come back later once this all has been addressed, if it will be addressed to allow that. The new ToS do have some good; for example, the old ToS said “you allow every GitHub user to fork your repositories” without ever specifying what that means. It’s just that the people over at GitHub need to understand that, both legally and technically¹, any and all OSS licences² grant enough to run a hosting platform already³, and separate explicit grants are only needed if a repository contains content not under an OSI/OKFN/Copyfree/FSF/DFSG-free licence. I have been told that “these are important issues” and been thanked for my feedback; we’ll see what comes from this.

① maybe with a little more effort on the coders’ side³

② All licences on one of those lists or conformant to the DFSG, OSD or OKD should do⁴.

③ e.g. when displaying search results, add a note “this is an excerpt, click HERE to get to the original work in its context, with licence and attribution” where “HERE” is a backlink to the file in the repository

④ It is understood those organisations never un-approve any licence that rightfully conforms to those definitions (also in cases like a grant saying “just use any OSS² licence” which is occasionally used)

Update: In the meantime, joeyh has written not one but two insightful articles (although I disagree in some details; the new licence is only to GitHub users (D.5) and GitHub (D.4) and only within their system, so, while uploaders would violate the ToS (they cannot grant the licence) and (probably) the upstream-granted copyleft licence, this would not mean that everyone else wasn’t bound by the copyleft licence in, well, enough cases to count (yes it’s possible to construct situations in which this hurts the copyleft fraction, but no, they’re nowhere near 100%).

How to use the subtree git merge strategy

20.12.2016 by tg@
Tags: debian grml pcli tip work

This article might be perceived as a blatant ripoff of this Linux kernel document, but, on the contrary, it’s intended as add-on, showing how to do a subtree merge (the multi-project merge strategy that’s actually doable in a heterogenous group of developers, as opposed to subprojects, which many just can’t wrap their heads around) with contemporary git (“stupid content tracker”). Furthermore, the commands are reformatted to be easier to copy/paste.

To summarise: you’re on the top level of a checkout of the project into which the “other” project (Bproject) is to be merged. We wish to merge the top level of Bproject’s “master” branch as (newly created) subdirectory “dir-B” under the current project’s top level.

	$ git remote add --no-tags -f Bproject /path/to/B/.git
	$ git merge -s ours --allow-unrelated-histories --no-commit Bproject/master
	$ git read-tree -u --prefix=dir-B/ Bproject/master
	$ git commit -m 'Merge B project as our subdirectory dir-B'

	Later updates are easy:
	$ git pull -s subtree Bproject master

(mind the trailing slash after dir-B/ on the read-tree command!)

Besides reformatting, the use of --allow-unrelated-histories recently became necessary. --no-tags is also usually what you want, because tags are not namespaced like branches.

Another command you might find relevant is how to clean up orphaned remote branches:

	$ for x in $(git remote); do git remote prune "$x"; done

This command locally deletes all remote branches (those named “origin/foo”) that have been deleted on the remote side.

Update: Natureshadow wishes you to know that there is such a command as git subtree which can do similar things to the subtree merge strategy explained above, and several more related things. It does, however, need the præfix on every subsequent pull.

“I don’t like computers”

13.11.2016 by tg@
Tags: debian pcli personal rant tip

cnuke@ spotted something on the internet, and shared. Do read this, including the comments. It’s so true. (My car is 30 years old, I use computers mostly for sirc, lynx and ssh, and I especially do not buy any product that needs to be “online” to work.)

Nice parts of the internet, to offset this, though, do exist. IRC as a way of cheap (affordable), mostly reliant, communication that’s easy enough to do with TELNET.EXE if necessary. Fanfiction; easy proliferation of people’s art (literature, in this case). Fast access to documentation and source code; OpenBSD’s AnonCVS was a first, nowadays almost everything (not Tom Dickey’s projects (lynx, ncurses, xterm, cdk, …), nor GNU bash, though) is on a public version control system repository. (Now people need to learn to not rewrite history, just commit whatever shit they do, to record thought process, not produce the perfect-looking patch.) Livestreams too, I guess, but ever since went dead due to a USA law change on 2016-01-02, it got bad.

Please save GMane!

28.07.2016 by tg@
Tags: debian news pcli rant

GMane has been down for a day or two, and flakey for a day before that. MidnightBSD’s laffer1 just linked the reason, which made me cry out loud.

GMane is really great, and I rely on the NNTP interface a lot, both posting and especially reading — it gives me the ability to download messages from mailing lists I don’t receive in order to be able to compose replies with (mostly) correct References and In-Reply-To headers. Its web interface, especially the article permalinks, are also extremely helpful.

This is a request for a petition to save GMane. Please, someone, do something! Thanks in advance!

The MirBSD Korn Shell R52c was published today as bugfix-accumulating release of low upto medium importance. Thanks to everyone who helped squashing all those bugs; this includes our bug reporters who always include reproducer testcases; you’re wonderful!

MirCPIO was also resynchronised from OpenBSD, to address the CVE-2015-{1193,1194} test cases, after a downstream (wow there are so many?) reminded us of it; thanks!
This is mostly to prevent extracting ../foo – either directly or from a symlink(7) – from actually ending up being placed in the parent directory. As such the severity is medium-high. And it has a page now – initially just a landing page / stub; will be fleshed out later.

Uploads for both should make their way into Debian very soon (these are the packages mksh and pax). Uploading backports for mksh (jessie and wheezy-sloppy) have been requested by several users, but none of the four(?) DDs asked about sponsoring them even answered at all, and the regular (current) sponsors don’t have experience with bpo, so… SOL ☹

I’ve also tweaked a bug in sed(1), in MirBSD. Unfortunately, this means it now comes with the GNUism -i too: don’t use it, use ed(1) (much nicer anyway) or perlrun(1) -p/-n…

Finally, our PDF manpages now use the PA4 paper size instead of DIN ISO A4, meaning they can be printed without cropping or scaling on both A4 and US-american “letter” paper. And a Бодун from the last announcement: we now use Gentium and Inconsolata as body text and monospace fonts, respectively. (And à propos, the website ought to be more legible due to text justification and better line spacing now.) I managed to hack this up in GNU groff and Ghostscript, thankfully. (LaTeX too) Currently there are PDF manpages for joe (jupp), mksh, and cpio/pax/tar.

And we had Grünkohl today!

Also, new console-setup package in the “WTF” APT repository since upstream managed to do actual work on it (even fixed some bugs). Read its feed if interested, as its news will not be repeated here usually. (That means, subscribe as there won’t be many future reminders in this place.)

The service appears to be gone. I’ll not remove our images, but if someone knows what became of it drop us a message (IRC or mailing list will work just fine).

PS: This was originally written on 20160304 but opax refused to be merged in time… Happy Birthday, gecko2! In the meantime, the Street Food festival weekend provided wonderful food at BaseCamp, and headache prevented this from being finished on the fifth.

Update 06.03.2016: The pax changes were too intrusive, so I decided to only backport the fixes OpenBSD did (both those they mentioned and those silently included), well, the applicable parts of them, anyway, instead. There will be a MirCPIO release completely rebased later after all changes are merged and, more importantly, tested. Another release although not set for immediate future should bring a more sensible (and mksh-like) buildsystem for improved portability (and thus some more changes we had to exclude at first).

I’ve also cloned the halfwidth part of the FixedMisc [MirOS] font as FixedMiscHW for use with Qt5 applications, xfonts-base in the “WTF” APT repo. (Debian #809979)

tl;dr: mksh R52c (bugfix-only, low-medium); mircpio 20160306 (security backport; high) with future complete rebase (medium) upstream and in Debian. No mksh backports due to lacking a bpo capable sponsor. New console-setup in “WTF” APT repo, and mksh there as usual. xfonts-base too. gone?

I just published the first version of git find on gh/mirabilos/git-find for easy collaboration. The repository deliberately only contains the script and the manual page so it can easily be merged into git.git with complete history later, should they accept it. git find is MirOS licenced. It does require a recent mksh (Update: I did start it in POSIX sh first, but it eventually turned out to require arrays, and I don’t know perl(1) and am not going to rewrite it in C) and some common utility extensions to deal with NUL-separated lines (sort -z, grep -z, git ls-tree -z); also, support for '\0' in tr(1) and a comm(1) that does not choke on embedded NULs in lines.

To install or uninstall it, run…

	$ git clone
	$ cd git-find
	$ sudo ln -sf $PWD/git-find /usr/lib/git-core/
	$ sudo cp git-find.1 /usr/local/share/man/man1/
	… hack …
	$ sudo rm /usr/lib/git-core/git-find \

… then you can call it as “git find” and look at the documentation with “git help find”, as is customary.

The idea behind this utility is to have a tool like “git grep” that acts on the list of files known to git (and not e.g. ignored files) to quickly search for, say, all PNG files in the repository (but not the generated ones). “git find” acts on the index for the HEAD, i.e. whatever commit is currently checked-out (unlike “git grep” which also knows about “git add”ed files; fix welcome) and then offers a filter syntax similar to find(1) to follow up: parenthesēs, ! for negation, -a and -o for boolean are supported, as well as -name, -regex and -wholename and their case-insensitive variants, although regex uses grep(1) without (or, if the global option -E is given, with) -E, and the pattern matches use mksh(1)’s, which ignores the locale and doesn’t do [[:alpha:]] character classes yet. On the plus side, the output is guaranteed to be sorted; on the minus side, it is rather wastefully using temporary files (under $TMPDIR of course, so use of tmpfs is recommended). -print0 is the only output option (-print being the default).

Another mode “forwards” the file list to the system find; since it doesn’t support DOS-style response files, this only works if the amount of files is smaller than the operating system’s limit; this mode supports the full range (except -maxdepth) of the system find(1) filters, e.g. -mmin -1 and -ls, but it occurs filesystem access penalty for the entire tree and doesn’t sort the output, but can do -ls or even -exec.

The idea here is that it can collaboratively be improved, reviewed, fixed, etc. and then, should they agree, with the entire history, subtree-merged into git.git and shipped to the world.

Part of the development was sponsored by tarent solutions GmbH, the rest and the entire manual page were done in my vacation.

If you install the xfonts-base package from my APT repository you now not only get the FixedMisc [MirOS] type from The MirOS Project type foundry for the X Window System, but now also for GNU GRUB2:

FixedMisc [MirOS] for GNU GRUB2 – Screenshot

Just add GRUB_FONT=/usr/share/grub/FixedMisc.pf2 to /etc/default/grub, make sure gfxterm is enabled (usually by commenting out GRUB_TERMINAL=console and removing the comment sign before GRUB_GFXMODE=640x480), run sudo update-grub and be happy at the next reboot.

The combining and Katakana characters depicted in the above screenshot are the result of manual grub.cfg editing and for demonstration (bragging) purposes only.

The RSS feed of my APT repository will also contain such news…

Go enjoy shell

27.08.2015 by tg@
Tags: debian fun pcli

Dimitri, I personally enjoy shell…

tglase@tglase:~ $ x=車賈滑豈更串句龜龜契金喇車賈滑豈更串句龜龜契金喇
tglase@tglase:~ $ echo ${x::12}
tglase@tglase:~ $ printf '%s\n' 'import sys' 'print(sys.argv[1][:12])' >
tglase@tglase:~ $ python $x

… much more than Python, actually. (Python is the language in which you do not want to write code dealing with strings, due to UnicodeDecodeError and all; even py3k is not much better.)

I would have commented on your post if it allowed doing so without getting a proprietary Google+ account.

For these who similarily suffer from having to use Googlemail at work. If anyone else has more of these, please do share.

Deactivate the spamfilter

The site admins can do that. Otherwise, you will have work-relevant eMails, for example from your own OTRS system, end up in Spam (where you don’t see it, as their IMAP sucks) and deleted without asking 30 days later. (AIUI, the only way to get eMails actually deleted from Google…)

Do not use their SMTP service

Use your own outgoing MTA. This brings back the, well, not feature but should-have-been-granted-but-Google-doesn’t-do-it-anyway that, when you write to a mailing list, you also get your own messages into your own INBOX.


I have no solutions for this. I stopped using the Googlemail calendars because they didn’t think it a problem that, when I accept an invitation in Kontact (KDEPIM as packaged in Debian sid), the organiser of the calendar item in the sender’s calendar (for which I do not have write permissions) changes to me (so the actual meeting organiser cannot change anything afterwards) and/or calendar items get doubled. I now run a local uw-imapd (forward-ported to sid by means of a binNMU) for sent-mail folders etc. and a local iCalendar directory for calendars.

exciting news, or so

07.04.2015 by tg@
Tags: debian event fun geocache mksh news personal pkgsrc plan rant security work

I implemented <? support (including <?php…) script embedding support for *.inc in MirWebseite today; the specific syntax was explicitely requested by Natureshadow. Ugh.

My own hacking activities are progressing, even if slowly. I do some other interesting, funny, social, beneficial, etc. stuff in between, though. I’ll even have to get some of my DD buddies to sponsor me some QA uploads of packages I formerly maintained, whereever changes are queued up… such as better old-format repo compatibility in cvs(GNU) ☺ Though some of the stuff I do at work is currently done only there… sorry.

Also: prepare to be fully enlightened about just what evil (nice picture) Docker is. I especially liked the comparison of containers to a herd of cattle, mere numbers, replaceable, whereas VMs are cats, each with their individual name, lovely petted each day, etc.

ObHint: Some may have noticed I do have a Twitter account now. I do not really use it much. I got it because I wanted to rant at someone who only gave Twitter as means to contact them (a European company running a lottery for USA citizens only). But I found one nice thing: @HourlyCats (though @FacesPics and @BahnAnsagen are funny too, and the Postillon anyway). The internet is there for cat content, anyway.
Ahem. Do not contact me there, use IRC, more specifically, the Freenode network, and possibly memoserv to mirabilos instead, I can’t fit things into 140 chars, that’s just ridiculous. Also, don’t follow me. It may contain rants, it’s NSFW, and I’m not censoring there. As I said: I do not use it. So should you. (But kudos for having a mostly functional “fallback” site (the “mobile” one), which even works in PocketIE (Windows Mobile) and Opera 9, though not so much lynx(1)…)

odc (from #!/bin/mksh on IRC) is hacking support to use mksh instead of GNU bash for bootstrapping pkgsrc® (e.g. on Solaris). Nice! Good luck!

… à propos mksh(1), dear Debian armel and armhf buildd maintainer colleagues, pretty please with strawberries and chocolate ice on top (I just had that on waffles at my favourite ice salon, so I may be biased), do like s390x and update your chroots and wanna-build give-back mksh, as we requested, so the privacy fix makes it into jessie. Thanks in advance!

Oh, and Y_Plentyn and I both have been putting more and updated packages into my APT repository. XTaran held a talk at CLT 2015 mentioning it… maybe I should write up some docs about how to use it for which purposes (e.g. how to avoid systemd but not get the other packages from it, or how to use it with systemd (trivial but has to be stated, it’s freedom of choice after all), etc.)?

Besides decent fanfiction (the stories in the Uzumaki Naruto universe seem, on average, to be much longer than those in the Harry Potter one), the weather is becoming good, so I’ve already been enjoying going out for some geocaching and will have the bike fixed at the shop RSN (it suffers a bit each winter, as it stands outside, since our basement is mouldy, which is worse than a bit of rust IMHO) to get more activity in. Also planning to head to the GPS Maze in Mainz and, besides what time FrOSCon (including preparation) allows, heading to DebConf for a while.

mirabilos’ Waypoints

… to my shame I must admit I fucked up, and we still do not have support in libssl for SHA2-signed X.509 certificates. Also, StartSSL fucked up, so currently https for is toast.

Also more on the rant side, services offered by web-based platforms, be they web (e.g. Groundspeak’s GC.COM) or not (Googlemail, which $orkplace switched to against my express veto some time ago) are getting worse and worse over time. I had hoped they realise that and improve, especially when seeing small signs (such as GC.COM pages shrinking to 20% of the formerly served bloat) but… no.

After seeing what the Wildfly (formerly JBoss AS) and Liferay combo does to /tmp, and somewhat attempting to fix it, I saw JVM_TMP in the Debian tomcat7 init script and thought, oh no, not another one.

Is that even safe, what they do here, or is that a possibility to instantly pwn?

The net is full of literature for how to obtain temporary files and directories, but there is nothing about how to reliably obtain paths under /tmp or, more generally, directories not just writable for one single user (think the g+w thing that got FusionForge CVE-2013-1423).

The scenario here is: I am root, and I want to start something as another user, and pass it a stable path, such as /tmp/liferay. So I can just mkdir /tmp/liferay || die; chown thatuser /tmp/liferay and, in the “stop” process, rm -rf /tmp/liferay, right? (Of course not. Also, bad example, as the liferay thing can also be started as thatuser, and our devs regularily need to do that, the init script is there just for the admin convenience and reboot-safety. But I still am interested if there is a secure way to achieve this.)

The tomcat7 scenario is “trivial”: on That Other Init System™, it would just get its private /tmp declared in the .service file, and good is, no more hassle. That's one I have to give you. (No idea if this is actually shipped in jessie. Our production systems run wheezy anyway, so there is not even the slightest bit of temptation. Plus, it would not solve the liferay issue, see above. Still, a point for going into the right direction.)

The idea here is the same. It creates a directory on start and tears it down on stop. If there was nothing to do on start, the init script could just use mktemp -d. Heck, maybe it still should, but it would need to note down, and communicate to the stop instance, the actual name used. What a drag…

This is something I see popping up from time to time. I want to use stable paths for SSH session multiplexing control sockets in my ssh_config(5) file, but have them on tmpfs (Linux) or mfs (BSD) so they get properly removed on reboot. No Unix traditionally has per-user temporary directories that are clean and created after reboot. (Adjusting the paths is trivial once you have them.) Android has it worse, what with not having a world-writable tmp directory, which the shell needs e.g. for here documents; there are two components here, to have a directory the current user can write to, and to know its location. Some fail at the first, some at the second, some at both, and the classic /tmp is not the cure, as we have seen. (But if you ever see mksh erroring out due to lack of write permissions somewhere (including /sqlite_stmt_journals which used to be it) as non-root on Android, or even as root, set TMPDIR to something writable; it's tracked, so the change gets active immediately.)

TIL: the encoding of the catalina.out file is dependent on the system locale, using standard Debian wheezy tomcat7 package.

Fix for ‘?’ instead of umlauts in it:

cat >>/etc/default/tomcat7 <<EOF
export LC_CTYPE

My “problem” here is that I have the system locale be the “C” locale, to get predictable behaviour; applications that need it can set a locale by themselves. (Many don’t bother with POSIX locales and use different/separate means of determining especially encoding, but possibly also i18n/l10n. But it seems the POSIX locales are getting more and more used.)

Update: There is also adding -Dfile.encoding=UTF-8 to $JAVA_OPTS which seems to be more promising: no fiddling with locales, no breakage if someone defined LC_ALL already, and it sets precisely what it should set (the encoding) and nothing else (since the encoding does not need to correlate to any locale setting, why should it).

TIL: the init script of tomcat7 in Debian is asynchronous.

For some piece of software, our rollout (install and upgrade) process works like this:

  • service tomcat7 stop
  • rm -rf /var/lib/tomcat7/webapps/appname{,.war}
  • cp newfile.war /var/lib/tomcat7/webapps/appname.war
  • service tomcat7 start # ← here
  • service tomcat7 stop
  • edit some config files under /var/lib/tomcat7/webapps/appname/WEB-INF/
  • service tomcat7 start

The first tomcat7 start “here” is just to unzip the *.war files. For some reason, people like to let tomcat7 do that.

This failed today; there were two webapps. Manually unzipping it also did not work for some reason.

Re-doing it, inserting a sleep 30 after the “here”, made it work.

In a perfect world, initscripts only return when the service is running, so that the next one started in a nice sequential (not parallel!) init or manual start sequence can do what it needs to, assuming the previous command has fully finished.

In this perfect world, those who do wish for faster startup times use a different init system, one that starts things in parallel, for example. Even there, dependencies will wish for the depended-on service to be fully running when they are started; even more so, since the delays between starting things seem to be less for that other init system.

So, this is not about the init system, but about the init script; a change that would be a win-win for users of both init schemes.

Update: Someone already contacted me with feedback: they suggested to wait until the “shutdown port” is listened on by tomcat7. We’ll look at this later. In the meantime, we’re trying to also get rid of the “config (and logs) in webapps/” part…

PS: If someone is interested in an init script (Debian/LSB sysvinit, I made the effort to finally learn that… some months before the other system came) that starts Wildfly (formerly known as JBoss AS) synchronously, waiting until all *.?ar files are fully “deployed” before returning (though with a timeout in case it won’t ever finish), just ask (maybe it will become a dialogue, in which we can improve it together). (We have two versions of it, the more actively maintained one is in a secret internal project though, so I’d have to merge it and ready it for publication though, plus the older one is AGPLv3, the newer one was relicenced to a BSDish licence.)

A coworker and I debugged a fascinating problem today.

They had a tomcat7 installation with a couple of webapps, and one of the bundled libraries was logging in German. Everything else was logging in English (the webapps themselves, and the things the other bundled libraries did).

We searched around a bit, and eventually found that the wrongly-logging library (something jaxb/jax-ws) was using, after unravelling another few layers of “library bundling another library as convenience copy” (gah, Java!), which contains quite a few com.sun.istack.localization.Localizable members. Looking at the other classes in that package, in particular Localizer, showed that it defaults to the java.util.Locale.getDefault() value for the language.

Which is set from the environment.

Looking at /proc/pid-of-JVM-running-tomcat7/environ showed nothing, “of course”. The system locale was, properly, set to English. (We mostly use en_GB.UTF-8 for better paper sizes and the metric system (unless the person requesting the machine, or the admin creating it, still likes the system to speak German *shudder*), but that one still had en_US.UTF-8.)

Browsing the documentation for java.util.Locale proved more fruitful: it also contains a setDefault method, which sets the new “default” locale… JVM-wide.

Turns out another of the webapps used that for some sort of internal localisation. Clearly, the containment of tomcat7 is incomplete in this case.

Documenting for the larger ’net, in case someone else runs into this. It’s not as if things like this would be showing up in the USA, where the majority of development appears to happen.

OK, time to clean up ↳ tarent so people can work again tomorrow.

Not much to clean though (the participants were nice and cleaned up after themselves ☺), so it’s mostly putting stuff back to where it belongs. Oh, and drinking more of the cool Belgian beer Geert (Linux upstream) brought ☻

We were productive, reporting and fixing kernel bugs, fixing hardware, swapping and partitioning discs, upgrading software, getting buildds (mostly Amiga) back to work, trying X11 (kdrive) on a bare metal Atari Falcon (and finding a window manager that works with it), etc. – I hope someone else writes a report; for now we have a photo and a screenshot (made with trusty xwd). Watch the debian-68k mailing list archives for things to come.

I think that, issues with electric cars aside, everyone liked the food places too ;-)

As I said, I did not certain events that begun with “lea” and end with “ing” prevent me from organising a Debian/m68k hack weekend. Well, that weekend is now.

I’m too unorganised, and I spent too much time in the last few evenings to organise things so I built up a sleep deficit already ☹ and the feedback was slow. (But so are the computers.) And someone I’d have loved to come was hurt and can’t come.

On the plus side, several people I’ve long wanted to meet IRL are coming, either already today or tomorrow. I hope we all will have a lot of fun.

Legal disclaimer: “Debian/m68k” is a port of Debian™ to m68k. It used to be official, but now isn’t. It belongs to, which may run on DSA hardware, but is not acknowledged by Debian at large, unfortunately. Debian is a registered trademark owned by Software in the Public Interest, Inc.

If you’re a Unix person instead of e.g. a Microsoft® Windows® person, you’ve probably been annoyed by Iceweasel (or Mozilla™ Firefox®) creating a ~/Desktop directory, among others (things like ~/Downloads).

Here’s a quick fix I found somewhere in the ’net:

mkdir -p -m0700 ~/.config
cat >~/.config/user-dirs.dirs <<'EOF'

Upon next start, Iceweasel (and other XDG-compliant applications) will throw stuff into ~/ instead.

WTF is Jessie; PA4 paper size

12.12.2014 by tg@
Tags: debian pcli rant

My personal APT repository now has a jessie suite – currently just a clone of the sid suite, but so, people can get on the correct “upgrade channel” already.

Besides that, the usual small updates to my metapackages, bugfixes, etc. – You might have noticed that it’s now on a (hopefully permanent) location. I’ve put a donated eee-pc from my father to good use and am now running a Debian system at home. (Fun, as I’m emeritus now, officially, and haven’t had one during my time as active uploading DD.) I’ve created a couple of cowbuilder chroots (pbuilderrc to achieve that included in the repo) and can build packages, but for i386 only (amd64 is still done on the x32 desktop at work), but, more importantly, I can build, sign and publish the repo, so it may grow. (popcon data is interesting. More than double the amount of machines I have installed that stuff on.)

Update: I’ve started writing a NEWS file and cobbled together an RSS 2.0 feed from that… still plaintext content, but at least signalling in feedreaders upon updates.

Installing gimp and inkscape, I’m asked for a default paper size by libpaper1. PA4 is still not an option, I wonder why. I also haven’t managed to get MirPorts GNU groff and Artifex Ghostscript to use that paper size, so the various PDF manpages I produce are still using DIN ISO A4, rendering e.g. Mexicans unable to print them. Help welcome.

Note, for arngc, you need a server component (MirBSD-current, of course; we’re rolling release nowadays). Config included, but I’m willing to open my firewall to people I know, provided they won’t use “too much” traffic (running a couple of arngc instances is fine, according to what I estimated).

A largish article about how to use some other packages in the repo, such as dash-mksh, is yet to come. In the meantime, I wrote a bit more in README.Debian in mirabilos-support.

A surprise to see my box booting up with the default GRUB 2.x menu, followed by “cannot find a working init”.

What happened?

Well, grub:i386 and grub:x32 are distinct packages, so APT helpfully decided to purge the GRUB config. OK. Manual boot menu entry editing later, re-adding “GRUB_DISABLE_SUBMENU=y” and “GRUB_CMDLINE_LINUX="syscall.x32=y"” to /etc/default/grub, removing “quiet” again from GRUB_CMDLINE_LINUX_DEFAULT, and uncommenting “GRUB_TERMINAL=console”… and don’t forget to “sudo update-grub”. There. This should work.

On the plus side, nvidia-driver:i386 seems to work… but not with boinc-client:x32 (why, again? I swear, its GPU detection has been driving me nuts on >¾ of all systems I installed it on, already!).

On the minus side, I now have to figure out why…

tglase@tglase:~ $ sudo ifup -v tap1
Configuring interface tap1=tap1 (inet)
run-parts --exit-on-error --verbose /etc/network/if-pre-up.d
run-parts: executing /etc/network/if-pre-up.d/bridge
run-parts: executing /etc/network/if-pre-up.d/ethtool
ip addr add broadcast   peer  dev tap1 label tap1
Cannot find device "tap1"
Failed to bring up tap1.

… this happens. This used to work before the cktN kernels.

Bernhard’s article on Plänet Debian about the “colon” command in the shell could use a clarification and a security-relevant correcture.

There is, indeed, no difference between the : and true built-in commands.

Stéphane Chazelas points out that writing : ${VARNAME:=default} is bad, : "${VARNAME:=default}" is correct. Reason: someone could preset $VARNAME with, for example, /*/*/*/*/../../../../*/*/*/*/../../../../*/*/*/* which will exhaust during globbing.

Besides that, the article is good. Thanks Bernhard for posting it!

PS: I sometimes use the colon as comment leader in the last line of a script or function, because it, unlike the octothorpe, sets $? to 0, which can be useful.

Update: As jilles pointed out in IRC, “colon” (‘:’) is a POSIX special built-in (most importantly, it keeps assignments), whereas “true” is a regular built-in utility.

d-i preseeding is not the answer

25.11.2014 by tg@
Tags: debian rant work

This post details what the d-i team currently shows as the only way.

It has several shortcomings and one missing documentation part.

Shortcoming: --purge is missing from the apt-get invocation. This leaves packages in “rc” state (requiring a manual dpkg --purge to completely remove them later, as they are then invisible to apt).

Worse shortcoming: this still leaves all dependencies pulled in by systemd around on the system, because packages installed by debootstrap are not eligible for “apt-get --purge autoremove”. Additionally, it does not influence debootstrap’s (nōn-existent, see #557322, #668001, #768062) dependency resolver, leading to possibly pessimistic package selections.

Missing: you can just hit Alt-F2 and enter the command…

	in-target apt-get --purge -y install sysvinit-core

… there, no need to preseed. But this does not eliminate the aforementioned shortcomings, of course.

Another PSA: something surprising about XML.

As you might all know, XML must be valid UTF-8 (or UTF-16 (or another encoding supported by the parser, but one which yields valid Unicode codepoints when read and converted)). Some characters, such as the ampersand ‘&’, must be escaped (“&#38;” or “&#x26;”, although “&amp;” may also work, depending on the domain) or put into a CDATA section (“<![CDATA[&]]>”).

A bit surprisingly, a literal backspace character (ASCII 08h, Unicode U+0008) is not allowed in the text. I filed a bugreport against libxml2, asking it to please encode these characters.

A bit more research followed. Surprisingly, there are characters that are not valid in XML “documents” in any way, not even as entities or in CDATA sections. (xmlstarlet, by the way, errors out somewhat nicely for an unescaped literal or entity-escaped backspace, but behaves absolutely hilarious for a literal backspace in a CDATA section.) Basically, XML contains a whitelist for the following Unicode codepoints:

  • U+0009
  • U+000A
  • U+000D
  • U+0020‥U+D7FF
  • U+E000‥U+FFFD
  • U-00010000‥U-0010FFFF

Additionally, a certain number of codepoints is discouraged: U+007F‥U+0084 (IMHO wise), U+0086‥U+009F (also wise, but why allow U+0085?), U+FDD0‥U+FDEF (a bit surprisingly, but consistent with disallowing the backspace character), and the last two codepoints of every plane (U+FFFE and U+FFFF were already disallowed, but U-0001FFFE, U-0001FFFF, …, U-0010FFFF weren’t; this is extremely wise).

The suggestion seems to be to just strip these characters silently from the XML “document”.

I’m a bit miffed about this, as I don’t even use XML directly (I’m extending a PHP “webapplication” that is a SOAP client and talks to a Java™ SOAP-WS) and would expect this to preserve my strings, but, oh my. I’ve forwarded the suggestion to just strip them silently to the libxml2 maintainers in the aforementioned bug report, for now, and may even hack that myself (on customer-paid time). More robust than hacking the PHP thingy to strip them first, anyway – I’ve got no control over the XML after all.

Sharing this so that more people know that not all UTF-8 is valid in XML. Maybe it saves someone else some time. (Now wondering whether to address this in my xhtml_escape shell function. Probably should. Meh.)

Apparently (the actual results have not yet been published by the Secretary), the GR is over, and the worst possible option has won. This is an absolutely ambiguous result, while at the same time sending a clear signal that Debian is not to be trusted wrt. investing anything into it, right now.

Why is this? Simply: “GR not required” means that “whatever people do is probably right”. Besides this, we have one statement from the CTTE (“systemd is default init system for jessie. Period.”) and nothing else. This means that runit, or upstart, or file-rc, or uselessd, can be the default init system for zurg^H^H^H^Hstretch, or even the only one. It also means that the vast majority of Debian Developers are sheeple, neither clearly voting to preserve freedom of choice between init systems for its users, nor clearly voting to unambiguously support systemd and progress over compatibility and choice, nor clearly stating that systemd is important but supporting other init systems is still recommended. (I’ll not go into detail on how the proposer of the apparently winning choice recommends others to ignore ftpmaster constraints and licences, and even suggests to run a GR to soften up the DFSG interpretation.) I’d have voted this as “no, absolutely not” if it was possible to do so more strongly.

Judging from the statistics, the only thing I voted above NOTA/FD is the one least accepted by DDs, although the only other proposal I considered is the first-rated of them: support for other init systems is recommended but not required. What made me vote it below NOTA/FD was: “The Debian Project makes no statement at this time on sysvinit support beyond the jessie release.” This sentence made even this proposal unbearable, unacceptable, for people wanting to invest (time, money, etc.) into Debian.

Update: Formal result announced. So 358 out of 483 voting DDs decided to be sheeple (if I understand the eMail correctly). We had 1006 DDs with voting rights, which is a bit ashaming as well. That’s 48.01% only. I wonder what’s worse.

This opens up a very hard problem: I’m absolutely stunned by this and wondering what to do now. While there is no real alternative to Debian at $dayjob I can always create customised packages in my own APT repository, and – while it was great when those were eventually (3.1.17-1) accepted into Debian, even replacing the previous packages completely – it is simpler and quicker to not do so. While $dayjob benefits from having packages I work on inside Debian itself, even though I cannot always test all scenarios Debian users would need, some work reduction due to… reactions… already led to Debian losing out on Mediawiki for jessie and some additional suffering. With my own package repository, I can – modulo installing/debootstrap – serve my needs for $dayjob much quicker, easily, etc. and only miss out on absolutely delightful user feedback. But then, others could always package software I’m upstream of for Debian. Or, if I do not leave the project, continue doing so via QA uploads.

I’m also disappointed because I have invested quite some effort into trying to make Debian better (my idea to join as DD was “if I’ve got to use it, it better be damn good!”), into packaging software and convincing people at work that developing software as Debian packages instead of (or not) thinking of packaging later was good. I’ve converted our versions of FusionForge and d-push to Debian packages, and it works pretty damn well. Sometimes it needs backports of my own, but that’s the corportate world, and no problem to an experienced DD. (I just feel bad we ($orkplace) lost some people, an FTP master along them, before this really gained traction.)

I’d convert to OpenBSD because, despite MirBSD’s history with them, they’re the only technically sound alternative, but apparently tedu (whom I respect technically, and who used to offer good advice to even me when asked, and who I think wouldn’t choose systemd himself) still (allying with the systemd “side” (I’m not against people being able to choose systemd, for the record, I just don’t want to be forced into it myself!)) has some sort of grudge against me. Plus, it’d be hard to get customers to follow. So, no alternative right now. But I’m used to managing my own forks of software; I’m doomed to basically hack and fix anything I use (I recently got someone who owns a licence to an old-enough Visual Studio version to transfer that to me, so I can hack on the Windows Mobile 6 version of Cachebox, to fix bugs in one of the geocaching applications I use. Now I “just” need to learn C# and the .NET Compact Framework. So I’m also used to some amount of pain.)

I’m still unresolved wrt. the attitude I should show the Debian project now. I had decided to just continue to live on, and work on the things I need done, but that was before this GR non-result. I absolutely cannot recommend anyone to “invest” into Debian (without sounding hypocriet), but I cannot recommend anything else either. I cannot justify leaving but don’t know if I want to stay. I think I should sleep over it.

One thing I promised, and thus will do, is to organise a meeting of the Debian/m68k people soonish. But then, major and important and powerful forces inside Debian still insist that Debian-Ports are not part of it… [Update: yes, DSA is moving it closer, thanks for that by the way, but that doesn’t mean anything to certain maintainers or the Release Team, although, the latter is actually understandable and probably sensible.] yet, all forks of Debian now suffer from the systemd adoption in it instead of having a freedom-of-choice upstream. I’ve said, and I still feel that systemd adoption should have done in a Debian downstream / (pure?) blend, and maybe (parts of) GNOME removed from Debian itself for it. (Adding cgroups support to the m68k kernel to support systemd was done. I adviced against it, on the grounds of memory and code size. But no downstream can remove it now.)

On a closing note: an Ewok told me I should not be surprised because of my communication style on the mailing lists. I just got private mails telling me that, indeed, I’ve been more civilised recently, plus I’ve not started out as aggressively as it became in the end of the heated systemd debate (with this GR result, I precisely lost what I had feared), plus I’ve hung on Usenet for too long… and I’m sometimes terse when I don’t want to repeat the, for me, same topic once again (I’ve usually looked at the things before and decided they’re just another hype, and know from experience to avoid them). So I feel this should not be held against me. Listen to advice, please. (I’m also somewhat shocked by certain people asserting systemd is “unavoidable”, now.)

We already edit /etc/tomcat7/server.xml after installing the tomcat7 Debian package, to get it to talk AJP instead of HTTP (so we can use libapache2-mod-jk to put it behind an Apache 2 httpd, which also terminates SSL):

We already comment out the block…

    <Connector port="8080" protocol="HTTP/1.1"
               redirectPort="8443" />

… and remove the comment chars around the line…

    <Connector port="8009" protocol="AJP/1.3" redirectPort="8443" />

… so all we need to do is edit that line to make it look like…

    <Connector address="" port="8009" protocol="AJP/1.3" redirectPort="8443" />

… and we’re all set.

(Your apache2 vhost needs a line JkMount /?* ajp13_worker and everything Just Works™ with the default configuration.)

Now, tomcat7 is only accessible from localhost (Legacy IP), and we don’t need to firewall the AJP (or HTTP/8080) port. Do make sure your Apache 2 access configuration works, though ☺

I just installed, for work, Hanno Böck’s bashcheck utility on our monitoring system, and watched all¹ systems go blue.

① All but two. One is not executing remote scripts from the monitoring for security reasons, the other is my desktop which runs Debian “sid” (unstable).

(Update, 2014-10-20: jessie, precise, trusty are also green now.)

This means that all those distributions still have unfixed #shellshock bugs.

  • lenny (with Md’s packages): bash (3.2-4.2) = 3.2.53(1)-release
    • CVE-2014-6277 (lcamtuf bug #1)
  • squeeze (LTS): bash (4.1-3+deb6u2) = 4.1.5(1)-release
    • CVE-2014-6277 (lcamtuf bug #1)
  • wheezy (stable-security): bash (4.2+dfsg-0.1+deb7u3) = 4.2.37(1)-release
    • CVE-2014-6277 (lcamtuf bug #1)
    • CVE-2014-6278 (lcamtuf bug #2)
  • jessie (testing): bash (4.3-10) = 4.3.27(1)-release
    • CVE-2014-6277 (lcamtuf bug #1)
    • CVE-2014-6278 (lcamtuf bug #2)
  • sid (unstable): bash (4.3-11) = 4.3.30(1)-release
    • none
  • CentOS 5.5: bash-3.2-24.el5 = 3.2.25(1)-release
    • extra-vulnerable (function import active)
    • CVE-2014-6271 (original shellshock)
    • CVE-2014-7169 (taviso bug)
    • CVE-2014-7186 (redir_stack bug)
    • CVE-2014-6277 (lcamtuf bug #1)
  • CentOS 5.6: bash-3.2-24.el5 = 3.2.25(1)-release
    • extra-vulnerable (function import active)
    • CVE-2014-6271 (original shellshock)
    • CVE-2014-7169 (taviso bug)
    • CVE-2014-7186 (redir_stack bug)
    • CVE-2014-6277 (lcamtuf bug #1)
  • CentOS 5.8: bash-3.2-33.el5_10.4 = 3.2.25(1)-release
    • CVE-2014-6277 (lcamtuf bug #1)
  • CentOS 5.9: bash-3.2-33.el5_10.4 = 3.2.25(1)-release
    • CVE-2014-6277 (lcamtuf bug #1)
  • CentOS 5.10: bash-3.2-33.el5_10.4 = 3.2.25(1)-release
    • CVE-2014-6277 (lcamtuf bug #1)
  • CentOS 6.4: bash-4.1.2-15.el6_5.2.x86_64 = 4.1.2(1)-release
    • CVE-2014-6277 (lcamtuf bug #1)
  • CentOS 6.5: bash-4.1.2-15.el6_5.2.x86_64 = 4.1.2(1)-release
    • CVE-2014-6277 (lcamtuf bug #1)
  • lucid (10.04): bash (4.1-2ubuntu3.4) = 4.1.5(1)-release
    • CVE-2014-6277 (lcamtuf bug #1)
  • precise (12.04): bash (4.2-2ubuntu2.5) = 4.2.25(1)-release
    • CVE-2014-6277 (lcamtuf bug #1)
    • CVE-2014-6278 (lcamtuf bug #2)
  • quantal (12.10): bash (4.2-5ubuntu1) = 4.2.37(1)-release
    • extra-vulnerable (function import active)
    • CVE-2014-6271 (original shellshock)
    • CVE-2014-7169 (taviso bug)
    • CVE-2014-7186 (redir_stack bug)
    • CVE-2014-6277 (lcamtuf bug #1)
    • CVE-2014-6278 (lcamtuf bug #2)
  • trusty (14.04): bash (4.3-7ubuntu1.4) = 4.3.11(1)-release
    • CVE-2014-6277 (lcamtuf bug #1)
    • CVE-2014-6278 (lcamtuf bug #2)

I don’t know if/when all distributions will have patched their packages ☹ but thought you’d want to know the hysteria isn’t over yet…

… however, I hope you were not stupid enough to follow the advice of this site which suggests you to download some random file over the ’net and execute it with superuser permissions, unchecked. (I think the Ruby people were the first to spread this extremely insecure, stupid and reprehensible technique.)


  • rsc points out that CentOS only supports 5.«latest» and 6.«latest», and paying RHEL get 5.«x».«y» but only occasionally. We updated one of the two systems in question and shut the other down due to lack of use.
  • trusty (14.04): bash (4.3-7ubuntu1.5) = 4.3.11(1)-release
    • none
  • Yes, comments on this blog are disabled; mail for feedback.
  • Since I was asked (twice): the namespace patches by Florian Weimer protect from most exploits. The bugs are, nevertheless, present:
    root@debian-wheezy:~ # env 'BASH_FUNC_foo()=() { x() { _; }; x() { _; } <<'"$(perl -e '{print "A"x1000}'); }" bash -c :
    Segmentation fault
    139|root@debian-wheezy:~ # dmesg | tail -1
    [3121102.362274] bash[1699]: segfault at dfdfdfdf ip 00000000f766df36 sp 00000000ffe90b34 error 4 in[f75ee000+15d000]
  • To one eMail sender: If you do not understand why a CGI or something else could invoke a shell, or what a segmentation fault trap is, do not bother me. Especially not in that tone.
  • To another eMail sender: Yes, quantal is end of life. It’s also upgraded. First and last time I ever used *buntu’s “do-release-upgrade”. Broke kernel and GRUB, and upgraded to saucy. I manually “apt-get --purge dist-upgrade”d to trusty (went surprisingly well).
  • precise (12.04): bash (4.2-2ubuntu2.6) = 4.2.25(1)-release
    • none
  • jessie (testing): bash (4.3-11) = 4.3.30(1)-release
    • none
  • Update 27.10.2014: Clearly, distributions are not fixing the lcamtuf bug series (Debian stable, CentOS), believing that the affix patch makes them invulnerable, while it just removes the most common/popular/exposed attack vector. Sad story.

Thanks to ↳ tarent for letting me do this work during $dayjob time!

mksh R50d released

07.10.2014 by tg@
Tags: bug debian mksh news pcli

The last MirBSD Korn Shell update broke update-initramfs because I accidentally introduced a regression in field splitting while fixing other bugs – sorry!

mksh R50d was just released to fix that, and a small NULL pointer dereference found by Goodbox on IRC. Thanks to my employer tarent for a bit of time to work on it.

mksh R50c released, security fix

03.10.2014 by tg@
Tags: android bug debian mksh news pcli release security

The MirBSD Korn Shell has got a new security and maintenance release.

This release fixes one mksh(1)-specific issue when importing values from the environment. The issue has been detected by the main developer during careful code review, looking at whether the shell is affected by the recent “shellshock” bugs in GNU bash, many of which also affect AT&T ksh93. (The answer is: no, none of these bugs affects mksh.) Stephane Chanzelas kindly provided me with an in-depth look at how this can be exploited. The issue has not got a CVE identifier because it was identified as low-risk. The problem here is that the environment import filter mistakenly accepted variables named “FOO+” (for any FOO), which are, by general environ(7) syntax, distinct from “FOO”, and treated them as appending to the value of “FOO”. An attacker who already had access to the environment could so append values to parameters passed through programs (including sudo(8) or setuid) to shell scripts, including indirectly, after those programs intended to sanitise the environment, e.g. invalidating the last $PATH component. It could also be used to circumvent sudo’s environment filter which protected against the vulnerability of an unpatched GNU bash being exploited.

tl;dr: mksh not affected by any shellshock bugs, but we found a bug of our own, with low impact, which does not affect any other shell, during careful code review. Please do update to mksh R50c quickly.

git rebase is problematic (from a version control system user point of view) because it rewrites history. We all knew that.

But did you know that git pull --rebase, commonly used before a git push, can also be harmful, destroy history, and surprise users in a negative way?

10:08⎜<Beuc:#fusionforge> Lo-lan-do, we must have committed at the same time :)  First time I rebase and my commit disappear ;)
10:11⎜«Lo-lan-do:#fusionforge» So who won?
10:11⎜«Lo-lan-do:#fusionforge» Aha, I did :-)

This does fit git’s model of managing patches and tracking content, but is just irresponsible for a version control system. (Also, imagine incensed contributors whose commits just vanish.) So, danger, beware of using git rebasing when you use git as distributed version control system!

In a related way: merge commits are good. Especially when merging between, into or from feature branches. (A friend had his .gitconfig set up to default to rebasing… ugh.) So there should have been one place where you used rebase: to avoid merge commits when people work on the same repository at the same time (but, ideally, on different files). Those were mostly annoying. But, as you can see above, the alternatives are even worse…

Okay. I just created three events at tomorrow 10:00 CEST (08:00 UTC), on three different accounts on the very same Debian Lenny machine. All three use nxclient to log into KDE 3, with Kontact/KDEPIM Version 1.2.9 (enterprise35 20131030.a834355).

Then, I looked at the event invitations (METHOD:REQUEST BEGIN:VEVENT) in a simple eMail program (alpine). What I got made me beg to differ:





I should mention that I created the events at roughly 13:45 CEST today (11:45 UTC), and the system timezone on the box as well as the “Time & Date” zone in the Kontact settings are all “Europe/Berlin”.

To add insult to injury: the calendar view on the accounts that created the event all do show it for tomorrow, 10:00 local time.

I cannot possibly imagine how this could go wrong, seeing as those are all on the same machine…

WTF is going on here?

Update: .kde/share/config/korganizerrc had TimeZoneId wrong. I had to change a different timezone (such as Europe/Bratislava), hit the OK button, confirm to “Keep Times”, then re-open the settings dialogue and change back to Europe/Berlin, for it to work.

Dear FSF, stop recommending Enigmail.

05.06.2014 by tg@
Tags: debian pcli rant security tip work

Dear FSF, stop recommending Enigmail, please. It is broken, simple as that. Even if you switch everything HTML-related off, it still defaults to the latin9 (ISO-8859-15) encoding instead of UTF-8, and possibly some other nasties. Worse, it’s based upon obsolete Thunderbird/Icedove technology, which is dead since the release of Firefox® 17 and will only degrate over time.

Side note: I was asked recently how much entropy is used while generating a PGP key using GnuPG on Windows®, after having done the same for OpenSSL on Debian (and possibly almost all other OSes). I had to try to find out which was the actual code (GnuPG 2 with libgcrypt, it turns out), and it was not pretty. (You are hereby adviced to create a 600-byte file ${GNUPGHOME:-~/.gnupg}/random_seed from a good source before even attempting to use GnuPG 2 for the first time. OK, you can run gpg -k once, to create the GNUPGHOME directory from a skeleton.)

I’m holding a Debian packaging workshop for our trainees at work tomorrow, and have prepared a sample package for a simple PHP web application (just a handful of files) with DB connection (PostgreSQL of course), automatic setup via dbconfig-common, and with support for both Apache 2.2 (wheezy, precise) and Apache 2.4 (jessie/sid), configuration-wise. (It is possible to install this without Apache, just it does not configure the webserver then.) Schema updates on software updates are also tested (there is neither Flyway nor Liquibase – which are the tools we use at work for this, other than Roland Mas’ wonderful scripts for FusionForge – in Debian, but to my delight I discovered that dbconfig-common can also do this).

Comments, suggestions, flames, etc. welcome. I know that this should not be a native package, and will address this tomorrow, but I wanted something that serves as decent example for how to do this easily, Policy conformant and using modern techniques (even those I dislike myself – for the sake of simplicity).

Permission was granted by the business administration to reproduce this all under a BSD-style licence, so, enjoy sharing!

Thanks to Roland Mas, for making FusionForge such a nice project, and Arno Töll for some instant IRC help on the Apache side of this.

This is my first time using dbconfig-common, and now, I finally feel I know enough to finish the packaging of Kivitendo which I’ve started earlier. Beta testers for that welcome, too.

(And next week or so, I’ll need this for a Maven thingy. I’ll probably opt out on the DB side, there, though. Never did anything with that, either, not being a Java™ guy. I guess something web to go with tomcat7… anyone got this already?)

Lügen haben lange Leitern

13.05.2014 by tg@
Tags: debian fun politics rant twitxr

Photo von Laternenmast mit Wahlplakaten, oben Pro NRW, unten…

Endlich tut mal jemand was gegen die rechte Hetzpartei! – Ein Arbeitskollege fragt, ob man die nicht einfach mit einem langen Heckenschneider abmachen kann… aber sie so lächerlich zu machen hat auch was ☺

Finally, someone is doing something against this Nazi party! A coworker wondered whether it’s legal to cut them off with a long tool, but making them ridiculous like this is also funny ☻

(Explanation: the “Pro NRW” people put their campaign thingies (sorry, I don’t speak English well) up on lamp posts very high, because they are taken down by other citizens immediately otherwise, so there’s now people making fun of them for using long ladders (to put them up there, so the offended citizens need equally long ladders or tools with long arms) in leaning on the saying that lies have long legs ⇒ here: ladders.)

Maibaum für Ada

04.05.2014 by tg@
Tags: debian fun twitxr

While taking the tram to our favourite Croatian restaurant, I spotted something dedicated to Ada. We’ll never know which one… the language, the famous programmer, or someone else. A “Maibaum (may pole, one of its many meanings). Click on the picture to get a slightly different one which has the text better legible.


Stay off my computer, puppet!

18.04.2014 by tg@
Tags: bug debian fun geocache pcli rant tip work

I was out, seeing something that wasn’t there yet when I was at school (the “web” was not ubiquitous, back then), and decided to have a look:


Ugh. Oh well, PocketIE doesn’t provide a “View Source” thingy, so I asked Natureshadow (who got the same result on his Android, and had no “View Source” either apparently, so he used cURL to see it). We saw (here, re-enacted using ftp(1)):

	tg@blau:~ $ ftp -Vo -
	<!-- pageok -->
	<!-- managed by puppet -->

This is the final straw… after puppet managed to trash a sudoers(5) at work (I warned people to not introduce it) now it breaks websites. ☺

(Of course, tools are useful, but at best to the skill of their users. Merely dumbly copying recipes from “the ’net” without any understanding just makes debugging harder for those of us with skills.)

ObQuestion: Does anyone have ⓐ a transcript (into UTF-8) and ⓑ a translation for the other half of the OpenBSD 2.8 poster? (I get asked this regularily.)
Update: One person sent me the Kanji and Kana for it in UTF-8 「俺のマシンに手を出すな!」, and they and one more person told me it’s “Hands off my machine!” or “Don’t lay a hand on my machine!”. Now I’m not studying Japanese, but it LGTM in FixedMisc [MirOS], and JMdict from MirPorts says: ore no mashin ni te (w)o dasu na (roughly: my machine; particle; hands; particle; put out; prohibition) ☺ Thanks all, now I know what to tell visitors who wonder about that poster on my wall.

ObTip: I can install a few hundred Debian VMs at work manually before the effort needed to automate d-i would amortise. So I decided not to. Coworkers are shocked. I keep flexibility (can decide to have machines differ), and the boss accepts my explanations. Think before doing automation just for the sake of automation!

Heartbleed vs. Startcom / StartSSL

09.04.2014 by tg@
Tags: bug debian news rant security work

First of all, good news, MirBSD is not vulnerable to The Heartbleed Bug due to my deliberate choice to stick to an older OpenSSL version. My inquiry (in various places) as to what precisely could leak when a vulnerable client connected to a nōn-vulnerable server has yet to be answered, though we can assume private key material is safe.

Now the bad news: while the CA I use¹ and a CA I don’t use offer free rekeying (in general), a CA I also use occasionally² refuses to do that. The ugly: they will not even revoke the certificates, so any attacker who gained your key, for example when you have been using a certificate of theirs on a Debian system, will be able to use it (e.g. to MITM your visitors traffic) unless you shell over lots of unreasonable money per certificate. (Someone wrote they got the fee waived, but others don’t, nor do I. (There’s also a great Twitter discussion-thingy about this involving Zugschlus, but I won’t link Twitter because they are not accessible to Lynx users like me and other Planet Debian authors.)

① I’ve been using GoDaddy privately for a while, paid for a wildcard certificate for *, and later also at work. I’ve stopped using it privately due to current lack of money.

② Occasionally, for nōn-wildcard gratis SSL certificates for HTTP servers. Startcom’s StartSSL certificates are unusable for real SSL as used in SMTP STARTTLS anyway, so usage isn’t much.

Now I’ve got a dilemma here. I’ve created a CA myself, to use with MirBSD infrastructure and things like that – X.509 certificates for my hosts (especially so I can use them for SMTP) and possibly personal friends (whose PGP key I’ve signed with maximum trust after the usual verification) but am using a StartSSL certificate for as my GoDaddy wildcard certificate expires in a week or so (due to the aforementioned monetary issues), and I’d rather not pay for a limited certificate only supporting a single vhost. There is absolutely no issue with that certificate and key (only ever generated and used on MirBSD, only using it in Apache mod_ssl). Then, there’s this soon-to-be tax-exempt non-profit society of public utility I’m working with, whose server runs Debian, and which is affected, but has been using a StartSSL certificate for a while. Neither the society nor I can afford to pay for revocation, and we do not see any possible justification for this especially in the face of CVE-2014-0160. I expect a rekey keeping the current validity end date, and would accept a revocation even if I were unable to get a new certificate, since even were we to get a certificate for the society’s domain from someplace else, an attacker could still MITM us with the previous one from Startcom.

The problem here is: I’d really love to see (all of!) Startcom dropped from the global list of trustworthy CAs, but then I’d not know from where to get a cert for MirBSD; Globalsign is not an option because I will not limit SSL compatibility to a level needed to pass their “quality” test… possibly GoDaddy, ISTR they offer a free year to Open Source projects… no idea about one for the society… but it would solve the problem of not getting the certificates revoked. For everyone.

I am giving Startcom time until Friday after $dayjob (for me); after that, I’ll be kicking them off MirBSD’s CA bundle and will be lobbying for Debian and Mozilla to do the same.

Any other ideas of how to deal with that? I’d probably pay 5 € for a usable certificate accepted by people (including old systems, such as MSIE 5.0 on Win2k and the likes) without questioning… most of the time, I only serve public content anyway and just use SSL to make the NSA’s job more difficult (and even when not I’m not dealing with any payment information, just the occasional login protected area).

By the way, is there any way to access the information that is behind a current-day link to with Lynx or Pine? I can’t help but praise GMane for their NNTP interface.

ObFunfact: just when I was finished writing this wlog entry, I got a new eMail “Special offer just for you.” from GoDaddy. Sadly, no offer for a 5 € SSL certificate, just the usual 20-35% off coupon code.

I would like to publicly apologise for the inconvenience caused by my recent updates to the mediawiki and mediawiki-extensions source packages in Debian wheezy (stable-security).

As for reasons… I’m doing Mediawiki-related work at my dayjob, as part of FusionForge/Evolvis development, and try to upstream as much as I can. Our production environment is a Debian wheezy-based system with a selection of newer packages, including MediaWiki from sid (although I also have a test system running sid, so my uploads to Debian are generally better tested). I haven’t had experience with stable-security uploads before, and made small oversights (and did not run the full series of tests on the “final”, permitted-to-upload, version, only beforehand) which led to the problems. The situation was a bit complicated by the need to update the two packages in lockstep, to fight an RC bug file/symlink conflict, which was hard enough to do in sid already, plus the desire to fix other possibly-RC bugs at the same time. I also got no external review, although I cannot blame anyone since I never asked anyone explicitly, so I accept this as my fault.

The issues with the updates are:

  • mediawiki 1.19.5-1+deb7u1 (the previous stable-security update) was not made by me but by Jonathan Wiltshire
  • mediawiki 1.19.11+dfsg-0+deb7u1 (made by me) was fine, fixed the bugs it was supposed to, but was delayed after being uploaded to security-master-unembargoed
  • mediawiki 1.19.14+dfsg-0+deb7u1 was supposed to be a mostly upstream update, but I decided to add changes to fix issues pointed out by lintian (not trivial ones), and mistakenly forgot to remove two lines that should not have crept in from sid
  • mediawiki 1.19.14+dfsg-0+deb7u2 was quickly uploaded to fix this issue but took about half a day to be ACCEPTed
  • mediawiki-extensions 3.5~deb7u1 should have be named 2.12 but could not, due to the aforementioned lockstep update requirement and version checks in maintainer scripts; it fixes the issues but does not add other changes from 3.5 in sid… unfortunately, the packaging uses cdbs (which I dislike quite a lot, but as the newcomer in the team I decided to accept it and go on; changing the existing packaging would be quite some effort anyway) and wants debian/control to be regenerated from… which I thought I had done, and normally do…
  • mediawiki-extensions 3.6 (in sid) fixes another dir/symlink conflict shown up after 3.5 was made. I’ve requested upload permission for regenerating debian/control and asked whether I am allowed to include this fix as well

My unfamiliarity with some of the packaging concepts used here, combined with this being something I do during $dayjob (which can limit the time I can invest, although I’m doing much more work on Mediawiki in Debian than I thought I could do on the job), can cause some of those oversights. I guess I also should install a vanilla wheezy environment somewhere for testing… I do not normally do stable uploads (jmw did them before), so I was not prepared for that.

And, while here: thanks to the Debian Security Team for putting up with me (also in this week’s FusionForge issue), and thanks to Mediawiki upstream for agreeing to support releases shipped in Debian stable for longer support, so we can easily do stable-security updates.


06.02.2014 by tg@
Tags: archaeology debian fun jupp pcli

Just saw this in my INBOX:

    B. The default init system for jessie will be a single /etc/rc script

I’d certainly vote that❣

In unrelated news, jupp 2.8 for DOS runs on cable3, which means it’ll still run on an original 8088/8086 ☻

Update 10.02.2014: The unobfuscated version of cable3 is called 8086tiny under the MIT licence. Thanks to the author for doing that (and not just dumping the IOCCC code) and to RT from the mksh(1) IRC channel for finding it on the ’net!

FOSDEM preparations… done.

20.01.2014 by tg@
Tags: debian event fun grml mksh twitxr work

I’ve produced several pin-on buttons to take with me to FOSDEM for giving away (as long as there are any left):

Several pin-on buttons I made

First row (nice projects), from left to right: MidnightBSD; Glenda, the Plan 9 bunny; Teckids e.V.

Second row (The MirOS Project): mksh; the Shilouette Dæmon; the “Triforce” (Live+Install CDs for i386 and sparc, with MirGrml); “the m” (alternative logo, vector)

Third row (things originating from tarent): Freedroidz (now a Teckids project); OSIAM (Identity and Access Management); tarent (tarent AG, tarent GmbH), who sponsored production of these buttons

Hm… jupp needs a button’able logo!

FOSDEM meetup


02.12.2013 by tg@
Tags: debian

I’ve did something I surely will (financially) regret, next year, and designated the Neo900 to be the successor to my PocketPC, due to the latter having only 64 MiB RAM and Geocaching applications being quite hungry. It’s got a lovely hardware keyboard, a “pen” display like the PocketPC (as opposed to the “wishy-washy” displays that Android and iPhone have), not only GPS but also GLONASS, fully free software with mostly free firmware (I’m okay with that, mostly), a Ctrl key (useful in ssh and locally and my text editor; ^I is Tab, so it’s useful in shell, too), WLAN, UMTS (I don’t think I need LTE and would rather it have the more RAM), USB host (OTG), and lots of other nice features.

In short, it’s a tinkerable device: one I can not only hack at, but also hack on.

Since I use a “dumbphone” for mobile phone anyway (pro: separate battery from the “toy” PocketPC/Smartphone – we’re talking two+ weeks of battery time when using it here, and easier use and less bugs, and a reliable fallback when I tinker “too much”), this is perfect for me.

I’m reposting this in the wlog mostly because it’s an interesting technical and OSS project, and because if 1000 people want one it will get less expensive for all of us (while here… shameless plug… any sponsors willing to contribute some EUR so I don’t ruin myself with this, in exchange for services of some kind?). I’ll probably run Debian on it (unless it goes systemd), maybe in a chroot – if the native OS has functionality needed that I can’t simply put into packages; they say Maemo has much better power management, but considering most use will have GPS, GLONASS and backlight on, battery isn’t going to last long anyway… – or maybe even native… I’ve been wanting to know what this “freesmartphone” stuff my m68k (Atari VM) buildd has been happily compiling, anyway… and some sort of Geocaching application (ideally a cross between something online, CacheWolf and an offline OSM (with most of Europe, but uninteresting tags stripped) and possibly access to the GS Live API but nevertheless supporting TC, NC, OC, gpsgames too), and my usual mksh(1), GNU screen, jupp(1), lynx(1), ssh(1) toolchain.)

Delivery is expected for mid to end of 2014, but once it’s there I’ll keep you informed ☺

On that matter… I’ve got my PocketPC (currently in production use) and another WinCE device and wonder about tinkering with them, too. It appears to be a rather open platform (compared to Android, anyway) but most official documentation is tied to Windows® host systems, and most utilities have been taken offline after the abomination called Windows Phone has taken over. Hm I’ve got PocketPython and some sort of cross GCC but nothing to tinker with the core OS / ROM image…

FrOSCon is approaching, and all MirBSD developers will attend… but why’s there no MirBSD exhibit? The answer to that is a bit complex. First let’s state that of course we will participate in the event as well as the Open Source world. We’ll also be geocaching around the campus with other interested (mostly OSS) people (including those we won for this sport) and helping out other OSS projects we’ve become attached to.

MirOS BSD, the operating system, is a niche system. The conference on the other hand got “younger” and more mainstream. This means that almost all conference visitors do not belong to the target group of MirOS BSD which somewhat is an “ancient solution”: the most classical BSD around (NetBSD® loses because they have rc.d and PAM and lack sendmail(8), sorry guys, your attempt at being not reformable doesn’t count) and running on restricted hardware (such as my 486SLC with 12 MiB RAM) and exots (SPARCstation). It’s viable even as developer workstation (if your hardware is supported… otherwise just virtualise it) but its strength lies with SPARC support and “embedded x86”. And being run as virtual machine: we’re reportedly more stable and more performant than OpenBSD. MirBSD is not cut off from modern development and occasionally takes a questionable but justified choice (such as using 16-bit Unicode internally) or a weird-looking but beneficial one (such as OPTU encoding saving us locale(1) hassles) or even acts as technological pioneer (64-bit time_t on ILP32 platforms) or, at least, is faster than OpenBSD (newer GNU toolchain, things like that), but usually more conservatively, and yes, this is by design, not by lack of manpower, most of the time.

The MirPorts Framework, while technically superiour in enough places, is something that just cannot happen without manpower. I (tg@) am still using it exclusively, continuing to update ports I use and occasionally creating new ones (mupdf is in the works!), but it’s not something I’d recommend someone (other than an Mac OSX user) to use on a nōn-MirBSD system (Interix is not exactly thriving either, and the Interix support was only begun; other OSes are not widely tested).

The MirBSD Korn Shell is probably the one thing I will be remembered for. But I have absolutely no idea how one would present it on a booth at such an exhibition. A talk is much more likely. So no on that front too.

jupp, the editor which sucks less, is probably something that does deserve mainstream interest (especially considering Natureshadow is using it while teaching computing to kids) but probably more in a workshop setting. And booth space is precious enough in the FH so I think that’d be unfair.

All the other subprojects and side projects Benny and I have, such as mirₘᵢₙcⒺ, josef stalin, FreeWRT, Lunix Ewe, Shellsnippets, the fonts, etc. are interesting but share few, if any, common ground. Again, this does not match the vast majority of visitors. While we probably should push a number of these more, but a booth isn’t “it” here, either.

MirOS Linux (“MirLinux”) and MirOS Windows are, despite otherwise-saying rumours called W*k*p*d*a, only premature ideas that will not really be worked on (though MirLinux concepts are found in mirₘᵢₙcⒺ and stalin).

As you can see, despite all developers having full-time dayjobs, The MirOS Project is far from being obsolete. We hope that our website visitors understand our reasons to not have an exhibition booth of our own (even if the SPARCstation makes for a way cool one, it’s too heavy to lift all the time), and would like to point out that there are several other booths (commercial ones, as well as OSS ones such as AllBSD, Debian and (talking to) others) and other itineries we participate in. This year both Benny and I have been roped into helping out the conference itself, too (not exactly unvoluntarily though).

The best way to talk to us is IRC during regular European “geek” hours (i.e. until way too late into the night – which Americans should benefit from), semi-synchronously, or mailing lists. We sort of expect you to not be afraid to RTFM and look up acronyms you don’t understand; The MirOS Project is not unfriendly but definitely not suited for your proverbial Aunt Tilly, newbies, “desktop” users, and people who aren’t at least somewhat capable of using written English (this is by design).

TANSTA “softlink”. ITYM: symlink(7)

Actually, there’s “link” (also “hardlink”) and “symbolic link” (short “symlink”). (Oh, and reparse points, but let’s not get there, lest we mention *.lnk files…)

Inspired by a posting on the klibc mailing list.

Following the Wiki I put “discard” entries into my fstab(5) for swap (but not / as it’s suggested to use fstrim instead, and I had noatime for /boot and relatime for / already) and changed the scheduler. What wasn’t written there was to set vm.swappiness=0 in sysctl.d/local.conf *shrug* but it helps.

What they also didn't mention:

mount -t tmpfs swap /var/cache/apt/archives

And in sid, which my dayjob-laptop is running, APT _finally_ creates the missing “partial/” subdirectory itself. Thanks!

I had the filesystems already created, so I changed the ext options with tune2fs:

tune2fs -E stride=1024,stripe_width=1024 /dev/sda2 # 1 KiB blocks: /boot
tune2fs -E stride=256,stripe_width=256 /dev/sda4   # 4 KiB blocks: /

I wonder how well that works. I also did _not_ manage to find out my device’s flash block size (search engine fodder: erase block size), so I assumed 1 MiB (also used for partition alignment already) as worst-case scenario. Input welcome on how to find *that* out.


18.07.2013 by tg@

Michael Langguth and Scalaris AG asked me to publish the mksh/Win32 Beta 14 source and binary archive, and it is with joy I’m doing this.

Checksums and Hashes

  • RMD160 (ports/ = 0dc8ef6e95592bd132f701ca77c4e0a3afe46f24
  • TIGER (ports/ = 966e548f9e9c1d5b137ae3ec48e60db4a57c9a0ed15720fb
  • 1181543005 517402 /MirOS/dist/mir/mksh/ports/
  • MD5 (ports/ = b57367b0710bf76a972b493562e2b6b5

Just a few words on it (more in the README.1st file included): this is a port of The MirBSD Korn Shell R39 to the native WinAPI; it’s not quite got the full Unix feel (especially as it targets the Weihenstephan unxutils instead of a full Interix or Cygwin environment) but doesn’t need a full POSIX emulation layer either. It’s intended to replace MKS ksh and the MKS Toolkit. Source for the compatibility library is also included under The MirOS Licence; we aim at publishing it as OSI Certified Open Source Software like mksh itself. (There is a situation with dlmalloc/nedmalloc being resolved, and the icon is derived from the BSD dæmon which is a protected unregistered trademark, but we’re not Mozilla and allow distro packages to keep using it ☺) Rebasing it on a newer mksh(1) followed by (partial) integration into the main source code is a goal.

Have fun trying it out and hacking on it. It’s currently built with -DMKSH_NOPROSPECTOFWORK (so coprocesses and a few other minor things won’t work), but a SIGCHLD emulation is being worked on – but if you want to help out, I’m sure it’s welcome, just come on IRC or post on the mailing list, and I’ll forward things to Michael as needed. Reports on testing with other toolchain and OS versions are also welcome.


07.07.2013 by tg@
Tags: debian fun


Time for more neighbours’ cat posts, apparently. It’s warm, so the cat’s sleeping outside. Not disturbed by much.

Me envious. Too warm to go to the ice salon (bike’s in repair, car’s hot enough to boil eggs on it, public transport not better).

virsh send-key guestname KEY_LEFTALT KEY_SYSRQ KEY_H

This doesn’t work in virt-manager, but the virsh CLI tool is just fine.

PS: zerofree rocks! And can be installed in Grml 2011.05 just finely.

Okay, so imagine this: you just generated an SSH RSA key and threw its public part on system B into ~foo/.ssh/authorized_keys and its private part on system A into ~bar/.ssh/id_rsa but can’t login. Why?

Automated processes (Jenkins *cough*) often need you to ssh(1) manually once, to accept the remote host’s server key. Do that.

The id_rsa file on system A must be owned by the user bar and chmod 0600 or 0400 (similarily, the .ssh directory has strict permission checks, and everything in the path until there). Check those.

And, the most surprising one of the day: if there’s an it will be used for offering a key to the remote host (B) even if it does not match the secret key. Deleting A:~bar/.ssh/ apparently makes OpenSSH generate the public part from the secret key each time (or just put the correct pubkey there), but if one’s there, it seems to like to use them. (That was the only part of this post that was news to even me, of course ☺)

And, as bottom line: hello to Planet Debian from “mirabilos at work”, too. I’ll occasionally tag posts so they show up here, if I think they’re of interest, since I’m doing Debian work at the dayjob, too.

Waypoint Statistics

08.06.2013 by tg@

mirabilos’ found waypoints

I’ve finally gotten around to listing all Waypoints (Geocaches, Opencaches, Closedcaches, Earthcaches, Terracaches including Locationless, Navicaches, etc.) I’ve found a box, enjoyful, educating, a good place to hide one myself, etc. and putting up a list and, of course, generate my own statpic.

I’ll put them up for the other project members, too (already made a picture for gecko2@ but bsiegert@ still needs one; we also need to collect offline lists of found, owned and attended waypoints)…

A bit of background story: I decided, years ago, to have an offline list of cache finds in case something would happen. Just, I had found way too many already, so this was a huge bit of work. Oh well… I of course procrastinated, and then something did happen (Opencaching wanting to force a Restricted Commons licence; me disagreeing and suggesting a change; some trigger-happy person immediately deleting my account without waiting for the discussion or the decision period to end; weeks of forum discussions; Opencaching allowing dual-licencing; them telling me they can’t restore my data – probably never heard of databa…sorry, MySQL backups). And I still didn’t have the list. Now I do; recreated even the OC information from what was still accessible and with help from one OC supporter (“mic@”, thanks); merged caches that are co-listed on several platforms, etc. (still need to put in the FTF/STF/TTF/4TF/LTF and voting/favourites information) and a statpic, all in Open Source and Open Data, in cvs(1) with mksh(1) and… a… frontend for libgd2 I admit, but we had been using that for the MirWebsite for a while already.

I suggest every geocacher keep an offline or local record of all their finds (and hides and attended logs) for things like this, in case some platform decides to… let’s say, “put your data into the cloud… where it is? I don’t know”.


20.05.2013 by tg@
Tags: archaeology debian

Apparently (hi Zhenech, found on Plänet Debian), a Man does not only need to fork a child, plant a tree, etc. in their life but also write a DynDNS service. Perfect for opening a new tag in the wlog called archæology ( – Some Assembly Required is also a nice example for these).

Once upon a time, I used SixXS’ heartbeat protocol client for updating the Legacy IP (known as “IPv4” earlier) endpoint address of my tunnel at home (My ISP offers static v4 for some payment now, luckily). Their client sucked, so I wrote on in ksh, naturally.

And because mksh(1) is such nice a language to program in (although, I only really begun becoming proficient in Korn Shell in 2005-2006 or so, thus please take those scripts with a grain of salt, I’d do them much differently nowadays) I also wrote a heartbeat server implementation. In Shell.

The heartbeat server supports different backends (per client), and to date I’ve run backends providing DynDNS (automatically disabling the RR if the client goes offline), an IP (IPv6) tunnel of my own (basically the same setup SixXS has, without knowing theirs), rdate(8) based time offset monitoring for ntpd(8), and an eMail forwarding service (as one must not run an MTA on dynamic IP) with it; some of these even in parallel.

Not all of it is documented, but I’ve written up most things in CVS. There also were some issues (mostly to do with killing sleep(1)ing subprocesses not working right), so it occasionally hung, but very rarely. Running it under the supervise of DJB dæmontools was nice, as I was already using djbdns, since I do not understand the BIND zone file format and do not consider MySQL a database (and did not even like databases at all, back then). For DynDNS, the heartbeat server’s backend simply updated the zone file (by either adding or updating or deleting the line for the client) then running tinydns-data, then rsync’ing it to the djbdns server primary and secondaries, then running zonenotify so the BIND secondaries get a NOTIFY to update their zones (so I never had to bother much with the SOA values, only allow AXFR). That’s a really KISS setup ☺

Anyway. This is archæology. The scripts are there, feel free to use them, hack on them, take them as examples… even submit back patches if you want. I’ll even answer questions, to some degree, in IRC. But that’s it. I urge people to go use a decent ISP, even if the bandwidth is smaller. To paraphrase a coworker after he cancelled his cable based internet access (I think at Un*tym*dia) before the 2-week trial period was even over: rather have slow but reliable internet at Netc*logne than “that”. People, vote with your purse!

mksh R45 released

26.04.2013 by tg@

The MirBSD Korn Shell R45 has been released today, and R44 has been named the new stable/bugfix-only series. (That’s version 45.1, not 0.45, dear Homebrew/MacOSX packagers.)

Packagers rejoice: the -DMKSH_GCC55009 dance is no longer needed, and even the run-time check for integer division is gone. Why? Because I realised one cannot use signed integers in C, at all, and rewrote the mksh(1) arithmetics code to use unsigned integers only. Special thanks to the people from musl libc and, to some lesser amount, Natureshadow for providing me with ideas what algorithms to replace some functionality with (signed shell arithmetic is, of course, still usable, it is just emulated using unsigned C integers now).

The following entertainment…

	tg@blau:~ $ echo foo >/bar\ baz
	/bin/mksh: can't create /bar baz: Permission denied
	1|tg@blau:~ $ doch
	tg@blau:~ $ cat /bar\ baz

… was provided by Tonnerre Lombard; like Swedish, German has got a number of words that cannot be expressed in English so I feel not up to the task of explaining this to people who don’t know the German word “doch”, just rest assured it calls the last input line (be careful, this is literally a line, so don’t use backslash-newline sequences) using sudo(8).

Since a while…

I am a proud
EarthCache Master

On the other hand… I should probably put up my own, local, list of found caches, considering what happened to me on “Open”caching. And maybe write intros for people new to geocaching, since it’d be virtually no work now had I done it initially. (And for fanfiction readers! I wish I’d kept a list of read fics, not just of these I currently read and/or are currently unfinished.)

GNU autotools generated files

20.02.2013 by tg@
Tags: debian rant

On Planet Debian, Vincent Bernat wrote:

The drawback of this approach is that if you rebuild configure from the released tarball, you don’t have the git tree and the version will be a date. Just don’t do that.

Excuse me‽

This is totally inacceptable. Regenerating files like aclocal.m4 and (for automake), configure (for autoconf), and the likes is one of the absolute duties of a software package. Things will break sooner or later if people do not do that. Additionally, generated files must be remakable from the distfile, so do not break this!

May I suggest, constructively, an alternative? (People – rightfully, I must admit – complain I’m “just” ranting too much.)
When making a release from git, write the “git describe” output into a file. Then, use that file instead of trying to run the git executable if .git/. is not a directory (“test -d .git/.”). Do not call git, because, in packages, it’s either not installed or/and also undesired.

Couldn’t comment on your blog, but felt strongly enough about this I took the effort of writing a full post of my own.

(But thanks for the book recommendation.)

git log -n 1 --all --full-history --pretty=format:'%cD'

This should™ scan all branches, take the chronologically last commit and output its committer date. Still doesn’t take into account git-receive-pack times, but we can just look at the mtime of the projectname-commits@lists.forgename mailing list for that.

PSA: Referring to Unicode codepoints.

If your Unicode codepoint is, numerically, between 0 and 65533, inclusive, convert it to hexadecimal and zero-pad it to four nibbles. For example, the Euro sign € is Unicode codepoint #8364 which is 20AC hex; the Eszett ß is 223 which is DF hex, padded 00DF.
Then write an uppercase ‘U’, a plus sign ‘+’, and the four nibbles: U+20AC U+00DF
In mksh, JSON, etc. it’s a backslash ‘\’, a lower-case ‘u’ and four nibbles.

Otherwise, your Unicode codepoint will be, numerically, between 65536 and 1114111, inclusive, that is hex 10000 to 10FFFF. (There’s nothing on 65534 and 65535, nor above these figures.) In this case, convert it to hex, zero-pad it to eight nibbles and write it as an uppercase ‘U’, a hyphen-minus ‘-’ and the eight nibbles. In C-like escapes for environments supporting the Unicode SMP, that’s a backslash ‘\’, an upper-case ‘U’ and eight nibbles. Do not, in either case, use less (or more) hex digits than specified here. For example, there’s a famous Unicode codepoint U-0001F4A9 “PILE OF POO”. That’s not the same as U+1F4A9. The latter reads as U+1F4A “GREEK CAPITAL LETTER OMICRON WITH PSILI AND VARIA” and a digit 9 (Ὂ9). Be educated.

Since this wlog runs on MirBSD, which limits itself to the Unicode BMP voluntarily, and as nōn-BMP is not widespread anyway, I cannot reproduce the “PILE OF POO” here, but you can just duckduckgo it.

Let’s start a convention: bare-metal machines have the linguistic male gender („der Computer“, he needs to be rebooted), whereas VMs have the linguistic female gender („die virtuelle Maschine“, she runs better since the last upgrade of Linux-KVM), and neutral linguistic gender is used when you cannot or do not want or need to make such distinction.
This is, of course, entirely unrelated to human gender, but not unrelated to #debian-68k (on OFTC) discussions ;-)

ObRant: DO NOT USE xz COMPRESSION LEVELS ABOVE 6! (For -7 we can make exceptions, for example in Debian *-dbg or *-source packages.) You may use -e if you absolutely need the better compression, but please think of the poor sods who have to create the archives. You must not use the highest compression levels -8 or -9 since they have absolutely insane memory requirements on compression and will still hinder machines with less RAM on decompression. (Using -e only affects CPU usage at compression time; decompression is exactly as fast and memory-consuming as without.) Furthermore, DO NOT CHOOSE A COMPRESSION LEVEL WITH A DICTIONARY SIZE MUCH LARGER THAN THE DATA TO COMPRESS, as that makes absolutely no sense and will rather worsen than improve compression. As a reminder, xz uses the following dictionary sizes:

  • 256 KiB at -0 (compresses better than gzip(1) and faster than either gzip(1) or bzip2)
  • 1 MiB at -1
  • 2 MiB at -2 (compresses better than gzip(1) and bzip2 without losing much speed)
  • 4 MiB at -3 and -4 (the difference is in the match finder between these two levels)
  • 8 MiB at -5 and -6
  • 16 MiB at -7 (186 MiB RAM used to compress a file)
  • 32 MiB at -8 (370 MiB RAM used to compress a file)
  • 64 MiB at -9 (674 MiB RAM used to compress a file)

Decompression uses less than 1 MiB more than the dictionary size, but the dictionary must always be allocated wholly. (You’re fine to use custom presets, but mind the RAM usage!) As a general rule, if you have something of up to 20 MiB to compress, -4 is fine, and -5 will only be better if you have similar data spread across the whole of the file instead of close to each other. When I make mksh distfiles, I instead put files close to each other that have related content, which improves compression much more nicely without penalising low-memory systems; for example, you could put documentation, Makefiles, scripts, m4(1) files, and C source code into groups before archiving, instead of doing it alphabetically.

Another note on bzip2: its decompression is slow. I see no reason to use it any more, at all. Use gzip(1) if you care for compatibility or have an issue with xz not having a free copyright licence, and xz otherwise.

mksh made quite some waves (machine translation of the third article) recently. Let’s state it’s not just Amigas – ara5 is a buildd running the Atari kernel, an emulated though. On the other hand, the bare-metal Ataris used to be the fastest buildds, so I expect we get them back online soonish. I’m currently fighting with some buildd software bugfixes, but once they’re in, we will make more of them. Oh, and porterboxen! Does anyone want to host a VM with a porterbox? Requirements: wheezy host system (can be emulated), 1 GiB RAM, one CPU core with about 6500 BogoMIPS or more (so the emulated system has decent speed; an AMD Phenom II X4 3.2 GHz does just fine). Oh, and mksh is ported to more and more platforms, like 386BSD 0.0 with GCC 1.39, and QNX 4 with Watcom… and more bugfixes are also being worked on. And let’s not forget features!

jupp got refreshed: it’s got a bracketed paste mode, which is even auto-enabled on xterm-xfree86 (though the xterm(1) in MirBSD’s a tad too old to know it; will update that later, just imported sendmail(8) 8.14.6 and lynx(1) 2.8.8dev.15 into base, more to come) and will be enhanced later (should disable auto-indent, wordwrap, status line updates, and possibly more), lots of new functions and bindings, now uses mkstemp(3) to create backup files race-free, and more (read the NEWS file).

In MirBSD, Benny and I just added a number of errnos, mostly for SUSv4 compliance and being able to compile more software from pkgsrc® without needing to patch. This is being tested right now (although I should probably go out and watch fireworks in less than a half-hour), together with the new imports and the bunch of small fixes we accumulate (even though most development in MirBSD is currently in mksh(1) and similar doesn’t mean that all is, or worse, we were dead, which we aren’t). I’ll publish a new snapshot some time in January. The Grml 2012.12 also contains a pretty up-to-date MirBSD, with a boot(8/i386)loader that now ignores GUID partition table entries when deciding what to use for the ‘a’ slice.

If you haven’t already done so, read Benjamin Mako Hill’s writings!

Der heilige… Frieden?

15.12.2012 by tg@
Tags: debian politics

(Apologies for putting this on Planet Debian, but it says the one or other non-English post is okay as long as it’s an exception. I feel I need to reach more people with this, but don’t feel like translating this into English right now.)
Update: Tanguy asked for a short English summary: it’s me ranting against the rioting against muslims and the call for more CCTV surveillance after a possible bomb was found at the train station.

In Bonn herrscht immer noch „Bombenstimmung“, wenn man z.B. auf die Webseite der Lokalzeitung schaut – von dem Amoklauf in Connecticut, über den sich im IRC gewunder wird, ist immer noch nichts zu sehen, dafür wird fleißig wider „Islamisten“ gehetzt.

Ich finde das besorgniserregend, muß doch jetzt jeder Angehörige des Islams fürchten, verfolgt oder benachteiligt zu werden. Das reizt doch erst recht zum Gegenschlag, bei dem dann auch Menschen, die absolut nicht mit der hier vorherrschenden Meinung und Politik übereinstimmen, getroffen werden können.

Ich persönlich habe kein Problem mit Menschen anderen Glaubens oder anderer Weltanschauung, solange wir friedlich miteinander leben können. Ich teile eure Unzufriedenheit mit dem herrschenden Staat, der immer weitergehenden Überwachung, Unterdrückung von Leuten, die nicht dem vorherrschenden Menschenbild entsprechen (egal an welchen Kategoriën), und bitte die, die dies lesen, nochmal nachzudenken, bevor sie etwas tun, was hinterher Unschuldige trifft oder gar in „friendly fire“ ausartet.

Hat eigentlich wer die in Bad Godesberg ausgegebenen Koran-Bücher sich mal angeschaut? Als ich davon las, war ich ja zugegebenermaßen neugierig, weil ich vom Koran leider eher wenig kenne, weiß aber nicht, wie neutral oder eben nicht die Übersetzung gehalten ist. Anhand dessen, was ich bereits mitbekam, sollte das eher friedlicher sein als was durch spätere Theologen festgelegt wurde – wie ja auch zum Beispiel im Christentum, aber über die Horrorepisoden der christlichen Kirche will ich jetzt auch nicht mich auslassen, in der Hoffnung, daß auch diese sich mit den Jahren gebessert hat. (Ist nur halt das Problem mit den Leuten, die die „alten Hetzparolen“ jetzt noch verbreiten. Ist wie im Netz mit den Groupies von Theo de Raadt, die noch asiger zu Leuten sind als er selber.) (Außerdem muß man ja befürchten, durch Besitz eines Korans schon vorverurteilt zu werden heutzutage *seufz*… ich finde das nicht gut!)

Update (ich vergaß): auch der Ruf nach mehr Videoüberwachung ist nur Panikmache. Das geht nur zu Lasten des Normalbürgers. Vielleicht lassen sich noch Kleinstdelikte wie Taschendiebstahl damit abschrecken, aber gerade diese Bomben und dergleichen sind doch oft von Leuten, die vor Konsequenzen keine Angst haben, organisiert. Die werden dann maximal Märtyrer. Ich wiederhole nochmal für die Politiker und die ganz langsamen unter den Lesern: Überwachung verhindert keine Straftat.

Update 11.01.2013: Mittlerweile hat auch Fefe was dazu.

Before we begin, everyone should read up on hashtables and what open addressing / closed hashing is. The context is lines 111‥190 of Python’s Objects/dictobject.c as of today (so we get the line numbers straight).

(I’ve reworded this wlog entry a bit; I originally wrote it too late at night for it to read coherent.) Basically, I’ve got an application where I’d like to use a hashtable for a number of things – not as generic as Python, and with focus on small footprint. I’d like to offer associative arrays in a scripting language, where the keys are always arbitrary byte strings excluding NUL. Also, I’d like to use the hashtable as backend for indexed arrays, where the keys are uint32_t and the usual use case is sequential. Finally, I’m using it for several internal tables, such as a list of keywords, one of builtins, one of special variables, etc. which is a reason for me to not use a self-balancing binary tree as data structure (reading further below might suggest that, but getting a sorted list of hashtable keys is not the focus, though not unimportant).
My questions on this are:

① Why is the shift on perturb done after its first use? In my experiments (using 32-bit width everywhere), for the pathological case of an 8-element (i = 3) table with three entries 0, 0x40000000 and 0x800000000, the “second round” yields 1 for all three, so it cannot have to do with the upper bits. My lookup looks like:

	mask = 2ⁱ - 1;
	j = perturb = hash(key);
	goto find_first_slot;

	j = (j << 2) + j + perturb + 1;
	perturb >>= PERTURB_SHIFT;

	entry = table[j & mask];
	if (!match(entry)) goto find_next_empty_slot;

This means that my first check is always the bare hash (so “only do it if needed” is no reason) and, since I’m using gotos, I could just move the perturb >>= PERTURB_SHIFT; line before the line recalculating the next j to use. This seems to make more sense, even in the face of Python. (I actually looked at the Python file’s comments again today because I thought to use a different resolution, but they have a good rationale for using the multiplication by 5.)

② Why can’t we just use i as the PERTURB_SHIFT? Sure, this changes a shift-right by a constant, which can possibly be encoded as immediate value in assembly (unless you’re on a pre-80186, which can only do SHR AX,1 and SHR AX,CL but not SHR AX,4, but that’s outside of mksh’s scope) into a right-shift by a variable, but i is already known, and I think the behaviour is better (it wouldn’t eat any bits; assume the same 8-entry hashtable and pathologic keys 0, 8 and 16). Again: who do I think I am to go against the wisdom of the Python people, who seem to have shed more thought on this than everyone else I saw, asked, read about (including Spammipedia). That’s why I’m asking here. On that reference: I don’t support spammers or people nagging for donations or premium accounts, like Xing and Groundspeak/Geocaching.COM, at all. In fact, I urge others to do the same, so it really hurts them; it may be their business model, but not if they spam me. Besides, OpenCaching.DE exists.

Another thing is: to avoid CVE-2011-4815, I’m randomising the hash used, with one “seed” value per hashtable, changed before a resize operation. I originally thought to seed it with nonzero, but then I have to rehash on hashtable resize, so I’ll be XORing the final hash value instead (thanks ciruZ for the idea). I’m thinking of omitting that for indexed arrays, as an attacker almost certainly cannot determine the keys there. (To directly use the indexed array keys, which are already uint32_t, as hashes makes using i from ② even more important.) The hash I’m using is a modified Jenkins one-at-a-time called NZAAT: it’s my new generic standard nōn-cryptographic hash, and the changes are thus: while adding a byte, another increment of the hash is done (so NUL counts), and the finaliser got prefixed with the shift-left-add+shift-right-xor sequence of the adder (but not adding any value or the +1), to get best avalanche for all bytes. I actually compiled several versions of Hash.cs on a Windows® VM at work to analyse the original one-at-a-time and all of my modifications; these turned out to be the simplest ones (I originally had added 0x100 instead of 1, but the effect was the same, and +1 is usually cheaper on most CPUs).

Also, to avoid people being able to get to the seed, a user will always get only a sorted list of hashtable keys (numeric for indexed arrays, ASCIIbetically otherwise; see also my thoughts on JSON from the previous wlog entry). What algorithm do I use? For strings, comparisons are much more expensive, so I’d like to keep them low. Memory use is also a factor; allocating one large(r) block is better than many small ones due to the pool allocator overhead and due to portability to ancient Unicēs (which is another reason for me to use a hashtable which is a small struct plus an array of pointers, and then pass the list of keys as array of string pointers, instead of a tree). For both reasons, I’m thinking a relatively simple MergeSort: I need to allocate the result array anyway, so I can just get two and free the one that isn’t the end result, and it’s AFAICT the cheapest on comparisons other than Tree Sort (which nobody really seems to use, and which would effect to using a balanced binary tree again). Since keys are unique, stability and duplicate handling is never an issue. I’d like to use only one algorithm and one data structure, not a combination, as compactness is a design goal.

Please drop your thoughts on Freenode, e.g. by /msg MemoServ send mirabilos your text here or per eMail to the domains debian, freewrt or mirbsd, which are organisations, with the localpart tg. Or just contact me as usual, if you’re already acquainted. Or lookup 0xE99007E0. Thanks in advance! (Especially, Python Developers’ thoughts are welcome.)

The following proposal extends the JSON specification, with the idea of using JSON as an information interchange format, rather than just a way of writing certain ECMAscript values. They do not add anything but only restrict valid JSON content and encoders with some rationale.

First of, I’d like to remind everyone, including JSON’s author, that JSON is case-sensitive, except in the four hexdigits after a backslash-u sequence in a String.

Second, I’d like to remind everyone that JSON is not binary-safe. No way around that, it implements Unicode (actually, 16-bit UCS-2, and it doesn’t guarantee that UTF-16 surrogates are correctly paired) text. I also consider only UTF-{8,{16,32}{B,L}E} valid encodings for JSON. (No PDP endian, either. Sorry, guys.)

For my first proposal, I’d like to point out CVE-2011-4815 which was about overflowing hashtables. The obvious fix is to randomise the hash per hashtable; to ensure this doesn’t leak, we sort ASCIIbetically the keys of an Object in the encoder. (Using Unicode is good here – we can just sort the keys as UTF-8 strings by their uint8_t value or as Unicode (UCS-2 or even UCS-4 or UTF-16) strings by the codepoints.) JSON was never preserving the order of elements in an Object anyway so we make it standardised (we still accept any order, and, when parsing, in collision cases, the later value wins). This also helps diffs.

For my second proposal, I’d like to forbid \u0000, \uFFFE, \uFFFF in strings. The first because many implementations use C strings, and for an information interchange format this is better; it also has security implications to allow NUL in a string. The other two, but not unpaired UTF-16 surrogates (as ECMAscript uses UCS-2 and got UTF-16 only later) because they’re not valid Unicode; JSON was not binary-safe already so why bother. Among other benefits, this also helps implementations.

For my third proposal, I’d like to agree that implementations should impose a nesting depth limit that may be user-defined, and in the face of which, cyclic checking may be ignored by an encoder. I emit nesting depth overflows as literalnull; might also throw an error. Since I was asked, the common “standard” value is to restrict nesting depth is 32, unless the user specified one. (I also saw 8, but 32 WFM pretty well.) Most seem to use it even if it may seem low at first. Only specialised applications probably need more, and they can always pass a value.

For my forth proposal, backslash-escape U+007F‥U+009F always. It may upset humans, editors, databases, etc. (This paragraph is newly added, after some IRC discussion.)

All these do not permit anything that wasn’t accepted to be accepted afterwards. I’ve got a fifth proposal which changes acceptance rules – but only for a subset of parsers: formally JSON is defined in ECMA-262 as industry standard that, in contrast to RFC 4627, always allowed any Value as top-level element of a JSON text. I’d like to make it so, and ignore the RFC’s requirement for it to be an Object or Array. Even so, the first two characters (after the BOM, if any) of a JSON text always are in the non-NUL 7-bit ASCII range, allowing for encoding detection. (This is done by the NUL octet pattern in the first four octets.)

JSON has only taken off because it’s a tightly defined simple format that can be used “everywhere” and isn’t too awful for humans (escaping not needed for U+0020‥U+D7FF and U+E000‥U+FFFD after all, although I’d also take the C1 control characters out, see my forth proposal above). I’ve started to use a trailing comma in indexed and associative arrays in code I write at work, when the array values are one a line, to help version control systems to do their diffs, but refrain from asking for a JSON extension to permit that in order to not endanger compatibility any (no comment needed, it’s just not worth it), but I’d like my above proposals to be followed by implementators (and I’m one of them).

Some more discussion with Jonathan pointed out that JSON5 allows for trailing commata in Object and Array; IMHO the only feature of it that is not bad or outright harmful. I’ll probably keep from accepting them because, on their own, they’re not that useful, and I usually would run JSON texts, even configs, through a parser/encoder roundtrip to pretty-print them which would lose them anyway.

As for binary-safeness: probably best to just use base64 and let the outer layers worry about compression. The data is usually unrelated to the JSON-encoded structure, and even if it’s related to other data the base64 representation is usually similar (unless misaligned).

Update 02.12.2012 – Wrong I was about the first two characters: “"€"” is a valid JSON text. Still possible to peek at four octets and determine the encoding by ordering the tests; updated my notes.

PostgreSQL hatte vor kurzem ein Problem, und zwar in der Version 9.1.5, welches zu Datenkorruption führen kann. Ist in der Version 9.1.6 (und 9.2.1) gefixt. Dummerweise muß aber jede Datenbank, die auch nur einmal mit 9.1.5 gestartet wurde, gefixt werden, weil es sonst zu Datenkorruption kommen kann.

Schlimmer: die kaputte Version 9.1.5 wird aktuell mit precise-security in Ubuntu ausgeliefert und war für ca. ein Dutzend Tage in wheezy!

Nach dem Upgrade auf 9.1.6 gestaltet sich das Fixen wie folgt, als Superuser:

  • Die /etc/postgresql/9.1/main/postgresql.conf editieren: die Konfigurationseinstellungen (ggfs. erst hinzufügen) vacuum_freeze_table_age = 0 und vacuum_cost_delay = 50 setzen
  • Die Datenbank stoppen: /etc/init.d/postgresql stop
  • Prüfen, ob ps ax | fgrep postgres wirklich nix mehr zurückliefert
  • Die Datenbank starten: cleanenv - /etc/init.d/postgresql start
  • Ggfs. alle Anwendungen, die PostgreSQL (dauerhaft) benutzen, wie apache2 (Evolvis) und tomcat6 (Domisol) neu starten
  • Zum Systemuser wechseln – su - postgres – und vernünftige Sprache auswählen: Debian export LC_ALL=C.UTF-8 Ubuntu export LC_ALL=C
  • Alle Indicēs regenerieren: reindexdb -a
  • Staubsaugen: vacuumdb -F -z -a (optional noch mit -v zum mehr (zu viel) sehen)
  • Den PostgreSQL-User wieder verlassen: exit
  • Die beiden Konfigurationsänderungen von oben wieder rückgängig machen
  • Falls gewünscht, die Änderungen aktivieren: cleanenv - /etc/init.d/postgresql reload

Ich hab’ das mal für alle EvolvisForge- und tarent-activity-Maschinen gemacht, aber eure Desktops und so aktualisiert ihr bitte selber, wenn auch nur die Chance besteht, daß mal ein 9.1.5 oder 9.2.0 installiert war!

I’ve been debugging a weird problem at work – after upgrading a complex system from lenny to wheezy, some https clients failed to connect: GNU wget and Debian’s version of lynx(1) which is linked against libgnutls26 fail. NSS applications continue to work, as does cURL; wget and lynx on MirBSD (linked with OpenSSL of course) work. Even Debian’s gnutls-cli tools from both gnutls26 and gnutls28 work. Huh. The error_log shows renegotiation problems, yet setting the new Apache 2 configuration option to “use insecure renegotiation” doesn’t help either. (The option is a total #FAIL: its only other value is “use secure TLSv1.x renegotiation”, but I don’t want/need SSL renegortiation at all, anyway.) Natureshadow told me this was a hot issue on Debianforum at the moment, yet, nobody had a clue or enough information to file a formal bugreport against (initially) apache2, as that’s what changed. I tracked it down on a new VM with no configuration otherwise, and here are my findings so others don’t run into it.

Tracking down the problem, this can be reduced to the following configuration (minimised, to show the problem) in /etc/apache2/sites-enabled/1one:

	<VirtualHost *:443>
		RedirectMatch permanent .
		SSLEngine on
		SSLCertificateFile /etc/ssl/W_lan_tarent_de.cer
		SSLCertificateKeyFile /etc/ssl/private/W_lan_tarent_de.key
		SSLCertificateChainFile /etc/ssl/

Do not mind the actual content, this is a very stripped-down demo on a not-actually-set-up-yet box.

Same is valid for the companion configuration file /etc/apache2/sites-enabled/2two:

	NameVirtualHost *:443

	<VirtualHost *:443>
		SSLEngine on
		# workaround for BEAST (CVE-2011-3389), short-term
		SSLCipherSuite RC4-SHA
		SSLCertificateFile /etc/ssl/W_lan_tarent_de.cer
		SSLCertificateKeyFile /etc/ssl/private/W_lan_tarent_de.key
		SSLCertificateChainFile /etc/ssl/
		SSLProtocol TLSv1

Turns out the BEAST workaround was at fault here: the differing SSLCipherSuites between the vhosts (on the same Legacy IP / TCP Port tuple, as we use Wildcard SSL Certificates) made Apache 2 want to renegotiate, so either commenting it on 2two or, better, adding it to 1one helped. Interestingly enough, the SSLProtocol directive did not matter (in my tests).

So, keep SSL settings synchronised between vhosts. In fact, those were already from include files, but 2two was from the “Evolvis 5” generation, whereas we added to 1one an Include of the file generated by the previous releases of EvolvisForge and had not switched those legacy vhosts to the new configuration, as everything worked on lenny.

This wlog entry brought to you by the system administrators of tarent solutions GmbH and the Evolvis Project, based on FusionForge.

Update 17.05.2013 – Absolutely do not use RC4-SHA for SSL/TLS (https)! It can leak over 200 initial plaintext bytes easily. (arc4random(3) is not affected from this, especially on MirBSD, nor arc4random(9).)

Originally posted by bubulle on Planet Debian, a shell prompt that displays the current git branch, in colour on some terminals, after the current working directory. The following snippet does similar things for mksh users, except it doesn’t redefine your prompt but amend it – just throw it at the bottom of your ~/.mkshrc before that last line beginning with a colon (copy from /etc/skel/.mkshrc if you haven’t done that yet):

	function parse_git_branch {
		git branch 2>/dev/null | sed -n '/^\* \(.*\)/s//(\1)/p'

	function amend_prompt_with_git {
		local p q='$(parse_git_branch)' r

		if [[ $TERM = @(xterm-color|xterm|screen*) ]]; then
			if [[ ${PS1:1:1} = $'\r' ]]; then

		p=${PS1%%*( )[#$]*( )}
		if [[ $p != "$PS1" ]]; then
			# prompt ends with space + #-or-$ + space, we can amend
			r=${PS1: ${#p}}
	unset -f amend_prompt_with_git

The indirection by use of a function is not strictly necessary but allows the use of locals. I took the liberty of adding an asterisk after “screen” to match the GNU/Linux nonsense of having TERM=screen.xterm or somesuch.

KiBi is my hero of the day. I’ve long wondered why I couldn’t select fixed-misc as font on my workstation at the dayjob, which is running K?buntu Hardon Heroin. (Luckily, I managed to avoid upgrading to Prolonged Pain.) Now I guess that’ll work again.

My work laptop (running testing) also has got this thingy. My keyboard layout now has got a grml branch (named after the person who first cursed about the insane idea of those toy-breaking boys to rearrange the keycodes) that works with it. Since Debian is marginally more sane than K?buntu, in contrast to the gnu branch I use on my orkstation, the grml branch still has Meta on the left Alt key, not Mode_switch, as it still works in uxterm, which reduces the diff between the MAIN branch (HEAD) on XFree86® and this beast.

And finally: defaults to a black screen and disabled mouse pointer until an application first requests it. Totally unacceptable for evilwm(1) users, and letting people think it crashed, to boot. The Arch Linux guys found this, among others; the fix is: startx(1) users edit /etc/X11/xinit/xserverrc to add -retro behind the X, or copy the file to ~/.xserverrc and change it there:


	exec /usr/bin/X -retro -nolisten tcp "$@"

For display managers, similar files exist in /etc/kde4/kdm and related places.

Update: Also, newer xterm(1) justify an update to ~/.Xresources for we can finally get rid of cut buffers, and get a blinking underline cursor to boot!

On the other front, worked on Debian packaging, and upstream on pax(1) and jupp, with more things to follow (especially in mksh). Also fixed about ⅔ Linux klibc architectures and learned why I’m a BSD developer despite all the bad parts of it ☺ and fixed fakeroot with pax(1) on Hurd… incidentally in code originally designed to support the Linux pax. My dayjob’s keeping me busy, but I’ve got plans to run mksh(1) through Sonar, in addition to the static code analysēs done by (once again, thanks!) Coverity (commits to mksh pending) and Clang/LLVM scan-build. Uhm, what can I say more, grab me in IRC if you need it. Ah, and some other mksh things coming up that may be of interest to people needing to support legacy scripts.

While wtf(1) always has been a bit central to MirBSD, and the acronym database has been accessible by CVSweb, what we never had was a DAU compatible (and shellsnippets compatible) lookup. This has now changed: the above link to the acronyms file is a persistent link to its latest version (well, latest when the website was last recompiled), tooltips may very well follow soon, and we’ve got an online WTF lookup service.
Contributions to the acronym database are welcome, of course; just eMail them to

Not to stop there, our online HTML manpage search is also new, shiny, and should replace the “!mbsdman” DuckDuckGo hash-bang shortly. (Both of these services offer a DDG search as fallback. Note that DDG is an external service included herein by linking, under their request to spread it, and not affiliated with The MirOS Project. They do, however, donate some advertising money to Debian.)
For all those who didn’t know: only manpages for software in the MirOS BSD base system and for the MirPorts Framework package tools are listed, not for third-party applications installable using ports or, recently, pkgsrc®. Still, if you want to have a peek at a modern classic BSD’s documentation, you’re welcome. (Not to mention content like re_format(7) and style(9) and that some of our documentation is much more legible than others.)

And because writing all that perl(1) made me ill, not to mention I don’t even know that language, I’ve hacked a bit more in the mirmake(1) and mksh(1) parts of the MirWebsite, finally implementing pointing out where in the navigation sidebar the visitor currently is.

We also have exciting mksh porting news involving RT trying a larger number of ancient platforms than I dare count, me fixing bugs in Linux klibc and diving into other things, learning more about why I consider me lucky for hacking a BSD operating system… sorry, I want to keep this short as it’s mostly an announcement.

The MirWebsite source code is, of course, also available. Improvements welcome. Except for these three CGIs, our website is fully statically precompiled, and that’s a good thing. Please help in making the CGIs secure.

On MirBSD and other sane OSes, you can just press ^T (Ctrl-T) when dd(1) runs; this sends it a SIGINFO (cf. sigaction(2)) which asks it to display (progress) information to the tty. This includes kFreeBSD, btw.

Update 07.01.2012 – this also works on Hurd. Linux neither has SIGINFO nor (cooked mode tty) support for it.

There’s also pv:

	dd if=/dev/mapper/vg01-${customername}--hudson bs=1048576 | \
	    pv -pter -B 1048576 -s 85899345920 | \
	    xz -0 >/mnt/ci-${customername}-snap-20120105-lenny.img.xz

I used this At wOrk today to back up a Jenkins VM before upgrading its underlying operating system for evaluation. Here, the -s flag is the total size (in bytes; don’t forget to multiply by 1024 when reading from Linux’ /proc/partitions) so pv can calculate a total and an ETA; -B is the same as bs; and xz is the currently best compressor to use, in any situation, unless you must stay compatible to gzip(1)-only systems. (Except that it’s not under an Open Source licence.)

clpbar might also be worth looking into. XTaran points out sid has this as bar.

PSA: Last of June, 2012, will be a leap second.

As reported earlier (in “Jenkins und die APT-Repositories” part 1) we’ve got some kind of Jenkins/APT integration, with automatically generating as many repositories as a job desires.

News are that the builds host has moved, so the URIs to the repositories have changed. The new syntax is for the Jenkins ⇒ “deb distribution suite …” and currently only usable in the company-internal network.

We’ve also got some more magic mksh code to automate the entire process – check the code out from SCM (required, as Jenkins’ svn checkouts are broken), build a Debian source package, NEW! ask cowbuilder to compile it in a clean chroot environment, and call for APT repository publication. Sample projects are ci-evolvis/virtualscreen (git) and ci-dev/portal-setup (svn). Talk to me if you have any questions.

This allows for a one-line “run a shell command” build step!

Everybody else is, of course, invited to take and re-use our code, and maybe even improve upon it and submit that back. It’s all Open Source, after all.

tl;dr Jenkins Jobs now have integration with cowbuilder. There’s a new script to automate the whole build pipeline. The APT repositories have moved with the recent move.

This is both a release announcement for the next installment of The MirBSD Korn Shell, mksh R40b, and a follow-up to Sune’s article about small tools of various degrees of usefulness.

I hope I don’t need to say too much about the first part; mksh(1) is packaged in a gazillion of operating environments (dear Planet readers, that of course includes Debian, which occasionally gets a development snapshot; I’ll wait uploading R40c until that two month fixed gcc bug will finally find its way into the packages for armel and armhf). Ah, we’re getting Arch Linux (after years) to include mksh now. (Probably because they couldn’t stand the teasing that Arch Hurd included it one day after having been told about its existence, wondering why it built without needing patches on Hurd…) MSYS is a supposedly supported target now, people are working on WinAPI and DJGPP in their spare time, and Cygwin and Debian packagers have deprecated pdksh in favour of mksh (thanks!). So, everything looking well on that front.

I’ve started a collection of shell snippets some time ago, where most of “those small things” of mine ends up. Even stuff I write at work – we’re an Open Source company and can generally publish under (currently) AGPLv3 or (if extending existing code) that code’s licence. I chose git as SCM in that FusionForge instance so that people would hopefully use it and contribute to it without fear, as it’s hosted on my current money source’s servers. (Can just clone it.) Feel free to register and ask for membership, to extend it (only if your shell-fu is up to the task, KNOPPIX-style scripts would be a bad style(9) example as the primary goal of the project is to give good examples to people who learn shell coding by looking at other peoples’ code).

Maybe you like my editor, too? At OpenRheinRuhr, the Atari people sure liked it as it uses WordStar® like key combinations, standardised across a lot of platforms and vendors (DR DOS Editor, Turbo Pascal, Borland C++ for Windows, …)

ObPromise: a posting to raise the level of ferrophility on the Planet aggregators this wlog reaches (got pix)

benz’ wedding, fun before

24.10.2011 by tg@
Tags: debian event fun

My dear MirBSD co-developer Benny did not only get his Doctor title but also recently married. There will be another post detailing this, including better photos of the two Doctors and the cake (with a Dæmon she made herself) on the wlog, but this is some fun beforehand:

No GPL cars!

Apparently, it is forbidden in France to drive GPL cars. (Without safety valve – but you have to admit the picture was fun. And we were like WTF? since the thing actually meant is LPG in German. Just like UTC is CUT (Coordinated Universal Time) in English, TUC (Temps Universel Coordonné) in French…)

I’m also working on improving our xterm(1) and GNU screen config, and other things. Explaining acronyms on our webpages is also coming some time. Benny is importing weird stuff from TNF for better pkgsrc® support, so there is activity. Just we’ve got dayjobs and a life… and mksh(1) still rocks (pdksh got orphaned in Debian today).


06.10.2011 by tg@
Tags: debian pcli rant tip

Would MTAs please stop sending hi-bit7 messages to other MTAs which do not advertise 8BITMIME! Recode it to QP or BASE64, damnit! The receiving MTA is entitled to strip the set bit7, which kinda makes things hard to read (while I know how to deal with blvde Stra_e, the advent of UTF-8 makes that blC6de StraC?e, introduces C0 control characters and makes typographic quotation marks into NUL-containing octet sequences (as their UTF-8 representation contains 0x80 octets) which let every sensible MDA terminate the line there). I even filed in the Debian BTS against the BTS (might be Drexim's fault, though).

Would MUAs please default to Quoted-Printable!

And mail hosters should use the same server when retrying delivery, to benefit greylisting. Or at least publish a list of outgoing IPv4 addresses they use for sending. Or use IPv6. Oh, and STARTTLS, while we are on my wishlist.

It's a sad day when the percentage of correctly encoded eMail messages in my INBOX is smaller than that of my Spambox...

Improvements welcome

30.09.2011 by tg@
Tags: debian tip

No I don’t really know any SQL. In fact, even at vocational school, where we focussed on database normalisation anyway, I tried hard to avoid the topic. Feel free to access here my entire knowledge about SQL ☺ (I did use Amaya, Arena and Arachne though. Liked only Arachne out of these three, and then, only under DOS, not its Unix version. Maybe the WWW could be named AAA instead? But then, lynx(1) is the one true browser…)

Ah, well. While at it… the entirety of my Perl knowledge is here: perltoc(1) with quick links to perlfunc(1).

The entirety of my (X)HTML and ECMAscript knowledge, DE: SELFHTML; although, the spec and DTD helped; and to write my notes on JSON, I took a peek at the formal ECMAscript spec as well… à propos, does anyone know a (good enough) indent(1) equivalent for ECMAscript, as I am trying to strip down some, inherited (GPL, yes) code for a hobby project, but Geo-people seem to produce illegible code?

Our MirBSD online manual pages and other assorted BSD documentation (except of course the merely copied ncurses, lynx etc. documentation and the texinfo generated HTML pages) has just gained a major facelift. They look alike in lynx(1) – best web browser ever – and less(1)/man(1) now, and remind of a DEC VT420 on a CSS capable Buntbrause.

Thanks to our contributor XTaran for aid with the colour scheme!

Since these are generated from catmanpages, heuristics are used for things like where should bold/underline begin/end (since nroff(1) is not always the brightest… but working on that), and hyperlinks can only be generated for other manpage references (whose targets may or may not exist, for example if they aren’t part of MirOS base/XFree86®). But on the other hand, Valid XHTML/1.1 and CSS speaks for itself ☻☺

Cat weather

04.09.2011 by tg@
Tags: debian fun

Another cat posting, about 100 KiB worth of images embedded so follow to the main article to read it, I don’t want Planet readers to suffer from traffic overuse.

Hot and humid (it’s rained a bit overnight, but has almost dried up quickly) seems to be cat weather. I went to buy breakfast at the local bakery when three cats lay around the house door in a half circle – my two black friends from the last posting and their human can opener’s third owner. When I came back I wondered whether the small guy wanted to travel:
cat car
what could that human possibly want from me? (focus follows eyes)

The big guy has hidden indoors, but needed only very little coaxing to head back outside in a measured speed:
can you see me?

The car’s owner arrived when I closed the door behind that cat, and not only did the little guy jump off… but also did the third cat… get out from under the car. Huh…

As written about here earlier, cats have a nice life. I walked into my home seeing three cats in a row, all black: two lazing around, the third (with white spots, and belonging to a different neighbour from the other two) ambulating. I went up and got my PocketPC with the already mentioned camera application to take a shoot. Sadly, the more shy cat went away, but I got some pictures of the other two – here they are, internet photo stars ☺ follow the hyperlink to get a large version.

two of three black cats I met today… in a row!

Later I came back from geocaching (2 GC.COM-only, 1 OC-only *yay!* found, one not found due to not taking any hardware with me) the bigger guy lazed around in the bush next to where I usually park my bike. Lucky…

Oh well – someone came into the #cvs channel on IRC without a clue, again. I’ve made a nice picture to show “the competition” (rival, whatever) to newbies (warning, sarcasm ahead)…

Subversion (svn, Suckwürstchen) for Dummies – or #cvs
 channel visitors • branches tags trunk


But trying to “cvs co” a websvn repository view… honestly!

Yes, I’m biased. And known to be proud of the things I use.

On Day 0, we were at my favourite Jugoslawian restaurant, and during eating and verpeiling, Andi took some pictures:
Jana und Jupp “ich habe die Macht” cnuke@ Henni und ciruZ (Jonathan) gecko2@ “geh weg” und bsiegert@ “waaah!” deer in the headlights
Take special note of the fun expressions everyone has…

Day 2, nothing of note at the conference itself – according to Jana, the only interesting talk (that tcpdump(8) GUI) was cancelled, and everything else was PHP and Web 2.0 crap. The food also was different, at least what I got, from Day 1. But it wasn’t as hot as on the previous day, and we did more socialising. I also managed to get the MirBSD ISO distributed some more.

Then I took my fellow DDs Enrico and madamezou geocaching for their first time, together with benz; they then took a Travelbug I found on Day 1 (with rsc) to Italy so it’ll end up in Rome, a next step on its mission.

Other rarely-seen people, such as Dr. Pfeffer, made an appearance, but overall the second day was quite relaxed. Ah, and Benny is a Doctor in Germany now as well.

On Monday, I slept quite a bit ☺

	14:31⎜*<* Signoff: XTaran (*.net *.split)

… doesn’t prevent me from telling him…

	14:39⎜<mira|AO> XTaran: n̲i̲e̲ n̲i̲e̲ n̲i̲e̲ n̲i̲e̲ n̲i̲e̲ n̲i̲e̲ n̲i̲e̲
	     ⎜empfiehlt man k̶i̶l̶l̶a̶l̶l̶, i̲m̲m̲e̲r̲ nur p̲k̲i̲l̲l̲!

“Now playing: Monzy — kill dash” ⇒ good idea… ☺

By the way, you were probably looking for this…

     -x      Require an exact match of the process name, or argument list if
             -f is given. The default is to match any substring.

… excerpt from the pkill(1) manual page, where you can see it stems from grep(1) clearly.

Yes, this website (and thus the RSS export) is Lynx on uxterm -fn -misc-fixed-medium-r-normal--18-120-100-100-c-90-iso10646-1 -fw -misc-fixed-medium-r-normal-ko-18-120-100-100-c-180-iso10646-1 on XFree86® optimised. Your browser might not do combining.

Built the ISO [torrent link deleted 2014-05-13] in the morning, today. Finally. Whew. It was much too warm in the mēnsa, and why did I have to get up so early anyway? Real Conferences™ don’t start before 10 o’clock, and there are no sensible activities before 11 o’clock anyway…

Talked to a lot of people, introduced my favourite Fedora Packager to Geocaching. Now my throat is sore and I’m tired. Social Event was not my case, as usual. (And even the vegetarian food now costs money as opposed to, I think, two years ago.) At least dry and not too loud. Still, best thing of FrOSCon is the Friday Evening Jugoslawian Food Mealtime ;-)

How not to create DEB files

18.08.2011 by tg@
Tags: debian

Once upon a time, there was Deb and Ian. That was about exactly 18 years ago. We don’t talk about the 0.939000 format any more, but they eventually settled on:

	$ ar rc pkg_1.0_all.deb debian-binary control.tar.gz data.tar.gz
	$ hexdump -C pkg_1.0_all.deb | head
	00000000  21 3c 61 72 63 68 3e 0a  64 65 62 69 61 6e 2d 62  |!<arch>.debian-b|
	00000010  69 6e 61 72 79 20 20 20  31 33 31 33 36 38 33 35  |inary   13136835|
	00000020  32 39 20 20 31 30 30 36  20 20 32 30 30 20 20 20  |29  1006  200   |
	00000030  31 30 30 36 34 34 20 20  34 20 20 20 20 20 20 20  |100644  4       |
	00000040  20 20 60 0a 32 2e 30 0a  63 6f 6e 74 72 6f 6c 2e  |  `.2.0.control.|
	00000050  74 61 72 2e 67 7a 20 20  31 33 31 33 36 38 33 35  |tar.gz  13136835|
	00000060  32 39 20 20 31 30 30 36  20 20 32 30 30 20 20 20  |29  1006  200   |
	00000070  31 30 30 36 34 34 20 20  31 33 39 31 20 20 20 20  |100644  1391    |
	00000080  20 20 60 0a 1f 8b 08 00  00 00 00 00 00 03 ed 59  |  `............Y|
	00000090  eb 6f db 36 10 f7 d7 f0  af b8 3a 5e 9b 74 b1 f5  |.o.6......:^.t..|

By then, systems were a.out(5), and everything was good. (Of course, if you look at the mtimes, you’ll notice I faked this. But it’s really equivalent to the real thing.

But oh horror! GNU binutils, not always everyone’s friend, switched from using BSD style “Unix Archiver” libraries in ar(1) to SYSV style libraries on elf(5) systems:

	$ ar rc on-elf debian-binary control.tar.gz data.tar.gz
	$ hexdump -C on-elf | head
	00000000  21 3c 61 72 63 68 3e 0a  64 65 62 69 61 6e 2d 62  |!<arch>.debian-b|
	00000010  69 6e 61 72 79 2f 20 20  31 33 31 33 36 38 33 35  |inary/  13136835|
	00000020  32 39 20 20 31 30 30 36  20 20 32 30 30 20 20 20  |29  1006  200   |
	00000030  31 30 30 36 34 34 20 20  34 20 20 20 20 20 20 20  |100644  4       |
	00000040  20 20 60 0a 32 2e 30 0a  63 6f 6e 74 72 6f 6c 2e  |  `.2.0.control.|
	00000050  74 61 72 2e 67 7a 2f 20  31 33 31 33 36 38 33 35  |tar.gz/ 13136835|
	00000060  32 39 20 20 31 30 30 36  20 20 32 30 30 20 20 20  |29  1006  200   |
	00000070  31 30 30 36 34 34 20 20  31 33 39 31 20 20 20 20  |100644  1391    |
	00000080  20 20 60 0a 1f 8b 08 00  00 00 00 00 00 03 ed 59  |  `............Y|
	00000090  eb 6f db 36 10 f7 d7 f0  af b8 3a 5e 9b 74 b1 f5  |.o.6......:^.t..|

Can you spot the difference?

Of course, ELF is what you want™, so there is little choice. Unix Archiver libraries are system dependent, and no format has ever been normed, but DEB files use it as format… so what is one to do?

	$ GNUTARGET=a.out-i386-linux ar rc with-aout \
	> debian-binary control.tar.gz data.tar.gz
	$ md5sum pkg_1.0_all.deb with-aout on-elf
	248f78d42f8ca8f2a3560f9800b2bf01  pkg_1.0_all.deb
	248f78d42f8ca8f2a3560f9800b2bf01  with-aout
	09eca70c9b11b6b55bbadcab5c3201fb  on-elf

“OK, and what do I do on my Debian/m68k system?”

ar(1) uses bfd, and GNU binutils can not only forcibly set the target emulation but also show them:

debian_m68k$ ar -h 2>&1 | grep '^ar: supported targets'
ar: supported targets: elf32-m68k a.out-m68k-linux elf32-little elf32-big plugin srec symbolsrec verilog tekhex binary ihex trad-core
debian_i386$ ar -h 2>&1 | grep '^ar: supported targets'
ar: supported targets: elf32-i386 a.out-i386-linux pei-i386 elf32-little elf32-big elf64-x86-64 elf32-x86-64 pei-x86-64 elf64-l1om elf64-k1om elf64-little elf64-big plugin srec symbolsrec verilog tekhex binary ihex trad-core
debian_i386$ ar -h 2>&1 | grep '^ar: supported targets' # binutils-multiarch
ar: supported targets: elf32-i386 a.out-i386-linux pei-i386 elf32-little elf32-big elf64-alpha ecoff-littlealpha elf64-little elf64-big elf32-littlearm elf32-bigarm elf32-hppa-linux elf32-hppa elf64-x86-64 elf32-x86-64 elf64-l1om elf64-k1om elf64-ia64-little elf64-ia64-big pei-ia64 elf32-m68k a.out-m68k-linux coff-m68k versados ieee a.out-zero-big elf32-tradbigmips elf32-tradlittlemips ecoff-bigmips ecoff-littlemips elf32-ntradbigmips elf64-tradbigmips elf32-ntradlittlemips elf64-tradlittlemips elf32-powerpc aixcoff-rs6000 elf32-powerpcle ppcboot elf64-powerpc elf64-powerpcle aixcoff64-rs6000 aix5coff64-rs6000 elf32-s390 elf64-s390 elf32-shbig-linux elf32-sh-linux elf32-sh64-linux elf32-sh64big-linux elf64-sh64-linux elf64-sh64big-linux elf32-sparc a.out-sparc-linux elf64-sparc a.out-sunos-big pei-x86-64 elf32-m32r-linux elf32-m32rle-linux elf32-spu plugin srec symbolsrec verilog tekhex binary ihex trad-core
mirbsd_i386$ ar -h 2>&1 | grep '^ar: supported targets'
ar: supported targets: elf32-i386 coff-a29k-big a.out.adobe aix5coff64-rs6000 a.out-zero-big a.out-mips-little epoc-pe-arm-big epoc-pe-arm-little epoc-pei-arm-big epoc-pei-arm-little coff-arm-big coff-arm-little a.out-arm-netbsd pe-arm-big pe-arm-little pei-arm-big pei-arm-little b.out.big b.out.little efi-app-ia32 efi-app-ia64 elf32-avr elf32-big elf32-bigarc elf32-bigarm elf32-bigarm-symbian elf32-bigarm-vxworks elf32-bigmips elf32-cr16c elf32-cris elf32-crx elf32-d10v elf32-d30v elf32-dlx elf32-fr30 elf32-frv elf32-frvfdpic elf32-h8300 elf32-hppa-linux elf32-hppa-netbsd elf32-hppa elf32-i370 elf32-i386-freebsd elf32-i386-vxworks elf32-i860-little elf32-i860 elf32-i960 elf32-ia64-hpux-big elf32-ip2k elf32-iq2000 elf32-little elf32-littlearc elf32-littlearm elf32-littlearm-symbian elf32-littlearm-vxworks elf32-littlemips elf32-m32r elf32-m32rle elf32-m32r-linux elf32-m32rle-linux elf32-m68hc11 elf32-m68hc12 elf32-m68k elf32-m88k elf32-mcore-big elf32-mcore-little elf32-mn10200 elf32-mn10300 elf32-msp430 elf32-nbigmips elf32-nlittlemips elf32-ntradbigmips elf32-ntradlittlemips elf32-openrisc elf32-or32 elf32-pj elf32-pjl elf32-powerpc elf32-powerpc-vxworks elf32-powerpcle elf32-s390 elf32-sh elf32-shbig-linux elf32-shl elf32-shl-symbian elf32-sh-linux elf32-shl-nbsd elf32-sh-nbsd elf32-sh64 elf32-sh64l elf32-sh64l-nbsd elf32-sh64-nbsd elf32-sh64-linux elf32-sh64big-linux elf32-sparc elf32-tradbigmips elf32-tradlittlemips elf32-us-cris elf32-v850 elf32-vax elf32-xstormy16 elf32-xtensa-be elf32-xtensa-le elf64-alpha-freebsd elf64-alpha elf64-big elf64-bigmips elf64-hppa-linux elf64-hppa elf64-ia64-big elf64-ia64-hpux-big elf64-ia64-little elf64-little elf64-littlemips elf64-mmix elf64-powerpc elf64-powerpcle elf64-s390 elf64-sh64 elf64-sh64l elf64-sh64l-nbsd elf64-sh64-nbsd elf64-sh64-linux elf64-sh64big-linux elf64-sparc elf64-tradbigmips elf64-tradlittlemips elf64-x86-64 mmo pe-powerpc pei-powerpc pe-powerpcle pei-powerpcle a.out-cris demo64 ecoff-bigmips ecoff-biglittlemips ecoff-littlemips ecoff-littlealpha coff-go32 coff-go32-exe coff-h8300 coff-h8500 a.out-hp300hpux a.out-i386 a.out-i386-bsd coff-i386 a.out-i386-freebsd a.out-i386-lynx coff-i386-lynx msdos a.out-i386-netbsd i386os9k pe-i386 pei-i386 coff-i860 coff-Intel-big coff-Intel-little ieee coff-m68k coff-m68k-un a.out-m68k-lynx coff-m68k-lynx a.out-m68k-netbsd coff-m68k-sysv coff-m88kbcs a.out-m88k-mach3 a.out-m88k-openbsd mach-o-be mach-o-le mach-o-fat coff-maxq pe-mcore-big pe-mcore-little pei-mcore-big pei-mcore-little pe-mips pei-mips a.out-newsos3 nlm32-alpha nlm32-i386 nlm32-powerpc nlm32-sparc coff-or32-big a.out-pc532-mach a.out-ns32k-netbsd a.out-pdp11 pef pef-xlib ppcboot aixcoff64-rs6000 aixcoff-rs6000 coff-sh-small coff-sh coff-shl-small coff-shl pe-shl pei-shl coff-sparc a.out-sparc-little a.out-sparc-linux a.out-sparc-lynx coff-sparc-lynx a.out-sparc-netbsd a.out-sunos-big sym a.out-tic30 coff-tic30 coff0-beh-c54x coff0-c54x coff1-beh-c54x coff1-c54x coff2-beh-c54x coff2-c54x coff-tic80 a.out-vax-bsd a.out-vax-netbsd a.out-vax1k-netbsd versados vms-alpha vms-vax coff-w65 coff-we32k coff-z8k elf32-am33lin elf32-ms1 srec symbolsrec tekhex binary ihex netbsd-core

Wow. While binutils share no single supported working target, they can be built multiarch, or (on MirBSD) with --enable-targets=all --enable-64-bit-bfd. Doesn’t help if you want to stay portable: GNUTARGET=srec is common on all Debian (sid) binutils versions (single or multiarch), but errors out on older binutils. The a.out-* targets are not common. Sure, you could hack around things, but… this is tedious. If you follow things or know me a little, you might already have guessed that I wouldn’t let that stand.

pax(1) to the rescue. On MirBSD, we use paxtar, which has cpio(1) and tar(1) front-ends and supports multiple formats (4 cpio and 2 tar variants) and has already been extended a lot and is lovingly called paxmirabilis (mirabilos’ peace in Latin) – it has options to anonymise archives: set uid and gid to zero, set mtime to zero, (for ustar) only write the numeric uid and gid to the archive, (for cpio formats) serialise inodes and device information, write content of hardlinked files only once (breaks partial extraction but saves a lot of space, e.g. 2 MiB off the Grml initrd.gz). And, recently, the ability to append a trailing slash to pathnames of ustar members which are directories (GNU tar does it – and I thought some Debian utilities check for it). So why not… (the -M dist and fakeroot set the uid/gid to 0)

	$ find debian-binary control.tar.gz data.tar.gz | \
	> mircpio -oHar -Mdist >with-mircpio
	$ mirpax -w -M dist -f with-mirpax -x ar \
	> debian-binary control.tar.gz data.tar.gz
	$ mirtar -M dist -A -cf with-mirtar \
	> debian-binary control.tar.gz data.tar.gz
	$ GNUTARGET=a.out-i386-linux fakeroot ar rc with-aout-ar \
	> debian-binary control.tar.gz data.tar.gz
	$ md5sum with-*
	a466e2fd57cdee141fe585a43245548f  with-aout-ar
	a466e2fd57cdee141fe585a43245548f  with-mircpio
	a466e2fd57cdee141fe585a43245548f  with-mirpax
	a466e2fd57cdee141fe585a43245548f  with-mirtar

Voilà. I got it, and even appending is possible. It supports the BSD format with special focus on DEB files, and deals with long filenames, but not symbol or filename tables (used by ranlib(1) or strange formats, respectively, but since we don’t create *.a files to use with some native linker/binder/loader, we don’t need that anyway).

On extraction (oh, and listing!) it deals with SYSV style filenames as well.

	$ mirtar tvf on-elf
	-rw-r--r--  1 tg       tg          4 Aug 18 16:05 debian-binary
	-rw-r--r--  1 tg       tg       1391 Aug 18 16:05 control.tar.gz
	-rw-r--r--  1 tg       tg      18135 Aug 18 16:05 data.tar.gz
	$ mirtar tvf with-aout
	-rw-r--r--  1 tg       tg          4 Aug 18 16:05 debian-binary
	-rw-r--r--  1 tg       tg       1391 Aug 18 16:05 control.tar.gz
	-rw-r--r--  1 tg       tg      18135 Aug 18 16:05 data.tar.gz

One of the real benefits is that you can use the front-ends interchangably – for example, “mirtar tzf foo.cpio.gz” would work (which GNU tar can’t do), and mircpio’s ustar implementation, unlike GNU cpio’s, is not horribly broken.

Of course, there are some drawbacks: it’s not GNU tar or GNU cpio, so there are absolutely zero --long-options. Some of their features are missing (but tar’s -O is implemented now), so it’s no replacement (but very well usable alongside it). The format called pax, committee-designed to replace ustar, isn’t yet supported ironically, but that’s on the TODO.

So, what do you think?

	tg@frozenfish:~/Debs/dists/sid/wtf/Pkgs/mircpio $ ll *.deb
	-rw-r--r-- 2 tg freewrt 78140 Aug 17 11:04 mircpio_20110817-0wtf2_amd64.deb
	-rw-r--r-- 3 tg freewrt 72262 Aug 17 11:00 mircpio_20110817-0wtf2_i386.deb
	-rw-r--r-- 1 tg freewrt 67446 Aug 17 18:21 mircpio_20110817-0wtf2_m68k.deb

Should I upload this to Debian proper?

As for the licence: 3-clause UCB (and 2-clause BSD, which is a subset of it), so no problem. I’m asking because the other package which I had been using for a long time and not uploaded, jupp, got uploaded recently (during DebConf) on user input (people wondered why it did not yet exist in Debian proper). I guess the old saying “if it’s not in Debian, it doesn’t exist” holds true in many parts of the OSS world.

It’s up to date wrt. standards btw, and lintian-clean save for two pedantic-class warnings (no upstream changelog file, no homepage link) which aren’t fulfillable (could link this wlog entry as homepage).

If you know Alioth you’re familiar with the software formerly known as SourceForge, formerly known as GForge, currently known as FusionForge. My employer both uses it and contributes to it, we run an adapted (mostly themed, prototyping new functions that often end up in FusionForge itself, and backporting functions from FF to our “production codebase”) version.

I’ve backported the extratabs plugin to appease project managers and other non-technical people while we move our codebase to FF 5.1, and I did so on an installed version of the plugin rather than the source because the latter was tightly integrated with rather heavy packaging style changes.

	# create fusionforge-plugin-extratabs binary package
	toplev=$$(pwd); cd plugins/fusionforge-plugin-extratabs; \
	p=$$(print -r -- $$(sed -n '/^Package: /s///p' C/control | head -1)); \
	v=$$(print -r -- $$(sed -n '/^Version: /s///p' C/control | head -1)); \
	a=$$(print -r -- $$(sed -n '/^Architecture: /s///p' C/control | head -1)); \
	d=$${p}_$${v}_$${a}.deb; \
	rm -f $$toplev/../$$d control.tar.gz data.tar.gz; \
	(cd control; find . | fgrep -v /.svn | sort | \
	    mircpio -oC512 -Hustar -M0x0B -Mgslash) | gzip -n9 >control.tar.gz; \
	(cd data; find . | fgrep -v /.svn | sort | \
	    mircpio -oC512 -Hustar -M0x0B -Mgslash) | gzip -n9 >data.tar.gz; \
	mirtar -M dist -Acf $$toplev/../$$d debian-binary cont*gz dat*gz; \
	rm -f control.tar.gz data.tar.gz; \
	cd $$toplev; dpkg-distaddfile $$d non-free/devel optional

The hardest part of extending debian/rules with that was to get the autobuild and dpkg-distaddfile call right. This works, even though I’d call it a temporary kludge. (No need to tell me I should have used && – I know. And I only shell out to mksh(1) because the “inner” part was already there from before, when I still used ar(1). This was slightly edited for the wlog.)

In the meanwhile, apt-extracttemplates can deal with SYSV style filenames in DEB files – on Debian sid, but not on K?buntu hardy, which some people are using as Desktop OS still…

Update 03.03.2012 – Jonathan Nieder replied quickly with a suggestion to instead take over the “pax” package in Debian. Eventually, I uploaded pax (1:20120211-1) from the former “mircpio” package to Debian, after I managed to talk to its previous maintainer Bdale Garbee (thanks for handing over). It is now present in Debian wheezy and Zubunt! precise as /bin/pax with /bin/paxcpio and /bin/paxtar offering the other interfaces.

FrOSCon 2011

18.08.2011 by tg@
Tags: debian event grml news

This year without our friends from Grml, but The MirOS Project (all two active developers and our Booth Babe gecko2@) will of course attend FrOSCon, nicknamed Froschkon, again.

We’ll have a pre-event meal time at my favourite Jugoslawian Restaurant on Friday (20:00 CEST) – contact me privately for the coördinates if interested. On Saturday and Sunday we’ll staff a booth and answer questions about the many projects we have (more or less) running, including but not limited to paxmirabilis (aka MirCPIO), The MirBSD Korn Shell aka mksh(1), jupp the editor, and developers’ private projects such as slowly undermining Debian or Google-Go. While slow we are still working on World Domination. And teaching people good shell programming by example code.

We might even bring CDs, but I’m still working on the ISO… last night’s build aborted because the OS grew a bit making the floppy image not fit any more. (Solution, drop ping(8) and rtsol(8), but re-add sf(4) and bce(4) now that they fit again.)

Not a good idea…

18.08.2011 by tg@
Tags: debian fun

Sometimes, when you develop WUIs (Web UI), you really have to test them against a variety of browsers, not all of which are available for the operating system installed on peoples’ desktop PCs, or working in Wine. (For theming QA, Wine is also a #FAIL, but for technical QA, MSIE 1.5, 3.0, 5.02, 5.5, 6.0 work fine, and MSIE 7.0 can be used under rare circumstances.) In these cases, you use VMs running certain operating systems. One VM had an interesting idea of which hardware you can “safely remove” a couple of days ago when I was hacking it anyway: Safely Remove… the RAM controller?

「??? ???」 • or • mira meets d-i

18.08.2011 by tg@
Tags: bug debian fun

(originally published on 2011-01-26, but reposting so the people on Plänet Debian can have some fun)

While helping a cow-orker setting up an encrypted hard disc (basically, putting / and swap into LVM inside cryptsetup, and /boot outside), mirabilos managed to discover an entirely new side of K?buntu 10.10 on his voyage…

… wo noch nie ein Debian Developer zuvor gewesen ist… oder?

(Only a reboot helped at that point. Earlier, the dialogue box was shown only once, but upon re-entry of the partitioning clickibunti d-i tool, neither button did anything save redrawing this… interesting, informative and intuitive error message.)

(Dieses Posting wurde ursprünglich im internen Blog veröffentlicht, ist jedoch hierhin umgezogen. Es sind keine sensitiven Informationen enthalten, allerdings können die Links nur im Firmennetz angesteuert werden.)

Neues aus der Adminstube (und vom Evolvis-Team) für unsere Entwickler, die wir, nachdem wir die Projektmanager schon bedient haben, auch nicht zu kurz kommen lassen wollen:

Wenn ihr einen Jenkins-Job habt, der Debian-Pakete (oder für Derivate) erstellt, könnt ihr ab sofort (mindestens) ein Repository pro Job haben, in dem ihr die Pakete „veröffentlichen“ könnt. Diese Repositories werden automatisch signiert, und durch tarent-keyring werden die Signaturen bereits seit Dienstag verteilt, sodaß auch SecureAPT funktioniert.

Im folgenden wird die Arbeitsweise am Beispiel von Saschas WIP-Portalinstaller erklärt.

Ausgangspunkt ist ein Jenkins-Job, in diesem Fall auf dem test-hudson, hier mit dem Namen „portal-setup“, der unter „Build“ den Punkt „Execute shell“ aktiv hat. Hier wird durch diverse Kommandos ein Debian-Paket (Source und Binary) gebaut, das wirklich wichtige hierbei ist folgender Ausschnitt:

	cd portal-setup-$pv
	dpkg-buildpackage -rfakeroot -us -uc
	cd ..
	#rm -rf portal-setup-$pv

Der Befehl dpkg-buildpackage übernimmt das Bauen und schmeißt die erstellten Pakete (Source *.dsc und Binaries *.deb) und, ganz wichtig, die *.changes-Datei, ins Elternverzeichnis (daher die cd-Aufrufe). Nun möchte Sascha, daß diese Pakete einfach von Kollegen getestet werden, und ändert daher den Codeschnipsel so ab:

	cd portal-setup-$pv
	dpkg-buildpackage -rfakeroot -us -uc
	cd ..
	#rm -rf portal-setup-$uv portal-setup-$pv

	/opt/mvn-debs/ portal-setup squeeze main *.changes

Im ersten Teil bleibt alles beim alten, aber ein neuer Befehl ist am Schluß hinzugekommen. Was macht der?

Nun, /opt/mvn-debs/ ist die Magie, die unterhalb des Pfades /opt/mvn-debs ein Debian-Repository mit dem Jobnamen portal-setup anlegt. Wir publizieren Pakete in die „dist“ squeeze, und darin in die „suite“ main. Das ist hier lediglich eine Konvention; normalerweise heißt die dist so wie die Debian-Version (sarge, dapper, etch, hardy, lenny, squeeze, …), für die das Paket gedacht ist, und die suite ist eine Unterkategorie – also zum Beispiel main/contrib/non-free oder main/restricted/universe/multiverse oder wtf/tarent/evolvis (in unserem Adminrepo). Man kann die aber auch frei Schnauze nennen (ich hab zum Beispiel im m68k-Repo eine dist namens „cross“, unterhalb derer eine Cross-Toolchain liegt).

Das letzte Argument ist *.changes, was die Shell für uns expandiert: alle Dateien, die auf „.changes“ enden, werden dort (ASCIIbetisch sortiert) der Reihe nach aufgelistet.

Das Skript prüft nun zunächst die Namen und Dateien auf Plausibilität (es sind schließlich immer nur gewisse Zeichen erlaubt) und schiebt dann vermittels dput ein Release (also eine *.changes + alle drin enthaltenen *.deb + dazugehörige *.dsc, *.diff.gz/*.debian.tgz, *.orig.tar.gz, oder *.tar.gz bei native packages) ins APT-Repository und läßt danach den Index regenerieren und PGP-signieren. dput ist ein schlaues Tool, es schreibt nämlich nach getaner Arbeit eine *.upload-Datei, in welcher dieser Fakt verzeichnet wird, sodaß Pakete auch wenn man den workspace nicht jedes Mal wegschmeißt nur einmalig hochgeladen werden. (Es existiert allerdings hier kein Schutz davor, den workspace zu leeren und ein Paket mit derselben Version aber anderem Inhalt ein zweites Mal hochzuladen. Nur die Systeme, die das installieren wollen, mögen einen dann ggf. nicht mehr so gerne.) ruft nun also auf, was allerdings mehr macht als nur das Äquivalent von Debians dak oder CentOS’ createrepo. Es erstellt nämlich auch einen Index. heißt der Gute, und ist eine Auflistung aller Pakete nach dist, suite, Source und Binary im Repository. Weiterhin steht oben nochmal, um welches Repository es sich handelt, und welche dists und suites verfügbar sind. Der Name bildet sich aus „http://“ + Jenkins-Systemname + „“ + Jenkins-Jobname + „/debidx.htm“ – wenn man den Dateinamen wegläßt kann man das auch direkt als Verzeichnisstruktur anschauen.

Der Clou: die passende sources.list-Zeile steht auch schon da! (Allerdings muß man die suiten ggf. auf die, die wirklich benötigt werden, reduzieren.)

deb squeeze main

Das funktioniert auf allen acht Jenkins-Systemen, allerdings natürlich nur aus dem tarent-Netz heraus – dafür ohne https oder aufwendige Authentifizierung. Einfach so.

Bitte achtet darauf, den Namen des Jenkins-Jobs als erstes Argument zu zu verwenden, ggf. gefolgt von einer Kennzeichnung, falls ihr mehr als ein Repo pro Job braucht (warum, weiß ich nicht, aber es geht). Da alles als maven-User läuft findet keine Abgrenzung statt.

Wenn ihr Pakete aufräumen möchtet, so loggt euch (als User maven) auf dem entsprechenden Jenkins ein und schaut auf der Verzeichnisebene nach; einen direkten Link gibt’s auch, wenn man das [dir] im tabellarischen Index anklickt, zum Beispiel, was im Dateisystem dem Verzeichnis /opt/mvn-debs/portal-setup/dists/squeeze/main/Pkgs/portal-setup/ entspricht.

Nachdem ihr händisch Änderungen dort vorgenommen habt, müßt ihr natürlich den Index neu generieren lassen, das geht dann so:

mksh /opt/mvn-debs/ /opt/mvn-debs portal-setup squeeze

Wenn ihr nicht nur eine dist angefaßt habt, könnt ihr die auch auflisten:

mksh /opt/mvn-debs/ /opt/mvn-debs portal-setup hardy squeeze

Oder einfach weglassen, dann macht er alle:

mksh /opt/mvn-debs/ /opt/mvn-debs portal-setup

Das ist dann auch der Unterschied zwischen dists und suites: letztere teilen sich ein Release-File, erstere nicht, nur den XHTML-Index.

So, ich hoffe, ihr findet das so nützlich wie Sascha und so arbeits- und zeitersparend wie ich ☺

PS: Der Code ist mittlerweile auch publiziert. Wer solch ein Szenario noch woanders aufsetzen möchte ist uns herzlich willkommen.

The pictures are hypertext references to large versions. Of course, your photographer (me, although Samuel helped to set up the PocketPC’s camera application correctly, 10x) also had some Kruškovac ☺ (imported from Croatia into Bosnia)…

spontaneous late night meeting at Front Desk

Of course we were not above closing Front Desk either ☻☺

Best Friends

jupp 3.1.17 uploaded today, mostly thanks to user input suggesting I improve things, especially the syntax highlighting. (Maybe more to come.) I like users who don’t complain but give helpful comments and send in patches even.

Since the Debian FTP masters complain that the NEW queue is empty for the first time in ages, I also uploaded jupp to Debian proper (got requests, several, from actual users – independent of each other). I originally thought I were the only user, it’s not worth it, maybe too close to joe (which segfaults a lot more and has some ugly things, so I cherry-picked the better features of it instead of rebasing jupp), but it’s had a package in mports (MidnightBSD ports) for ages, users submitted one to FreeBSD® last year and keep it updated, there’s even a WIP package in pkgsrc®, and who knows where else or how many people are using my OpenSuSE Buildservice package or have had installed the previous DEB package I uploaded to my play repo. So now I feel it worth to upload.

I even invested some major packaging rework, such as splitting the build-arch and build-indep parts from each other, and importing the upstream source into the packaging VCS, as I have learned in the “packaging with git” talk here at DebConf. (No guys, I will stick to CVS as git doesn’t give me anything.)

Been hot and dry today (although the sky is now back full of dark clouds), so I had a headache most of the morning until way past noon. Better now though, and I found a place where I could get Cevapi, which are really some sort of quick imbiss / fast food here (no Đuveč pirinač though, and she didn’t have any Ajvar nor did she speak any language other than the local, but that wasn’t a problem, only a bit dry because I didn’t give in and took the offered Ketchup). Bought a 1ℓ bottle of Kruškovac (from Hrvatska, though) and some small plastic glasses, then.

I wonder how many people would, now, be willing to give Bosna i Hercegovina a try as holiday region (which might have been the intent of having a Balkan DebConf). I’m sure I do.

To all attendees: the hotel will give you some kind of stamped hardpaper card which states where you stayed on the trip, and for how long – give that to the border guards when exiting Bosnia.


25.07.2011 by tg@

Sitting in Бања Лука, Република Српска, Босна и Херцеговина (Banja Luka, Republika Srpska, Bosna i Hercegovina) let’s just say the country is pretty nice. People are okay, the beer is not called “Nektar” by accident, and the Mark (subunit Fennig, funnily enough) is worth 1 DM. Price niveau is below Germany (even when we had the DM) in some things, below or at modern European in others. In short, very affordable. They don’t accept paper money though, it’s really hard to get coins in most places, and they only want those. The food is okay, and my hotel is very luxurious. It’s also got LAN.

The weather is not so nice at the moment though: raining a lot, and expecting 30°C too-hot sun in two days. And there are still no Geocaches in the area.

Anyway, DebConf is going on, I’m acclimating and trying to get people, faces, nicknames and realnames connected. And accents. (And pronunciation of names – for example, Ian differs totally from what I’d use.) We even have working wire network (LAN) most of the time ;-)

We’re indeed still working on resurrecting m68k, but that’s no news. More on that later, I’d say.

mksh R40b (nowadays with filled in user’s caveats (for R40, too!) and packager’s upgrade hints) has just been released. This is a should-have upgrade, fixing a number of – admittedly some obscure – bugs, changing things begun in R40, improving upon others. Thanks to the PLD Linux guys for spotting all these errors; thanks to them and both for adopting mksh so well.

I have also fixed a bug in nroff(1) which will lead to an even nicer looking HTML manpage mksh(1) (after the next rebuild and upload of a MirBSD snapshot – scheduled RSN).

jupp 3.1.16 took on the task of merging Debian joe changes (aiming at an upload). I also split the jupprc file into three versions (2.8 generic/DOS, 3.1+jupp and 3.7/Unix) because of the differences in the baseline executables making rc files partially mutually incompatible (think Insert key), annoyingly warning (think syntax, hmsg), or less usable (joe’s new menu system).

jupp 2.8.2 is a companion to jupp 3.1.16 – mostly because of the new help window “character map” ☺

Binaries for jupp should be updated RSN too.

Considering Banja Luka is arriving quickly, the “r” in RSN should be taken with a few grains of salt. I’ve also scheduled working on the pcc Debian package for the next future; updating lynx and maybe others like OpenSSH in MirBSD is also due; cvs(1) will receive more of my time, but before the next Upload I’d like to fix LP#12230 once verified.

Builds for Debian/m68k are also still running. I note I did in fact not manage to make a new base image, yet (but 2.6.39 kernels miss a patch, anyway, so waiting for 3.0 is ok). It’s still using gcc-4.4 because nobody tests gcc-4.6 and gcj-4.6 FTBFS due to SIGSEGV, but that’s ok in my books. rsyslog is broken but sysklogd works.

The #ksh|Freenode page finally got a well-deserved link to Planet Commandline. Throw more my way!

Acronyms and translations, too. (Got Norwegian and Rumanian covered in the meantime. No idea whether any RTL languages will work in that beast. But I’m young and need the money)

Since I’m writing a wlog entry anyway… let me thank Gunnar for a nice summary on the current Free Culture discussion; my comments on Nina’s site seem to be eaten, but let me support it fully, although, of course, I normally use a copycenter style licence, which is specifically written for general works of authorship under copyright law, not limited to software. I did in fact have that in mind. Maybe some people will like it (it’s less than one Kibibyte long) either generally or just for their everyday random musings (they can then keep CC-BY-SA for the “big works” if they so desire).

Wouter, grass background makes green headlines illegible. I’ve never liked, and never installed manually, cups either. (Benny tells me that Apple’s new version refuses to talk with a non-Apple cups, kinda defeating the whole idea I think.) Port 9100 is JetDirect (probably with an HP in front and some subset of ©®™ trailing) and just nice. (Being able to talk ESC/P with your printer like print '\033K\x07\0\x3E\x81\x99\xA5\xA5\x81\x3E' >/dev/lpa too rocks though, IMHO. Yes, mine can, and I still can. /dev/lpa is BSD.)

Kai, thanks for your vimrc lines:

	:highlight TrailWhitespace ctermbg=red guibg=red
	:match TrailWhitespace /\s\+$\| \+\ze\t/

Automatic removal is harmful, though – I just fell into the trap since jupprc contains needed whitespace at EOL… but manual removal (bound to ^K] in jupp) rocks. And I like that your solution uses such strong a colour – vim users are the single most represented offender group for actually leaving the redundant whitespace at EOL there, and it should hurt their eyes. (Sadly there is some vehement disagreement preventing them from inclusion in grml-etc-core – but that’s why I re-post them here.) Ah, and jupp can of course display whitespace visibly (although it uses ‘·’/‘→’, replacing the arrow with ‘¬’ if no UTF-8, not ‘»’), accessible with ^Ov.

Steve, want to put up a checklist for sites? We can “crowdsource” the… testing… to maybe get some interesting results…

Some other people would get more comments if they were idling in IRC (Freenode) or allow comments on their blog, specifically without too high an entrance barrier – OpenID is ok, but many other things, and ECMAscript, are not; but I can’t really say that loud because our wlog is static HTML compiled from a flat plaintext data source so it doesn’t allow such either. I often forget what I wanted to add if I can’t get it out quickly enough (especially at work). Sowwy…

Me like the cat picture postings (Amayita, Tiago, ¡Gracias!).

New releases

11.07.2011 by tg@
Tags: debian mksh

You might have noticed the release of mksh R40 recently, after more than a year of development. Well, stay tuned for both R40b (with accumulated fixes) and R41 (intent to speed up array handling a lot and prepare for what we postponed to mksh R42 now – associative, multi-dimensional arrays).

You should also upgrade, if you have not yet done so, to kwalletcli 2.11.

Finally, jupp 3.1.15 was left out to the world, including Minix 3 users this time, by special request of one of these on our mailing list. In addition to the MidnightBSD mport – which has been there in like forever – and the MirPort and the FreeWRT package, in December 2011 a user submitted it to FreeBSD® ports, and Benny is going to add it to NetBSD® pkgsrc® soon, he said. (He also updated their mksh source package. Thanks!) I’ve been asked by two people, independent from each other, when I’ll upload it to Debian proper, instead of the private-repo packaging. Maybe I should indeed do that, comments?

  • √ Agreement to pay from company
  • √ Going to drive with some apparently speed-loving brits
  • √ Registration accepted
  • √ Dienstreiseantrag prepared
  • √ Sent that beast to the office ticket queue

So yes, this means I’m going to DebConf 11 to what used to be Yugoslawia when I was there the last time, although in the Poreč region of Istria, Hrvatska.

mksh R40 has been released Sunday 12nd June. So, what can you do with The MirBSD Korn Shell?

Have a look at the collection of shell snippets, which admittedly is only the beginning – most scripts don’t make use of the cool new features yet. But they eventually will. The Shell-Toolkit project is definitely worth a visit! (Admittedly not using my most ❦ dear SCM, but with a DVCS, every checkout is a clone, i.e. a backup, and none of the contributors must fear a central repo being taken down, which, for such a loose collection, is desirable.)

(First posting to Plänet Commandline! Tag: pcli)

Vutral asked in IRC how to synchronise two shells’ environment while they’re running. As you may know, POSIX systems cannot change a process’ environment vector after it has been started, only the process itself can. Well, the shell can, and we’ll use a variety of things for this.

This trick assumes you have $HISTFILE set to the same pathname in both shells (obviously, they run under the same user). It uses export -p to render the current list of exported variables, then transforms the list from newline-separated to a single big one-line export statement.
Then it transforms all remaining newlines (which will be part of a single-quoted string, since that’s mksh(1)’s export format) into the sequence '$'\n'' which means: terminate current single-quoted string, append $'\n' and open up a new single-quoted string immediately; concatenate these three.
Now, $'\n' is just a fancy way of saying newline, and part of mksh because David Korn (yes, the Korn in Korn Shell) strongly suggested to me that this functionality be included – but, as we can see here, it pays off.
Finally, the so transformed string is prepended by unset \$(export); which, when executed, will cause the shell to unset (and unexport) all currently exported variables. The shell parameters that are not exported, i.e. not in the environment, are not affected by this code (except for $x and $nl, but… whatever).
This string is then passed to read -s (plus -r and clearing IFS to enable raw mode), which means, read into the parameter $REPLY (which we conveniently don’t use – but it’s trashed too, thus) but store into history at the same time.

Ah hah! Now, the persistent history feature comes into effect! After running the below statement in the “source” shell, switch into the terminal running the “destination” shell, press Enter once on the empty line (Ctrl-U to empty it if it wasn’t), then Cursor-Up (↑) to recall… voilà, an insanely large line with the previously created string sorta expanded… and press Enter again to run it. Now your set of exported parameters is the exact same (minus if you exported IFS, nl, x or REPLY) as in the “source” shell.

I’ve added extra spaces and a linewrap below, this is really just one big line:

nl=$'\n'; x=$(export -p); x=${x//${nl}export/}; IFS= read -rs <<<"unset \\\$(export);${x//$nl/\'\$\'\\\\n\'\'}"

Of course, this makes a nice function, for your ~/.mkshrc or somesuch.

I was just tracking down why some mails seem to have garbled Subjects.

It looked like this in Alpine:
Subject: [BOFH commits] r2040: fix directory struct =?UTF-8?Q?ure=E2=86=B5=20unix?=/mirror/=?UTF-8?Q?=20=E2=86=92=20mirror?=. bonn

The raw header sight was this:
Subject: [BOFH commits] =?utf-8?q?r2040=3A__fix_directory_struct__=3D=3FUT?=

Ein Schelm, wer Böses dabei denkt… First suspect was, of course, Mailman – after decoding, this showed the classical signs of a double-encode. (After ruling out general header brokenness, but no, 76 chars is ok.) I thus hand-crafted an eMail with the correct header line and sent that out:
Subject: [BOFH commits] =?utf-8?q?r2040=3A_fix_directory_structure?=

Huh? Mailman and Python weren’t the culprit, thus – this is correct mangling. Okay, let’s dive into the Perl code that actually sends out the eMails. To make a long story short, have a look at this, then RFC 2047:
tglase@tglase:~ $ perl -MEncode -e '$subject = "r2040: fix directory structure↵ unix/mirror/ → mirror.bonn"; Encode::from_to($subject, "UTF-8", "MIME-Q"); print "{Subject: ".$subject."}\n";'
{Subject: r2040:=?UTF-8?Q?=20fix=20directory=20struct?=

Amazingly enough, PHP’s mb_encode_mimeheader, despite being talked to trash in the comments on its online documentation, does manage to get it right:
tglase@tglase:~ $ php -r 'mb_internal_encoding("UTF-8");echo "{".mb_encode_mimeheader("Subject: r2040: fix directory structure↵ unix/mirror/ → mirror.bonn", "UTF-8", "Q")."}\n";'
{Subject: r2040: fix directory =?UTF-8?Q?structure=E2=86=B5=20unix/mirror/?=

Wow. Now, the Perl guys I know told me to use Perl’s Mail tools… which are much too high-level though, for all I have and want is the subject string and an RFC 822 header line. I told them I’m not above doing this, and so I did. The 3P languages can really be annoying.

Why’s Perl’s output wrong anyway? I don’t know for sure, but I think the atoms must be separated, so unquoting /mirror/ in the middle, with no spaces around it, are wrong. (Besides, Encode::from_to can’t do the job right anyway, as it misses the name of the header, which is included in the 76 chars allowed for the first line. BAD • Broken As Desdigned.)

Disclaimer: I don’t really know any Perl, I fight my way through PHP and, barely, Python. (But I can code.)

tl;dr: Full JSON encoder/emitter and decoder/parser in Pure PHP now available as part of FusionForge/Evolvis under GPLv2. Do not use PHP’s built-in code, it’s broken. Read on for details and links.

I’ve pimped the minijson code in EvolvisForge to be a full-blown JSON encoder and decoder (lexer/parser) now. You wouldn’t believe just how much is broken in PHP (its own json_encode handles floats wrong in most locales, and mb_check_encoding doesn’t check the encoding of a multibyte string… at least, unlike Python, PHP manages to get 8bit okay though).

Anyway, thought you want to know. If you ever need a Pure PHP JSON encoder/decoder, you know where to look. It’s 100% my code (although written during dayjob, so the exploitation rights are with tarent GmbH). Current licence is GPLv2+ or AGPLv3+. If you spot any bugs, you know where to report them. I think it should be good though. (Do note that JSON is case-sensitive, so “NULL”, “True” and “\N” or “\U20AC” are not valid, “null”, “true”, “\n” and “\u20AC” or “\u20ac” are.)

I plan to store the “art_cust123” information in the user_prefs table in JSON now, instead of ‘|’-separated values, to avoid problems like these we had with broken database format and subsequent corruption of values, by using a dictionary (JSON Object) instead.

The ECMA 262 standard (ECMAscript and JSON) is freely available, though. JSON is also additionally specified in RFC 4627 which differs slightly (see my notes for details, mostly the goal element).

Update: fixed links

in-target: E: Kaputte Pakete

11.04.2011 by tg@
Tags: bug debian rant

*buntu Hardy kann zur Zeit nicht installiert werden (der Kernel (in main) dependet auf Pakete aus restrictet, das ist aber zum Installationszeitpunkt nicht aktiv und sowieso unfrei; und wieso ist eigentlich das hardy-updates Repo im d-i eingeschaltet und nicht erst hinterher?).

Lustiger aber: „Einige Pakete konnten nicht installiert werden. Das kann bedeuten, dass[sic!] Sie eine unmögliche Situation angefordert haben oder dass[sic!], wenn Sie die Unstable-Distribution verwenden, […]“

gecko2s Kommentar dazu nur, daß unstable bei *buntu stable heiße. Ich habs dann auf LTS korrigiert (ist nicht das erste Mal – und sowieso, wieso tauschen die in einer stabilen Version PostgreSQL-Majorversionen aus?) und dabei haben wir’s belassen: Debian unstable = *buntu LTS.

Naja, wie wir das letztens Simon gesagt haben (Upgrade innerhalb einer Version von *buntu auf einem Server hat grub durch grub2 ausgetauscht): Mit Debian wär’ das nicht passiert!

Various joys

26.03.2011 by tg@
Tags: debian

I’m online again. (In case you didn’t notice, duh…) Seems as if we (the Telco/ISP guy and me) just needed to look at it hard enough for it to go away – first he could dial in, using my account data, which I probably should change now, then herc with ppp(8) and pppoe(8) was working (although at about 50 KiB/s down, he showed me 508 KiB/s – a rate I had never achieved – with his WiXP), then I took my notebook, which worked with pppoe(4). Now herc’s working again. (Maybe altq(9) can explain the slowdown? Hm, from I get 500 so it looks okay.)
But eurynome isn’t, oh the joy. Luckily, gecko2 who administers its host system just woke up.

Things we do want to see: the Telco/ISP guy accepting that I run MirBSD on a P-233MMX box with Hercules graphics card and a 9″ monitor with no comment other than considering its age (and that it usually runs 24/7) as partial cause for the bug. Thanks, Netcologne!

Things we don’t want to see:
Mar 26 10:40:02 blau /bsd: signal 11 received by (screen:16857) UID(2999) EUID(2999), parent (screen:19111) UID(2999) EUID(2999)
“Suddenly the Dungeon collapses!! - You die...” (luckily, I get it about once a year only)

ObCoffeeSpices: Marrakech (Cumin, Allspice, Cumin Aroma) – though, due to its relative strength compared to the others, the only coffee spice I have left. And another hint: pre-warming the coffee cup with hot water, so it doesn’t cool down too fast with the amounts of milk I put in, rocks.

I just wore the Squeeze Release (FOSDEM, Spacefun) T-Shirt to the bakery and got asked by a neighbour: “Oh, a Debian fan?” “Developer, even” – now imagine the typical “informed interested guy” talk for a conference booth of your OS of choice here. How proud he was to get his wife and himself Windows®-free at home; how he likes to tinker a bit (if he’s got any time left), which has become harder with Windows; how his time constraints have him at OpenSuSE currently but asked how squeeze is; and the usual complaints at places like $ork where they have to use Windows® and MSIE (apparently you can’t centrally manage Firefox, eh, good someone tells me, because that’s what we do…). Wow. Anyway, it’s spring, so people, wear your shirts. (Hrm, what do I make of the fact that this is my only Debian shirt – although I’m thinking how to get Tartan Trousers if money were no issue – and nobody had ever commented on my various BSD, FOSDEM, FrOSCon, etc. wear…)

’M back.

21.03.2011 by tg@

Two DNF out of four geocaches, well… one was too muggled, the other was no longer there, judging from the previous visitors’ log entries. Cached with natureshadow and bought his book on how not to cycle across Germany.

CLT was a blast, and it’s refreshing to attend an event without having to drive a booth of our own. Talked to lots of people. Since the boss was paying, even did some mingling in that area.

My ADSL line has been hiccupping ☹


19.03.2011 by tg@
Tags: debian event geocache grml

Will drive to Chemnitz now. Maybe meet me there. No booth, just visiting to meet everyone again, rather spontaneous.

Rhonda suggested I document how to use the LLS (Launchpad Login Service – their implementation of an OpenID provider) as Delegate, which basically means, you can put something up on your webpage, which can be a simple static (X)HTML page like mine (a /index.htm is especially nice, a /~user/index.htm works too), and use its URI and not to login. For example, this often hides the LLS from view e.g. in blog comments, such as those where Canonical is being criticised ☺ – but it’s also yours, easier to type and to change if you switch service providers.

The basic idea is to go to your Launchpad user page and view its page source. Look for openid relation links in the header – on Rhonda’s the value we’re looking for is “cyLQbcp”, and you see it several times.

Now you put this on your web page:

	<!-- begin: OpenID delegation to LP -->
	 <link rel="openid.server" href="" />
	 <link rel="openid.delegate" href="" />
	 <link rel="openid2.provider" href="" />
	 <link rel="openid2.local_id" href="" />
	<!-- end: OpenID delegation -->

Of course, insert your, not Rhonda’s, ID. Do note that we don’t copy the X-XRDS-Location tag (that breaks things for some unknown reason), but otherwise, what we insert on our page is pretty much a copy of the info on the user page (maybe it’s a Delegate page, too?).

As usual, try at your own risk, bug Canonical if it breaks. It works with AO3, Gerrit Code Review, and others though (interestingly enough, better in Lynx than GUI browsers because I stay logged in across Lynx sessions (and just have to confirm sending “my information” to the accessing site), whereas I have to re-login to the LLS in every GUI browser session).

As with the LLS generally, “to access a site which is not recognised” is expected and worked on with low urgency (mostly cosmetical, I think).

mksh-current has just gained an experimental recursive parser for command substitutions, fixing RedHat BZ#496791 and decades-old complaints about the pdksh codebase, compared to AT&T ksh93. (GNU bash could also do the example, but not some other things mksh(1) parses fine now.)

This means that things like the following work now.

	# POSIX, should “always” work
	echo $(case 1 in (1) echo yes;; (2) echo no;; esac)
	# POSIX optional, works now in mksh, works in GNU bash
	echo $(case 1 in 1) echo yes;; 2) echo no;; esac)
	# GNU bash seems to choke on comments ending with backslash
	# a comment with " ' \
	echo yes
	# a comment with " ' \
	# No non-recursive COMSUB parser can pass all of the above
	# tests and these below at the same time (some extensions)
	echo $(typeset -i10 x=16#20; echo $x)
	echo $(typeset -Uui16 x=16#$(id -u)
	) .
	echo $(c=1; d=1
	typeset -Uui16 a=36#foo; c=2
	typeset -Uui16 b=36 #foo; d=2
	echo $a $b $c $d)
	# the ‘#’ is especially tricky, that’s why the above cases

Next on my TODO is the complete rewrite of the read built-in command, as well as its documentation. I think that the (reduced) goals for mksh R40 will have been met by then, except porting to LynxOS and MPE, but we’re working on it, and re-testing Syllable and Plan 9). Of course, a release implies testing on a lot of the supposedly supported platforms, so it won’t be out “immediately”. Though, associative arrays have been removed from the R40 goals, so that I can at least get a new release out. Note that Debian and OpenSuSE Buildservice users have been provided with somewhat well-tested mksh-current snapshots for a while already, and Gentoo users can use the “live ebuild”; there’s always compiling from source too…

(Free)BSD vs. Linux

03.03.2011 by tg@
Tags: debian event ill mksh rant

Warning: this is a rant against BSD (specifically FreeBSD®, but don’t let me get started on DragonFly, who think it’s wise to drop all shells except ash from the base system and rely on pkgsrc® – yay let’s compile a dozen packages just to get a shell with tab completion, not to mention boxen with no network access – for the task – although others seem to go into that direction too…; you know, there’s BSD, and then there’s FreeBSD…) – don’t like, don’t read.

If you want to change something in the BSD world, you gotta fork your own BSD – no other way around the thickheads. Ok, back then, I ran into a particularily thick one, but others tend to not be much better. Users share the thickness. If you want to change something in the GNU/Linux world, just make a package, have someone upload it, prod (or pay, Hanno got a Radler) people to do it, or just upload it yourself.

At the BSD booth at FOSDEM, despite me bringing the Windows® Mobile 6 Professional devive, strictly for Geocaching mind you, Macintosh boxen had a share of more than 50% – I didn’t manage to tip the scale. At the Debian booth, almost everyone had a “I want to buy a new laptop some day, but it just keeps on working and doesn’t break” pre-Lenovo IBM laptop. No hyping of Google either. (Last year’s CLT saw BSD people advocating pro-Schily – the guy with the broken encoding in his name – shockingly.)

Honestly, tcsh, FreeBSD® people? Sorry. While I agree that there is merit on having the same script and interactive shell, as someone has pointed out (copy-paste examples into the command line), there’s those zsh users who use mksh or GNU bash for scripting. Or just POSIX shell. And that’s with an interactive shell which can be used for scripting. On the other hand, the C shell (both csh and tcsh) cannot.

And what’s with pretending the accent gravis is non-combining, called “backtick” (such a thing does not exist); and advocating it? Sorry, if your csh/tcsh doesn’t handle the POSIX $(…) you should just drop it. (By the way, there is a convention that example command lines are prefixed with for csh and for sh (or but we write $ sudo  instead, these days). Use it. Or leave it. If you have examples that substitute another process’ output, be specific.) It’s funny to see how one person tries to defuse my arguments against csh by telling me “it’s just an interactive shell”, while the other argues that people copy-paste between them, to which that was my response. Read the thread!

And please, get your facts right. “I would prefer that the standard shell be at least Bourne-compatible.” You don’t want Bourne (“^” instead of “|” for pipes), you want POSIX. That GNU bash is called the Bourne-Again Shell in one of their usual semi-bad puns doesn’t help the global perception of such things any. Also, the root shell and /bin/sh are disjunct.

(Plus, why change the root shell, use sudo(8), plain and simple.)

ObNote: in jupp (should I package that for Debian, btw? rather upload, packages are ready…) the ‘`’ key is used as præfix for Ctrl-X (`X) or to directly enter numerical (decimal, octal, sedecimal/hexadecadic) ASCII, 8-bit or Unicode codepoints. Yay!
And even the FSF has seen the light; for a few releases already, GCC uses “'…'” instead of “`…'” for quoting in messages, even without locales. Great job there! (LC_MESSAGES=en_GB.UTF-8 usually works, too, though.)

ObDisclaimer: I have an (yes, Google…) Alert on the word “mksh”, so I know when it’s being discussed. This obviously includes certain fora. Also, I’m a shell implementer and bound to know a certain amount of details. Plus, mksh’s build script runs with pretty much any Bourne/POSIX/Z Shell which has functions and not too many bugs. I wrote it. Go figure. No lowly trolling.

FWIW, mksh(1) has the cat(1) builtin both because Android has no cat(1), and as speed hack. Almost all other shells have worse speed hacks, like a printf(1) builtin. And recently, builtins have become direct-callable, so this actually reduces the overall system footprint. (Its inclusion also provides for some other possibilities, internally.) And as two final side notes, if you haven’t seen this: determine which shell we are run under (CVS) and I still offer a prompt conversion service (send me any GNU bash or oksh $PS1 and I’ll send that to you in mksh(1) syntax – optionally with adjustments/improvements, like cwd uses only up to 1/3 of screen width).

Eh. Why does mksh built with (a patched: mkstemp(3) added) klibc work suddenly, unexpectedly?

To reproduce, I just uploaded mksh_39.3.20110218-1.dsc and you can run DEB_BUILD_OPTIONS=mksh-static=klibc,dietlibc,eglibc dpkg-buildpackage -rfakeroot to verify it, once you have mkstemp(3). (I will probably send a smaller implementation of that in, later.) I have that and the open fix and the m68k patch applied, nothing else… where did my bug go?

ObQuestion: what’s the legal (copyright/trademark) status of the Atari logo (the one in rainbow colours, with three things going up, right and left “leg” looking like an umbrella stand’s)?

FOSDEM was a blast!

13.02.2011 by tg@

I just need to work more on bilocality. While I did find two geocaches, one at the South/Noon Train Station (taalverwarringen…), one in the buurt of the University, I did manage to miss the AW building completely and utterly. Wow. Except, that Haiku guy came over to talk for a bit (nice). And I drew. An Atari logo with swirl, for that weird stuff I recently have been found doing.

More mksh-current news coming soon, stay tuned. In the meanwhile, I met bonsaikitten IRL (at FOSDEM, yes, too) who kindly made a “live ebuild”, i.e. a source package building -current.

Finally let’s say a big thank you to the person mostly manning our booth, gecko2, and to Benny for talking to people, getting That Other Packaging Thingy working, and pimping the website a bit.

On SecureAPT

25.01.2011 by tg@
Tags: debian rant

Dear Opera Software A.S.A.

It’s nice that you employ SecureAPT for your package repository, however, the effect is slightly lost by you replacing the key each year, never signing any of them, and putting them up on an http but not https site.

Update 18.08.2011 – please refrain from putting a file /etc/apt/sources.d/opera.list inside your binary DEB packages, as well. As a system administrator, we may very well have mirrored the binary packages, and do not want accesses to external APT repos, for a shitload of reasons. Honestly. (Lintian could warn about it…)

If you don’t get the message, please contact me. Or any of my fellow Debian Developers. Thank you very much.

tl;dr: New Evolvis release, scmgit plugin, bidirectional merge with FusionForge 5.1; committing tarent features back to Mediawiki and Mailman packaging in Debian; Fairtrade Software

tarent GmbH now offers both git and Subversion as Source Code Management systems in their Evolvis platform.

The scmsvn plugin of EvolvisForge was amended with the scmgit plugin, backported from a newer FusionForge release, while work to upgrade the main forge code to the soon-to-be-released FusionForge is ongoing in parallel. Of course, most features, such as commit mails (in git, they’re really sent out after a “git push” from the developer), are already available for both; others will follow after the code base upgrade for simplicity. Already, git repositories can be used for all regular tasks – although it’s currently not possible to have both git and svn repositories in a project.

During the migration (as well as regular development), many features of Evolvis will show up in regular FusionForge to benefit the broad community of Forge users, courtesy of the Open Source policy of tarent GmbH, who has contracted the FusionForge project leader to improve both projects, following the mantra of “Fairtrade Software”.

Among other changes, the SOAP WSDL has been corrected and is now versioned, and other areas (especially the Tasks and Tracker) have been improved.

There’s a release announcement of the new version at Evolvis, in case you want to know the details.

Finally, an Evolvis developer has set foot not only in the Mediawiki packaging team at Debian but also the Mailman packaging team. Expect many, if not all, improvements to show up there within some time.

Did you know that Hudson is now called Jenkins? This is a change in name only, due to trademark concerns, but rest assured the Evolvis platform will continue to offer Continuous Integration in a usable fashion, no matter the technology behind (there was Continuum, now Hudson, soon Jenkins).

In our evergoing quest to improve the Evolvis platform, we wish you a pleasant user and developer experience!
//The Evolvis team

FOSDEM, the Free and Open Source Software Developers' European Meeting

❧ Who’s not? ☺ Same procedure as every year.

(okay, lolando prefers skiïng but…)

Anyway. A cow‐orker told me that Belgium again/still has no gouvernment, and they have been asked to grow out their beards until they do. I found “evidence” on the ’net but won’t link it here, also it’s on German… anyway. Let’s all join in. (Besides, I now have an excuse to not shave, maybe even my grandmother will accept this one…)

RT said on IRC that mksh will probably work on MSYS.

My Debian/m68k stuff is coming around nicely, but I still haven’t gotten around to do everything planned, plus I need to grow a new kernel and eglibc, after the latest uploads, and the 2.6.37 based one panics. Also I’ve got to take care to not overwork myself. (And make a MirBSD ISO for FOSDEM.) But hey, it’s been not working for some time and better now. And slow anyway ☺ yet we’re progressing. Does anyone know how to debug that a C programme only calling res_init(3) segfaults?

Benny is apparently not just working on making NetBSD® pkgsrc® available on MirOS BSD (picking up my work from 4+ years ago) but also replacing The MirPorts Framework with it. Sad, as I got a request for a gajim MirPort over a cocktail just this evening…

When doing porter uploads, one must not forget to pass -m"My Name <my@email.addr>" to dpkg-buildpackage, e.g. with --debbuildopts for cowbuilder. Thanks Aurélien and sorry to everyone who got the upload mails.

(How are the rules for sponsored uploads? I get conflicting info on these, and indeed, the one I sponsored never showed up on my QA DDPO page…)

I’m almost finished with “sort of re-bootstrapping” Debian/m68k (I can use etch-m68k as well as what was in unstable at the moment as dependencies, so it was not that much, still, 305 binary packages build from 84 source packages, most for unstable (very few for unreleased, with very responsive maintainers, thanks all, who will include the patches in their next uploads) is a bit… including rebuilds with newer versions, more patches, more testing or newer dependencies installed. I’ll probably upload on Sunday evening, as I’ll be off for 2-4 days at least from then (see below). Ingo tried to test on real hardware, but as Murphy wants a hard disc failed… we’ll still try to get something done over the weekend. If you want to have a look, see my repository index (sources.txt contains a sample sources.list file, 0-NOTE.txt some hints, including the right debootstrap/cowbuilder magic and speed tricks). I’ll need to learn how to use LVM and set up a buildd now…

I’ve not been in much of a hacking mood recently – all these visits to the dentist leave me in unrest and disturb my equilibrium. Hence, not much activity even in mksh even if there was need, almost none in MirBSD. This is only temporary, but I won’t attend OpenRheinRuhr, or, if I come at all, it’ll be for socialising only and probably only one day. Benny’s done with his Doctor (in France, no idea whether it’s one in Germany as well) of Chemistry and has returned to hacking some (World of) Google-Go(o) code. I expect MirBSD activity to slowly raise once we can come back. Please accept our apologies.

I’m currently working on something which will eventually amount to a re-bootstrapping of sorts of the Debian/m68k (Linux) port – patches to the Linux kernel, gcc, etc. are prepared (some have been accepted into upstream or the packages already). I will probably have more, once the compile processes finish, anyway (even emulated, it’s slow).

I think that, once I get past that TLS (thread-local storage, needed by eglibc) migration, I will try to find out a list of packages needed for debootstrap (AFAICT: all packages marked Essential, or of priority important, and all marked Build-Essential (for the *-builder variant), and their dependencies (although I’ll substitute sysv-rc with file-rc, which is better and needs less deps)) and pull arch:all from sid, then build the rest myself using a consistent snapshot of sid possibly with patches going to unreleased. Then I can use cowbuilder to make cleaner packages, which can eventually be uploaded (once I get enough to get a buildd running – kernel, bootloader, etc) – binNMUs are way to go here I suppose. I will only upload once it’s self-hosted, installable (seen by edos-debcheck), clean, etc. (i.e. I’ll rebuild all binaries) and probably keep a bootstrap repo around (until m68k caught up) so that unstable (possibly amended by unreleased for a while) will not again become uninstallable, e.g. if arch:all packages change their dependencies (Python, gcc-defaults are some I’ve seen). That bootstrap repository is needed anyway because debootstrap can’t install from two separate repositories (unstable+unreleased for example).

Progress is slow because I try to keep as close to official packages as possible, refuse to cross-compile, and try to produce uploadable if possible packages all the time. Getting patches into packages, so that I can build from unstable, instead of unreleased, has proven time-consuming and occasionally frustrating as well. Although I would like to thank the people who helped me on the way already. (I am not naming any in fear of forgetting some, but you know who you are ☺) They are among the Debian (gcc, kernel, m68k) and Linux-68k crowd.

(Why does genattrtab in gcc-4.4 take 3½ hours when it took less than half an hour in gcc-4.3 anyway?)

I’m also still working on mksh and some Python ISO hacks for mika and some minor stuff, and further cleaning up MirBSD.

Well, did I mention dentists are sadists?

mksh, encodings, MirBSD, BitTorrent, WinCE

28.08.2010 by tg@
Tags: android debian geocache hardware mksh news release snapshot

mksh was merged into Android (both AOSP and Google’s internal master tree) in the night 24/25th August, and is expected to be the one shell to rule them all, for Gingerbread.

mksh(1) now also has a cat builtin, for here documents mostly. It calls the cat(1) command if it receives any options. The shell is nevertheless smaller than yesterday because of improved string pooling.

There’s another reason to use the MirOS OPTU-16 encoding instead of PEP 383, on which I already wrote: try passing a wide-char filename to a function such as MessageBoxW, or create a filename on a system using wide chars, such as FAT’s LFN or ISO 9660’s Joliet, or one that only allows Unicode (canonically decomposed – ü → ü – out of all things) like HFS+. OPTU-8 at least maps to somewhat reserved codepoints (would, of course, be better to get an official 128 codepoint block, but the chance’s small of getting that in the BMP). Still.

Oh well, the torrents. I’ve remade them all, using one DHT seed node and OpenBitTorrent as tracker and put them on a very rudimentary BT page that will be completely redone soonish. Please re-download them. I currently do not believe will return.

Finally, I fell victim to a selling-out and may have just bought a Windows Mobile 6 based phone (Glofiish X650) and an SDHC card and an extra battery with double capacity. Well, at least it’s said to run CacheWolf well. I still would like to have something like Interix, Cygwin, UWIN, coLinux, or maybe some qemu-for-WinCE variant that runs Android, Maemo, Debian/armhf (or armel or arm) at near-native speed (and is usable – the device sadly doesn’t have a hardware keyboard, but it comes with SiRFstar Ⅲ GPSr). It only has 64 MiB RAM, like the Zaurus SL-C3200 and the jesusPhone, though. ☹ Any chance to get MirWorldDomination onto that device as well?

Tomorrow, eight years ago, is the date we now use as birthing point for MirOS. The thing is, we did not really want to create a BSD of our own, fork, or whatnot. We were mostly happy OpenBSD users (really happy before the first eMail exchange with its developers, where Theo de Raadt did indeed stand out but was not the only one – just the one with the authority to deny us), improved it locally and submitted patches and ports. We were flamed for that or, worse, ignored. I begun putting up my “OpenBSD patchkit” on my homepage (back then, at Tripod) and still tried to feed things to upstream and OpenBSD. Then, at some point, Theo de Raadt made it clear he did not want me and the patch kit had grown (from one 4M file into several of them), so I ended up doing a “cvs -d /cvs init” and went from there. Benny’s story is similar – he laughed at me while trying to get ports added to OpenBSD, then discovered his ports were added to the MirPorts Framework and getting commit access there was easier than getting some random developer to commit something of his to OpenBSD. (This trend ended there though… every single person I approached since has become OpenBSD ports committer – I wonder whether they used my invitation letter to blackmail Theo?) It’s often thought that there was a clash of opinions between Theo and me. I think while we might disagree in certain aspects or priorities things should have, in the end we both wanted the same thing, I just was promised to never become a member of the OpenBSD project, so it’s really just “them” being uncooperative. (They (Henning and others) did burn the T-Shirt I gave Theo as a gift some day for making OpenBSD what it was. I won’t comment on that, again, now.)

FrOSCon was a blast. I had two booths of my own – MirBSD and FreeWRT – as well as shares of Debian and Grml. Well, MirBSD was run by Benny and gecko2 because I just didn’t have any time for it, despite XTaran’s help with the FreeWRT booth. All I did was the initial setup of both booths, while at the same time answering about three questions regarding FreeWRT in parallel. Wow. What a little small, open hardware can do to you. XTaran and I had fun and we’ll do FreeWRT booths again; I managed to flash my two FON2100 devices (“La Fonera” – the FON2200 can use the same image, says nbd of OpenWrt) and will fix the port’s remaining few bugs I found; XTaran will try to push the WL-500gPv2 development. The social part was nice as well, although I think the greek restaurant in the city will not be visited by me again. Anyway, if you didn’t attend FrOSCon, your own fault…

Since the BitTorrent tracker used by MirOS is down, here’s the link to the [updated 2010-08-28] [deleted 2014-05013] current (FrOSCon 2010 Edition) snapshot’s torrent, Triforce as usual. We’ll probably rewrite torrent files for all our ISOs and publish them on the MirBSD website. I’m currently considering OpenBitTorrent plus one or two DHT seed nodes with no statistics. Maybe with webseed. (Need to update the libtorrent/rtorrent MirPorts first, though…) Other options would be different trackers or running one of our own. I will announce the outcome as news entry, once done.

On the plus side, the review process of mksh(1) in Android continues, and I fixed the realpath builtin to behave even more POSIX-ish.

FrOSCon 2010 and other sundries

06.08.2010 by tg@
Tags: debian event grml

The FrOSCon 5 - 21./22. August 2010 booth plans have finalised, I am rather content:

	┌──┐              I ❦ STANDPLAN FROSCON 2010
	│ F│reeWRT
	│  │
	│  │                 C = Collectd
	│C │   Grml

This is especially good, as XTaran will be shared among (at least) Debian, Grml, FreeWRT; same for me plus of course MirBSD; kimnotyze is FreeWRT but may help with MirBSD; benz and gecko2 probably are MirBSD but gecko2 could help with FreeWRT, tokkee was interested in FreeWRT too… anyway.

Some days, you just love software.

	Aug  6 13:55:01 blau firesomething-bin: stack overflow
	    in function VFY_EndWithSignature
	Aug  6 13:55:01 blau /bsd: signal 6 received by
	    (firesomething-bi:1146) UID(2999) EUID(2999),
	    parent (sh:9059) UID(2999) EUID(2999)

Thus, let me reïterate it for all of you:

Well, now that the Debian Release Managers have sent their freezing bits around… *shudder*… Squeeze is frozen. Well, at least everything I have my hands in has migrated. I’m still… not persuaded. I also can’t decide which looks worse (KDE 4 or Win 7), tending towards KDE 4…

Why does all the horrid software (Solaris, Java™, OpenSSO, MySQL, etc.) tend to end up at Oracle at the moment? Let me quote from some Debian mailing list:
>>What happened to the Unix philosophy?
>Modern Solaris engineers
Is that similar to high-speed horse carriages?

My RPM repository has been pimped a bit – I ported some stuff from my DEB repository and updated them in both (rdate(8) and ntpd(8), specifically). Still ought to work more on them, but currently MirBSD base is most important, although I’m dying for mksh associative and multi-dimensional arrays, as well as more sh(1) conformance assorted bug fixes.

Well, there’s a life besides the computer. I’ve taken today off, wanting to hack on MirBSD’s most urgent problems (but probably end up doing that tomorrow), slept long, and will meet with cnuke@ and gecko2@ for Greek style dinner. The latter will almost certainly end up with a long-time work contract at the same place where I run a lot of things already, so congratulations. In the meanwhile, bsiegert@ has almost become a Doctor of Chemistry, and my brother’s finished his Maths and Economics diploma.

Also, I’ve put up the logo of the company where my new dedicated server is hosted; they reduce the monthly fee in exchange for this, so humour me and pay them a visit. They’re IPv6 pioneers, actually. (The server is now installed but not completely set up yet, and I have yet to begin moving services; it’ll be better than the VM eurynome is, but the clock could use the new timekeeping subsystem in the kernel as well as socket send (ÆrieBSD) and receive timestamps as it’s off by 0-1000 ms.)

Speaking of kernel stuff, yesterday I considered moving wscons(4) to UTF-8 again (since everything is CESU-8, we need to take raw octets into account also). I’ve seen OpenBSD begun importing Citrus… *shudder* Anyway, that’s my part, but I’d like volunteers for backporting things like the timekeeping stuff (and possibly more hardware support), and writing a pivot_root like thing (explained on some mailing list already) so we can use ramdisc root to do loopback root.

FrOSCon 5 - 21./22. August 2010 is approaching rapidly. I’m a bit envious at some of the tracks (I mean, really, geocaching (ok, I did the surrounging caches over the last years but still), learning python by means of game programming, etc. really sounds interesting – and I know people who could benefit from a non-kids version of that as well) but this year’s FrOSCon is nothing for me to curse about either: I managed to get both a booth for The MirOS Project (MirBSD, mksh and other subprojects) as well as one for Waldemar’s FreeWRT (although wbx@ – if he comes – won’t join there since he forked his own fork since its conception). Booth staff are, currently: tg@ and bsiegert@ (Developer), gecko2@ (Staff) for MirOS, tg@ and “XTaran” abe@ (Developer), kimnotyze (Hacker) for FreeWRT. (XTaran will probably be helping Debian/Grml too.) This year, it’ll be my job (after 2 years of aptituz) to keep the Altbier-Fraktion watered, I’m thinking one crate of Schlösser Alt and one crate of Hannen Alt?

Have a look at the Program and don’t tell me you won’t come! It will rock! (Except there won’t be Formorer’s Chilli, but that’s no reason, there’s enough other stuff in manageable distance.)

Besides interesting booths and talks, FrOSCon is still looking for helpers who will not only get free entrance but also catering during operation.

Well, I suppose I should be happy that mksh is actually used…

  • [tg] Correct shf buffer I/O routines to avoid a memory corruption bug discovered by Waldemar Brodkorb and other bad effects
  • [tg] Fix NULL pointer dereference during iteration loop when checking for alias recursion; discovered by Michal Hlavinka

That’s OpenADK (Waldemar’s fork of FreeWRT, which is Waldemar’s fork of OpenWrt), and Red Hat Enterprise Linux, respectively. Popcon in Debian and its derivates is also pleasant.

I could use some help bugfixing this though:

	(sleep 3; exit 12) &
	sleep 6
	# background process is done by now
	wait $bgprocpid
	# POSIX mandates that, since $! was asked
	# for, wait is to reply its errorlevel

Somehow, JF_KNOWN is never set – and I can’t debug this with gdb(1).

(There’s also a dashism in some *buntu start scripts that does pretty much the same except it uses “wait %1” there. In fact it doesn’t even seem to use $! – no idea whether we can support that at all in a POSIX shell – which dash clearly isn’t… – without keeping track of background processes forever.)

I’ve got some interesting results using r1.1 of an example test programme (r1.2 got cleaned up and more output) on various systems, regarding ASLR. The 1.1 revision tests everything mksh R40+ will use (except there will probably no larger than page sized allocations) for its LCG PRNG. On OpenBSD (MirBSD, ÆrieBSD) malloc(3) uses in fact mmap(2), which is randomised. (Though -pie doesn’t yet work as it’s supposed to.) Some OSes are better than others… but look for yourself. (Read on to continue, not part of the RSS for size reasons. This wlog entry may be updated – with bumped date – unperiodically.)


tg@blau:~ $ mgcc -static x.c
x.c:0: note: someone does not honour COPTS correctly, passed 0 times
x.c: In function `foo':
x.c:27: warning: function returns address of local variable
tg@blau:~ $ ./a.out
0xa9332000 0xaba65000 0xa0ae7000 0xcfbed990 0xcfbed994
tg@blau:~ $ ./a.out
0xa91b4000 0xa02b1000 0xa1602000 0xcfbf8680 0xcfbf8684
tg@blau:~ $ ./a.out
0x9f731000 0x9cb2a000 0xa94ca000 0xcfbf5840 0xcfbf5844
tg@blau:~ $ ./a.out
0x9c2af000 0xa6a0b000 0xa4ce1000 0xcfbefac0 0xcfbefac4
tg@blau:~ $ ./a.out
0xa3b61000 0xa96de000 0xa96df000 0xcfbedcc0 0xcfbedcc4

Debian Ätsch/i386

tg@frozenfish:~ $ gcc -static x.c
x.c: In function ‘foo’:
x.c:27: warning: function returns address of local variable
x.c: In function ‘bar’:
x.c:33: warning: function returns address of local variable
tg@frozenfish:~ $ ./a.out
0x80b2a20 0x80b2a30 0xb7745008 0xbf985ce0 0xbf985cd4
tg@frozenfish:~ $ ./a.out
0x80b2a20 0x80b2a30 0xb7726008 0xbfb911b0 0xbfb911a4
tg@frozenfish:~ $ ./a.out
0x80b2a20 0x80b2a30 0xb7784008 0xbf83d040 0xbf83d034
tg@frozenfish:~ $ ./a.out
0x80b2a20 0x80b2a30 0xb77e8008 0xbfc0f840 0xbfc0f834

tg@frozenfish:~ $ sid
I: [sid chroot] Running command: “mksh -l”
tg@frozenfish:~ $ gcc -static x.c
x.c: In function ‘foo’:
x.c:27: warning: function returns address of local variable
x.c: In function ‘bar’:
x.c:33: warning: function returns address of local variable
tg@frozenfish:~ $ ./a.out
0x80c86a8 0x80c86b8 0xb77c3008 0xbfaa1900 0xbfaa18f4
tg@frozenfish:~ $ ./a.out
0x80c86a8 0x80c86b8 0xb77d2008 0xbfcc0260 0xbfcc0254
tg@frozenfish:~ $ ./a.out
0x80c86a8 0x80c86b8 0xb77c1008 0xbfbe2120 0xbfbe2114

uname: Linux frozenfish 2.6.18-6-686 #1 SMP Fri Feb 19 23:40:03 UTC 2010 i686 GNU/Linux

Solaris 8/sparc64

tg@stinky:~ $ gcc -static x.c
x.c: In function `foo':
x.c:27: warning: function returns address of local variable
tg@stinky:~ $ ./a.out
595f0 59bf0 59d00 ffbefbb4 ffbefb5c
tg@stinky:~ $ ./a.out
595f0 59bf0 59d00 ffbefbb4 ffbefb5c
tg@stinky:~ $ ./a.out
595f0 59bf0 59d00 ffbefbb4 ffbefb5c
tg@stinky:~ $ gcc x.c
x.c: In function `foo':
x.c:27: warning: function returns address of local variable
tg@stinky:~ $ ./a.out
20950 20f50 21060 ffbefb3c ffbefae4
tg@stinky:~ $ ./a.out
20950 20f50 21060 ffbefb3c ffbefae4


mirabilos@stargazer:~ $ gcc -static x.c
x.c: In function 'foo':
x.c:27: warning: function returns address of local variable
x.c: In function 'bar':
x.c:33: warning: function returns address of local variable
mirabilos@stargazer:~ $ ./a.out
0x800603080 0x800605040 0x800700000 0x7fffffffe62c 0x7fffffffe62c
mirabilos@stargazer:~ $ ./a.out
0x800603080 0x800605040 0x800700000 0x7fffffffe63c 0x7fffffffe63c
mirabilos@stargazer:~ $ ./a.out
0x800603080 0x800605040 0x800700000 0x7fffffffe62c 0x7fffffffe62c

uname: MidnightBSD 0.3-CURRENT MidnightBSD 0.3-CURRENT #1: Thu May 27 22:13:45 EDT 2010 amd64

Debian sid/mipsel

(QEMU, thanks to Aurélien! Debian unstable from approx. Jan 2010)

root@debian-mipsel:~ # gcc-4.4 -static x.c
x.c: In function 'foo':
x.c:27: warning: function returns address of local variable
x.c: In function 'bar':
x.c:33: warning: function returns address of local variable
root@debian-mipsel:~ # ./a.out
0x4aa740 0x4aa750 0x2aaa8008 0x7fa417e8 0x7fa417d8
root@debian-mipsel:~ # ./a.out
0x4aa740 0x4aa750 0x2aaa8008 0x7fc67708 0x7fc676f8
root@debian-mipsel:~ # ./a.out
0x4aa740 0x4aa750 0x2aaa8008 0x7fb68238 0x7fb68228
root@debian-mipsel:~ # ./a.out
0x4aa740 0x4aa750 0x2aaa8008 0x7fc586c8 0x7fc586b8

uname: Linux debian-mipsel 2.6.32-trunk-4kc-malta #1 Mon Jan 11 03:45:08 UTC 2010 mips GNU/Linux

Gentoo GNU/Linux on amd64

gcc-4.4.4, glibc-2.11.2-r0, 2.6.35-rc4 x86_64

0x20cc010 0x20cc030 0x7fef0c497010 0x7fff32148fec 0x7fff32148fec
 0xa35010  0xa35030 0x7f575d0e4010 0x7fff0dd7220c 0x7fff0dd7220c
0x1f90010 0x1f90030 0x7f8657107010 0x7fff6116813c 0x7fff6116813c
 0x9dd010  0x9dd030 0x7f1eab0a6010 0x7fff3dcc638c 0x7fff3dcc638c


Not everyone does ASLR… but there’s enough variety (and with eglibc’s AT_RANDOM even proper entropy) inside for our purposes. On OpenBSD and MirBSD, we’ll still use KERN_ARND as it’s extremely cheap entropy (code paths checked on both) but not for every call of $RANDOM. On things like Debian/m68k mksh(1) ought to have gained a possibly noticeable speed-up.

Back home

11.07.2010 by tg@
Tags: bug debian event geocache mksh news release snapshot

Bordeaux was very nice (and towards the end much cooler… it’s actually hotter here at more than 50½° north – too warm to think, or do anything) but the LSM/RMLL was very french. They’ll be in Straßburg and Lüttich the next two years so we can probably be expected to attend. I don’t think I can eat duck (which, in south-west france, is a vegetable) or like all that classic french multi-course food so much, but I had enough Couscous Merguez and Thé à la menthe fraîche… and similar good stuff. Many people spoke English and actually asked me whether I do (probably they couldn’t bear me trying to spea^W^W^Wbutchering the language of the Grande Nation) and in general were a friendly bunch. I did see some people with machine guns in the city on the last day, though. No idea what/why… didn’t dare asking ☻

Just another reason to boycott flying: Mario Lang (one of the speakers) was apparently held on the airport and treated as a terrorist due to his Braille line… they thought it was a bomb or somesuch thing.

Read on for more…

Travelling with the Thalys and TGV was nice (but I loathe the Métro parisienne… they should build a ring train like the Berlin S-Bahn and just put another stop before Paris Nord and Montparnasse for people who just want to switch trains to take the ring train to the other line). And I want air conditioned trams in Germany too!

I met Uriel (invited him for some food and talked lengthy with him and some 9grid guy), XTaran (who was rather busy organising things), and a number of other people. Did some PGP keysigning as well. There’s now an experimental MirOS presence at Launchpad, not sure what exactly we’re going to do with it but, as Canonical does not care (as Jonathan said in his talk – great slides, by the way, really impressive), there’s no harm in having it. Some Perl guy from America (USA… just to make sure ☺) wanted a photograph of me with a sign “I love CVS” just so people back at home would believe him he’s met such a person *grins* of course I plugged in a little advertising but cvs(GNU) is honestly good. The forge hacking session was a little under-visited (but still a success in terms of getting more communication and maybe collaboration underways, especially thinking of common interfaces, DC, semantic web, OSLC-CM) and since the room was (in contrast to my hotel room and the trams!) not air conditioned we didn’t get much hacking done. The Debian booth was about 40% of one FOSDEM style table wide… and subsequently crowded. There were more people (of course, I was trying to get mksh into Haikuports, Mandriva, and other things; talked about KDE 3.5.11 (Trinity), Qt 3 vs Qt 4, and kwalletcli, and in general to a not-so-usual bunch of suspects – like I said, LSM/RMLL really is pretty french-only).

It is too hot, but I still committed src/etc/rc,v version 1.110 which you want to upgrade your /etc/rc to before upgrading mksh(1) in MirBSD. (All in the name of better performance on platforms such as Debian/m68k and not raiding Linux’ inferior RNG… but it does simplify things.)

I could probably write more but at the moment just want to lie down and die until it gets cooler… even the rain didn’t help. My feet hurt (Montparnasse-Bienvenue didn’t help) too.

The current version of mksh had use of arc4random(3) removed, including “set -o arc4random”, to speed it up (on some architectures, a lot) – this will break some existing scripts (such as /etc/rc *cough* on MirBSD…). Hence I decided to publish the next version of mksh(1) as R40 based upon current development, and defer plans for associative arrays (and multidimensional arrays) for mksh R41. There’s also already the change to arguments, so this suits me quite fine.

(Read: if running MirBSD, don’t upgrade mksh at the moment.) There will be a new MirBSD snapshot once this is fixed, maybe a few more changes to the shell for better POSuX compliance, and the recently mentioned patent on LFNs (long filename) in FAT will be taken into account with a patch to msdosfs.

I’ll travel to LSM/RMLL 2010, the Libre Software Meeting (Rencontres Mondiales du Logiciel Libre) tomorrow until the weekend, to hack some on FusionForge (this is worktime for me), visit XTaran, Uriel, and maybe a couple of other “usual people”.

Thundersday, between 10:00 UTC and 12:00 UTC, eurynome will be shut down by gecko2@ due to power supply maintenance on the host system data centre.

We have a new mirror in the Americas, thanks a lot to Mike 'Fuzzy' Partin! Benny will mention it on the webpages once it’s working.

SPARCstation 20 (75 MHz)

09.06.2010 by tg@
Tags: debian rant

Wenn Du denkst, MirBSD wäre langsam und sein Installer in irgendeiner Art und Weise doof, dann installiere mal Debian (etch, neuere Versionen können nur noch sun4u und sun4v, kein sun4m mehr) auf einer SPARCstation 20.

Lahm wie Sau, das Teil. Und der Installer ist schwarz auf weiß – was nicht so schlimm wäre, wenn der Cursor (und die dialog-Markierungen) nicht ebenfalls schwarz auf weiß wären…

Ich bin ja mal gespannt, ob das durchläuft.

In response to a planet.d.o series (mentioned in #grml on IRC) of postings: In a sensible shell, Esc+# not only pushes it back but also re-enables the command. Try it out: l s Esc # Cursor-Up Esc #

tg@blau:~ $ #ls
tg@blau:~ $ ls

The command sticks in the history, and is not immediately shown in the next interactive input line, which I consider a plus in most use cases. Anyway, try mksh (just a-g i it), there are a lot of goodies. I found out about Ctrl-O only a year or so ago myself…

I wonder why schizo didn’t write about how to do it in posh tho ☺

Some things are ugly.

Waldi’s suggestion fails.

db4.6_upgrade: Program version 4.6 doesn’t match environment version 4.4
db4.6_upgrade: DB_ENV->open: DB_VERSION_MISMATCH: Database environment version mismatch

Can’t start it manually.

debian-sks@dev:~$ /usr/sbin/sks recon
Fatal error: exception Bdb.DBError(“Program version 4.6 doesn’t match environment version 4.4″)

The log only shows:

2010-05-09 16:59:29 Opening log
2010-05-09 16:59:29 sks_db, SKS version 1.1.0
2010-05-09 16:59:29 Copyright Yaron Minsky 2002, 2003, 2004
2010-05-09 16:59:29 Licensed under GPL. See COPYING file for details
2010-05-09 16:59:29 http port: 11371
2010-05-09 16:59:29 Malformed entry
2010-05-09 16:59:29 Malformed entry
2010-05-09 16:59:29 Malformed entry
2010-05-09 16:59:29 Malformed entry
2010-05-09 16:59:29 Malformed entry
2010-05-09 16:59:29 Malformed entry
2010-05-09 16:59:29 Malformed entry
2010-05-09 16:59:29 Malformed entry
2010-05-09 16:59:29 Malformed entry
2010-05-09 16:59:29 Malformed entry
2010-05-09 16:59:29 Malformed entry
2010-05-09 16:59:29 Malformed entry
2010-05-09 16:59:29 Malformed entry
2010-05-09 16:59:29 Opening KeyDB database
2010-05-09 16:59:29 Shutting down database

The solution is ugly as hell, too:

root@dev:/ # su – debian-sks
debian-sks@dev:~$ cd DB
debian-sks@dev:~/DB$ db4.4_checkpoint -1
debian-sks@dev:~/DB$ db4.4_recover
debian-sks@dev:~/DB$ db4.4_archive
debian-sks@dev:~/DB$ db4.6_archive -d
debian-sks@dev:~/dump$ cd ../PTree/
debian-sks@dev:~/PTree$ db4.4_checkpoint -1
debian-sks@dev:~/PTree$ db4.4_recover
debian-sks@dev:~/PTree$ db4.4_archive
debian-sks@dev:~/PTree$ db4.6_archive -d
debian-sks@dev:~/PTree$ logout
root@dev:/ # /etc/init.d/sks start
Starting sks daemons: sksdb.. sksrecon.. done.

Wow, our internal keyserver works again. Thank you, Debian…

This solution courtesy of Uwe Hermann, although it was for Suckwürstchen.

CLT 2010

07.03.2010 by tg@

Quite surprisingly, I’ll attend the Chemnitzer Linuxtage 2010 in Eastern Germany. This is a happenstance, I managed to get fast transportation (via my boss) and accomodation (in a hotel). I will try to help staffing the booth of Debian this time (so I cannot be called Traitor any longer). Schedule, due to the spontaneousness of this, no, though. I may not even be there on Sunday, dunno…

No RCBD (or night) though, some real life and a new release (with fix of an FTBFS-on-hurd-i386 bug) though:
RMD160 (/MirOS/dist/mir/makefs/makefs-20100306.tar.gz) = f65bd8ef5cf3306a9112587dd4915b6255e479fe
This version pulls in NetBSD® changes (Acorn Archimedes support, for one), but I’ve also coded support for boot-info-table (J�rg compatible), as well as setting the PVD dates (used by GNU GRUB 2 for “UUID”s).

On MirBSD, cdio(1) can now be used to burn (TAO) and blank (quick) CD-RW media (I backported some OpenBSD changes) too.


04.03.2010 by tg@
Tags: debian

My ex-AM now sort-of mentor Zack asked for help of an Autoconf/Libtool guru with an RC bug... well, The MirPorts Framework taught benz and me, under consumption of a bottle of wine, how to deal with that stuff. So I've fixed #559822 (CVE-2009-3736, another of them...) and NMU'd.

Sort of a PITA, considering gettextize runs interactively, and there's a lot of files to remove in debian/rules:clean for a double-build to not add nonsensical files to the .diff.gz; but I did it in the end.

On ranting...

04.03.2010 by tg@
Tags: debian

Yes, my rant was more against the things I encountered during keysigning, not keysigning itself. However, I still feel X.509 doesn't have these problems, and nothing I can think of will persuade me to think PGP/MIME better than Inline PGP. (Especially when the recipient's key contains a notation that he wants the latter, but not the former.)

Jonathan does have some good points about the (PGP) Web of Trust.

Again, that wlog entry of mine was a rant; I had let the topic stew over night, trying to get the anger out, but on the next day it just wanted to get out of me, I merely wanted a good old rant. I think I'll not include Planet Debian next time I do rant, though (it's not the place to do so).

The Command-Line Interface for the KDE Wallet, Version 2.02, has been released and dput into Debian unstable. (The lenny-backports version will follow.) It took me quite a while to reproduce, then track down, the bug; having unrelated problems at the same time didn’t help either…

Update: This wlog entry uses aggressive tone because I somehow needed to vent frustration from using some of the tools. I should probably provide some constructive critics, too... but this is a rant. Be warned.

Keysigning is useless. I boot up a suitable live GNU/Linux system, install signing-party, take the trouble that is to set up caff, transfer my secret key from the secure box, sign. I think caff providing the keys in a different order than they're given on the command line sucks and just run caff once per key. I did even start Postwreck. But no, people just don't accept any mail from "EHLO grml" systems, and I still cannot control my reverse DNS despite having a static IPv4 address (and IPv6, which looks to be unused). People also pretend I'm on dial-up. Great!

I will no longer participate in any (mass) PGP keysigning but will continue to do so on a per-person basis. Probably sign but one uid, either apply common sense and upload it to t̲h̲e̲ keyserver, or mail the entire signed key to one address.

By the way, how crazy is it that I need to use the deprecated $CONFIG{'mailer-send'} to pass an envelope-from to the mailer? It also suffers from the same delusion as e.g. nmudiff, namely that my Debian box is a fully set up workstation able to send out eMail and configured correctly. At least, it, unlike a number of others, does not assume I use mud (Mutt). grml…

Oh, and caff does a protocol violation (by always sending out GnuPG/MIME and not offering the standardised Inline OpenPGP), I think people just don't care about such. (There is a notation people can use to signal they want PGP/MIME, Inline PGP – which is called "partitioned" – or both (and which order of priority) but, alas, despite Inline PGP being the only one useful for the MUAs without integratin, and being more widely spread than that PGP/MIME crap, the followers of the latter do some (FSF-style?) kind of vendor lockin by not speaking anything else.

Anyway. I'm all for X.509 except there seems to be no sane CA (Startcom is... trouble, even with Opera; is dying). I'll just buy a certificate (not from Verisign though) for www, and roll my own again (I can do it, I have experience with that actually).

On an unrelated side note, still waiting for an OpenSSL patch for that recent TLS extension...

ObRant: password policies, be they required characters or any kind of length restrictions, suck. People I will eventually end up with less secure passwords on such systems, because even if some of mine may appear to be derived from some kind of dictionary (what language that is I'd be interested in, though...) they aren't, and I have my schemes. You got to have them with a gazillion of passwords used. And I probably will forget them more often (and sending them via eMail is also not a solution).

Unrelated notice: mksh R39c with bug fixes coming RSN.

(Updated 24.02. because I was, rightfully, told the language, and the title, were too strong. I also would like to excuse for going so low as to write an ad-hominem attack, which I've since redacted.

FOSDEM 2010, day #0

05.02.2010 by tg@

Yesterday, I arrived in Bruxelles, coming from the Issy/Paris FusionForge meeting which will be described more later by Roland on Plänet Forge. Please tell Ohlol if you use it, for more visibility.

There is a new inter-forge mailing list as well, see the info page. People from Coclico and the various *forges may want to subscribe there (forge developers, not so much users (hosters) or end-users (hosted project developers/users) though).

At FOSDEM, Benny and I (and maybe gecko2) will be running the MirBSD booth, so no Debian staffing for me, sorry. But I will be there. Also please do ask me about mksh – the MirBSD Korn Shell etc.

There are flyers in German (not updated), English and French too! (One of the *forge guys did install mksh(1) after reading it, in fact.)

Don't you people dare miss the two talks: from Benny about how to package with autotools and libtool correctly and from XTaran explaining Debian GNU/kFreeBSD. Benny's also famous for his talk about Painless Perl Ports with cpan2port; XTaran's famous for a whole bunch of other things.

I still have some catching up (wlog entries, keysigning, webpages, etc.) to do, please bear with me. I don't really have a proper work environment with me.

There's a chance I will not be attending the Beer Event in the Delirium Tremens café (last year's still remembered). Benny will certainly be there, though.

Could someone please order nice weather? I still need to eat some lunch and find a supermarket to shop for the weekend!

Hello, Debian!

31.01.2010 by tg@
Tags: debian

I got an eMail tonight. I guess this means I can say hello officially now. (Everything else is details, waiting and fixing some bugs and technicalities, or so.) Thanks to everyone involved, I learned a lot already. Oh, and I had a look into madduck's book (the old sarge edition, which I got for free recently) and found a nice graphic explaining what non-native English speakers (I even had Latin first, and 3 programming languages!) don't, from the Debian constitution.

Congratulations XTaran for making it as well, even visible on the website already!

Please don't file an "Please package mksh R39b" bug again, I am aware there's a new version ;-) as I'm upstream too. I'm just short of time at the moment, and I'd like to put out high-quality packages. Besides, the webpage needs fixing first (while the checksums and the changelog for the release are there, no proper announcement is yet, and I'd like, for this version, to add a "upgrading caveats" section, since due to bugfixes and better standards compliance some scripts need to be updated; some of the pdksh behaviour favoured Bourne over POSIX even!

To do.

28.01.2010 by tg@

I’m going to FOSDEM, as usual

The MirOS Project will have a booth at FOSDEM 2010, business as usual. If you thought otherwise, you’re crazy ☺

I know I should write a wlog entry about the BSP, write more, release mksh R40, fix the TaC of it and the kwalletcli webpage (thanks again, it’s now in Debian sid!) etc.pp but I also need to prepare an ISO for FOSDEM, etc. Heck, I should prepare a talk for FOSDEM, but I’m not going to. If I need to stand there and talk, I’ll talk, not hold a presentation. I’ll just see what people are interested in, talk about The MirOS Project, and improvise.

I’m busy, and there’s only so much computing you can do in a day. This does include the dayjob. At least, my NMUs are in Debian now and probably can help people (and I submitted info about other bugs too).

Anyway, watch the news in the months to follow… can’t talk about everything now.

Marc Fleury, JBoss founder joins the ranks of Tonnerre, me, and other people requesting that MySQL (and MariaDB!) please finally die. Everyone, don't even fork it. Use a real database instead. Or, at least, SQLite. Really.

We're going to FOSDEM 2010 (of course – I've been at every FOSDEM that was not just an OSDEM, Benny and gecko2 are regular attendees as well, as are other projects of mine such as FreeWRT and Debian GNU/kFreeBSD, by proxy). There will be a recent MirBSD snapshot I've yet got to build, with the new floppy format ustarfs (idea, but no single line of their stinking ridiculously huge code, stolen from NetBSD®) and other improvements (albeit less than I wanted to get done by then). The days before, I'll attend the first FusionForge meeting to break up the French Cabal, with my work hat on. That is also my first time in France (outside of the Elsaß). People, make a good impression on me to overcome the classic prejudices ;-)

This weekend I'm going to meet my Debian Application Manager zack, have some good beer (ugh... first this, then Paris, then good belgian beer...) and fix some bugs, all while learning even more. Sounds like fun, but I almost feel overwhelmed, in contrast to the years of much less travelling from my past. I've also started sort-of mentoring Simon, one of our apprentices at work, into the Debian processes. (On an unrelated side note, formorer recently said bpo will become bp.d.o soon. Great!)

Please don't laugh at this excuse for a webpage, as I've yet to fill it in, but my CLI for the KDE Wallet is hereby deemed ready for public consumption, with a bug-fix release 2.01 (bugs actually found during preparation of a port to Debian sid and KDE 4, which is much much worse than KDE 3, plus it looks so absolutely disgusting I'm not even sure Windows® Mistda is worse). I hope the package will end up in NEW soon (and once progressed to testing I may be able to make the KDE 3 variant official via lenny backports; my WTF *.deb repo will hold them until then.

There are more webpages I need to fill in... mksh's TaC, arc4random (which needs some major redesign as well) and BSD::arc4random, the RANDEX protocol (entropy exchange over IRC) and its plugins and patches, ...

Not just Mac OSX (and, I hope, iPhoneOS) will soon come with mksh(1), but also Android (I prepared patches to make it /bin/sh, which works quite well – although I need to find out how to make a hardlink so that #!/bin/mksh scripts will run) and Maemo, for which I wrote an mksh package in a garage project, which also needs some love w.r.t. testing on actual devices, menu integration, etc. (Please contact me if you can help with either of the three.) We also have «lewellyn:#ksh» making a package for the new OpenSolaris system (thanks again). People persuading Apple to put it on the jesusPhone are also welcome. (This does not mean I endorse any of these – right now, I'd probably get the most of a WinCE PDA with built-in GPSr, WLAN and maybe GSM/GPRS.)

English and French native speakers, please review, and Dutch native speakers may contribute a translation of, our flyers. (Source code for these is not available, sorry. Benny makes them in Quark on System 7 in Basilisk II, used to be Classic until Apple yanked it. But still, they use only free fonts, free imagery or such the MirOS Project is allowed to use, and beat every single other FOSS project flyer I've ever seen by far!)

There's probably more I could write, I bet I forgot half of it anyway, but I'll leave it at that for now. Get yourself a nice cup of hot chocolate, pour an Espresso into it, and enjoy the mix with a piece of cake (I'd say strawberry or mousse-pear but all they had was cassis-créme) and pity me for not knowing any French next month.

The MirBSD Midi-ISO (bi-arch manifold boot) and NetInstall for both i386 and sparc have been upgraded to the 20091226 snapshot (sorry for the delay). A separate news announcement will be done when a full ISO (MirBSD + MirGRML) is done.

Other than that, I have fixed a couple of things all over the place, jupp for example. The planned release of mksh R39b is still not done though, as I’m only human as well, and too much hacking isn’t something one can do without relaxing some in between.

On the Debian front, my RCBD #1 was continued, here’s #1½ results:

  • Carry over from day #1
  • bug #552791 – acorn-fdisk – Copyright file does not contain verbatim copy of the license or a pointer to one
  • bug #562647 – gidentd – Does not work with ipv4 after recent change in netbase
  • bug #558812 – dietlibc – incorrect license in debian/copyright
  • bug #531937 – autossh – FTBFS on mipsel due to missing -fPIC
  • New ones
  • bug #563522 – acorn-fdisk – cleanup patches
  • bug #563525 – gidentd – cleanup patches

Explanations: I did go overboard during the first patching session, but I suppose this is what the NM learning period is for too. The autossh maintainer said thanks and will probably integrate my patches, so I don’t need to NMU. I could close the dietlibc bug. The other two didn’t look as good, I had to separate the fix for the RC bug (and other required fixes, such as ftp-master rejects – there were none though) and my other fixes; I submit the former as NMU diffs again and pointed Zack to the .dsc files, and opened the aforementioned two new bugs with the rest of the diffs, so the proper maintainers can take and apply them.

There’s questioning if gidentd should be removed (see the PR for more); the acorn-fdisk upstream (arm-fdisk it’s called there) is not actively developing but will receive patches; the autossh maintainer said thanks but I didn’t yet hear back from upstream.

The binutils as intel_mode bug was fixed upstream and in experimental for my case, but I had to reopen things because the variant documented in binutils-current still doesn’t work, so others (who use the new, more intel-like, syntax) don’t run into it.

Luk sent me a request to do more mipsel-FTBFS-due-to-toolchain-bugs workarounds. Will do (but can’t promise to do so before the upcoming BSP.

Robert Millan incorporated something like manifold-boot into GRUB 2, after I described it to him (the debian-bsd@ people are currently sorting out some heisenbugs with it, though). Now there’s three variants (but then, this helps spotting bugs that don’t appear in all implementations).

sendmail 8.13.4 is out, I wonder when OgreBSD will upgrade… I could do it myself again, but this time it’s not that urgent. Still waiting for the TLS extension, though…


01.01.2010 by tg@
Tags: debian

My first RC Bug-squashing Day (or rather night):

  • bug #552791 – acorn-fdisk – Copyright file does not contain verbatim copy of the license or a pointer to one
  • bug #562647 – gidentd – Does not work with ipv4 after recent change in netbase
  • bug #558812 – dietlibc – incorrect license in debian/copyright
  • bug #531937 – autossh – FTBFS on mipsel due to missing -fPIC

I picked all of them mostly randomly from the list Zack gave me, and except dietlibc they are packages I had not even heard of before. The first one begun easily (track down licencing information, pimp debian/copyright, but I ended up fixing compiler and lintian warnings and even wrote a manpage for it while there (but for this one, I didn’t bump the Standards-Version). The second one was only the second one to complete because the others took longer; it’s basically a change of a dæmon to use two instead of one listening socket, to work with a “doble stack” OS instead of just a “dual stack” OS by not using v4-mapped IPv6 addresses (I considered if to use select(2) or poll(2), or to just fork and have two dæmons running, but that seemed too ressource-consuming to me so I chose the less-complicated poll(2) method, looking at popa3d(8)’s source code (inherited from OpenBSD) in the MirBSD tree since I could not find my network programming book. The third one was basically communicating with upstream; the bug can be blosed with no change to the package. The fourth one took me a while; luckily I have qemu 0.11.0 on MirBSD, and aurel32’s mipsel qemu images helped a great deal; however, cowbuilder --create failed for me, so I ended up waiting almost the entire night for a-g d-u to finish; in the end, it was simply a bug in upstream’s which is only exposed due to a toolchain bug on mips(el).

To do: my AM Zack needs to upload the NMUs (after checking, of course); I need to communicate some of the fixes upstream (the gidentd upstream is NXDOMAIN ☹), produce a very small testcase for the mipsel toolchain bug, maybe fix some more mipsel FTBFSen as I have a working qemu instance now, but maybe I’ll do that at the BSP when I can’t find IPv6 bugs or so that I feel I can fix (I also want to do an mksh release which has to be prepared first RSN, and there’s still the need to formally publish the MirBSD-current bi-arch snapshot and make another ISO out of it for BT and prepare the multi-BSD USB stick for…).

Annoyances: a-g d-u could ask me things at the start before working for some four+ hours instead of in the middle, and the same questions several times (PAM restart). The sid kernel doesn’t boot today but did yesterday, the lenny kernel produces this:
Starting the hotplug events dispatcher: udevdudevd[320]: udev: missing sysfs features; please update the kernel or disable the kernel's CONFIG_SYSFS_DEPRECATED option; udev may fail to work correctly
(I hate udev.) And, worst of all, these annoying fireworks (some sounded like originating from inside our staircase, I pity the neighbour’s cats) when one wants to hack… Finally, I *loathe* CDBS. Debhelper v5 rules!

Oh, and I also was under the impression that “Firstname LASTNAME” was a French thing, and to a much lesser extent Asian. (@bubulle)


27.12.2009 by tg@
Tags: debian

With the Lintian 2.3.0 saturday-after-christmas release (by the way, over here if it’s done twice it’ll really become tradition) I’ve run its spelling tests over all of MirOS CVS repository. The result: 293 kinds of typos in 35857 souce files. (Although there are the case things too. Without them, I have 51 typos in 7206 files. Asides from false positives (I used fgrep -rwl[i], and -i and -w don’t play well together, and -w mis-catches “GTK+” as “GTK”) I probably can’t (API, source code) or won’t fix all of them though.)

However, I have some rather hot asia-style food to eat right now, and will need to get up early tomorrow for work, so I am not applying/fixing them right now. (bsiegert@ and gecko2@ however are enjoying themselves at 26C3, see their wlog entries.)

Note that all of today’s fixes will not make it into the next MirBSD snapshot already, since it’s built (i386) and building X11 already (sparc). On the other hand, the next bunch of WTF *.deb files will have them. I also need to fix makefs upstream for Hurd… and continue the T&S questionnaire… *sigh*

Update: I suppose this is my “Hello, Plänet Debian!” posting (thanks aptituz!)… well, my packages in the archive were already lintian clean, in case someone wonders (I did recheck with 2.3.0 though). My point was, why not use checking tools from one “universe” for another one, viceque versa? (Similar to synergy effects from knowledge.)

I managed to create an avd "Android 2.0-current", with stuff completely built by myself. Now I "just" need to get project/external/mksh.git to be created and writable by me. Or, even better, nuke that NetBSD® ash they're currently using and replace it with a sensible shell, at least mksh-small. Then adb can be built without -DSH_HISTORY (which, with mksh, is required for usability).

I wonder if I could take over Mæmo as well... *grins*

On unrelated side notes, I'm trying to get the "debian" tagged entries aggregated on Plänet Debian, and I'm – again – in the NM process trying to become a DD, with slightly different goals this time. (But I'd also like these porting machines... 'sides, there's still an mksh+dietlibc on hppa bug open...)

I also got HP-UX back at HP PvP (not player versus player though ;) for mksh(1) porting/testing. Sadly, Itanic only, no humppa machines.

In case someone ever needs it, a collection of scripts called BitWeaver → MediaWiki does exactly that and has been released under GNU GPLv2 (only). Cheers!

Still happy with the eKey

25.11.2009 by tg@
Tags: debian

As I wrote, I asked for flute notes. Well, piano notes are ok too, although I don’t have my electric organ any longer, they can easily be transposed, even if I don’t know the software (could do it by hand though). And I might give midiplay(1) a shot (I bet it’ll sound like PC-Speaker emulation…). Vincent kindly provides more input (apparently one more of these Simtec people, but that’s just my guess).

Since ports/security/ekeyd runs happily on herc and most of my patches were not just applied but even appreciated, thanks Daniel, and the results speak for themselves (I even get stats from daily.local mailed to me every night), and we had some fun discussions, I like it. I think these whom I ordered additional ones for are, too. (I wonder if I should invest into a ten-pack bulk ones and re-sell them at conferences, but the next one is so close to the UK they probably will be there by themselves.)

I must admit I also have the context switching problem (but hey, that’s what you get for being a sysadmin, and our coffee (GEPA, ganze Bohne, im Eimer, fair gehandelt), even if not Café Libertad, who, incidentally, are Debian Wine distributors, is good), but since I’m usually not working for customer projects, I’m rarely time bound, and quite some good ideas have come from distraction (or timeouts, such as personal needs or getting coffee/food/…).

Now I still wish I could split myself in half to get more time for all the projects I have…

I am happy with my eKey

12.11.2009 by tg@
Tags: debian

Neil, I am happy with my eKey, and I would blog it if I had a blog ☺ (And yours doesn’t allow comments. But then, Daniel’s doesn’t, either.) I’d have liked proper (C flute / piano / voice) notes, though… never got the hang of string instruments.

Of course I still have to make a MirPort for that Lua dæmon, but for now, things work quite well. (I do have a rather large TODO and woke up with headaches and slight cold today.)

Due to heavy load at work, as well as some minor things, I'm either taking back interest altogether, involvement altogether, time spent on projects, or any of these on aspects/particulars of projects.

Sometimes, when you're burnt out, it's best to concentrate on living and on core projects. mksh is one of these for me, as is keeping MirBSD userland and MirPorts infrastructure working well, with small, evolving improvements (no big jumps). Other things, no matter how nice, interesting or useful (to me as well as to others) they are, need to stay back. I poured most of the last seven years of my life into MirBSD.

Sometimes, you want to give back, but it's too much effort, or you cannot afford to spend more time on it. I'll close one of my Debian ITP bugs for this reason. (I also rarely send in patches from ports for this reason, but sometimes point upstream to our CVSweb.)

Sometimes, people like Ulrich Drepper, Marco d'Itri, Gerrit Pape let you realise that every project has its Theo de Raadt-alikes. I've still not ported jupp's latest release to Debian (but an OpenSuSE Buildservice SRPM exists), nor uploaded the current mksh(1) version even to my own wtf repo. I will do so, when I feel like spending private time with Debian again, at least for the etch and lenny (and hardy – for work) branches, as dash and mksh in sid have... issues I predicted ages ago. (For one, I'm still waiting for Gerrit to contact me. Maybe our eMail systems don't like each other? Waldi or formorer will probably pass on any messages, as will the trusty BTS.) I'll probably not open any ITP bugs again and send in much less of the bugs I notice, simply because I don't like being ignored (or flamed, but sometimes, being ignored is worse – which is why Benny works on MirPorts, btw). Maybe, if I feel the need to, my wtf repo will grow instead; DDs or DMs are free to take from there if they like.

Sometimes, one realises that he just doesn't fit in. While Cachewolf is a useful project, working together with Java™ developers that communicate over web fora only and don't even see the need for compatibility with Unix or proper processes most of the time (svn:eol-style comes to mind, and switching the source code to UTF-8 is something I've given up to dream of – I would even have fixed bugs where Ewe wouldn't do UTF-8 right, but I run into a wall of bliss ignorance there) proved impossible for me. I won't budge either: web fora are simply not for me to use. Period. This is my fault (for not fitting in) as well as the fault of some of the rest of the team (for ignoring years of experience, or for simply nicht über den eigenen Tellerrand schauen (however one says this in English, I don't know) and not caring of these who do; for supporting the commercial site over the three alternatives too). I will continue to use it, maybe the iPAQ H3600 a colleague gave me proves useful, otherwise, MirBSD will do just fine.

So, when I leave or pull back a little, no prejudices. Sometimes with reason, but mostly due to lack of available resources on my part. I hope nobody who has been or will be noticing me ceasing to contribute as much as usual thinks ill that's why.

Not an mksh bug

08.04.2009 by tg@
Tags: bug debian mksh

When R37c was brought out, I fixed a bug on (among others) IA64. The simple memory allocator added a pointer (or two, in Espie's) to the storage, placed before what the user got. Of course, gcc wanted to align the struct not taking this into account, failing evilly. Luckily, another FTBFS was not my fault, but sigsetjmp(3) was merely broken on S/390 with dietlibc; waldi fixed it in the meanwhile, but I uploaded another version of mksh to Debian for now whose mksh-static binary links against glibc instead and added me a TODO bug.

All the testsuite failures are certainly interesting though; the hppa one looks like a bug in ed(1) there; as to the others, either Perl, or binfmt_misc was configured to accept or drop (but not reject) shebangs præfixed with a BOM. Whatever.

Maybe I can now finally go back to working on MirBSD instead? :D
After all, we want a new snapshot (for NetInstall, at least).

All 1 2 3 4 5 6 7 8 9 10 11 12

MirOS Logo