I’ve updated a lot of things in MirBSD and for use with the Debian operating system. More to come, pax(1) has been converted to Mirtoconf (the successful Build.sh system of mksh’s) but needs to be re-ported to a lot of systems (and some more bugs squished). My “WTF” APT repository also received a number of updates, such as to the ever-desired wtf(1), but it’s the time of that two-year cycle which invites general care for all of one’s packages.

On the other hand, MirBSD stops offering RSS feeds by tags. The world has become more insular, first by DSGVO, now by other cultural issues. I’ll be at FOSDEM, as usual, though, so rejoice!

You can now directly download, for all platforms and synthesisers, the soundfonts shipped in Debian for MuseScore (and others) I maintain. This service may cease at any time, without notice. Also, do mind the MIT licence.

On an unrelated note, happy new year in the western calendar!

The “properly quote eMail messages and on Usenet” documentation is hosted on a server that appears to not get too much care at the moment. I’ve dug out workable versions:

• current all-in-one-page German version (aktuelle alles-auf-einer-Seite-Fassung auf Deutsch), with a short link on http://deb.li/quote
• last archived Dutch version (laatste versie in het Nederlands), the server is currently unreachable, but the Wayback Machine still has it
• current English translation, still workable

The original link, with its http://learn.to/quote/ redirection, which contained the links to the translations into Dutch and English, unfortunately no longer works.

I’m asking everyone to please honour these guidelines when posting in Usenet and responding to eMail messages, as not doing so is an insult to all the (multiple, in the case of Usenet and mailing lists) readers / recipients of your messages. Even if you have to spend a little time trimming the quote, it’s much less than the time spent by all readers trying to figure out a TOFU (reply over fullquote) message.

Ich bitte jeden darum, sich bitte beim Posten im Usenet und Verfassen von eMails sich an diese Richtilinien zu halten; dies nicht zu tun ist ein Affront wider alle (im Falle von Usenet und Mailinglisten viele) Leser bzw. Empfänger eurer Nachrichten. Selbst wenn man zum Kürzen des Zitats ein bißchen Zeit aufwenden muß ist das immer noch deutlich weniger als die Mühe, die jeder einzelne Leser aufwenden muß, herauszufinden, was mit einer als TOFU (Text oben, Vollzitat unten) geschriebenen eMail gemeint ist.

Mag ik iederéén verzoeken, postings in het Usenet en mailtjes volgens deze regels te schrijven? Als het niet te doen is vies tegen alle ontvanger’s en moeilijk om te lezen. Zelfs als je een beetje tijd nodig heb om het oorspronkelijke deel te korten is het nog steeds minder dan de moeite van alleman, om een TOFU (antwoord boven, fullquote beneden) boodschap proberen te begrepen.

Nik wishes you to know that the Movim packaging sprint (sponsored by the DPL, thank you!) is handled under the umbrella of the Debian Edu sprint (similarly sponsored) since this package is handled by the Teckids Debian Task Force, personnel from Teckids e.V.

After arriving, I’ve started collecting knowledge first. I reviewed upstream’s composer.json file and Wiki page about dependencies and, after it quickly became apparent that we need much more information (e.g. which versions are in sid, what the package names are, and, most importantly, recursive dependencies), a Wiki page of our own grew. Then I made a hunt for information about how to package stuff that uses PHP Composer upstream, and found the, ahem, wonderfully abundant, structured, plentiful and clear documentation from the Debian PHP/PEAR Packaging team. (Some time and reverse-engineering later I figured out that we just ignore composer and read its control file in pkg-php-tools converting dependency information to Debian package relationships. Much time later I also figured out it mangles package names in a specific way and had to rename one of the packages I created in the meantime… thankfully before having uploaded it.) Quickly, the Wiki page grew listing the package names we’re supposed to use. I created a package which I could use as template for all others later.

The upstream Movim developer arrived as well — we have quite an amount of upstream developers of various projects attending MiniDebConf, to the joy of the attendees actually directly involved in Debian, and this makes things much easier, as he immediately started removing dependencies (to make our job easier) and fixing bugs and helping us understand how some of those dependencies work. (I also contributed code upstream that replaces some Unicode codepoints or sequences thereof, such as 3⃣ or ‼ or 👱🏻‍♀️, with <img…/> tags pointing to the SVG images shipped with Movim, with a description (generated from their Unicode names) in the alt attribute.)

Now, Saturday, all dependencies are packaged so far, although we’re still waiting for maintainer feedback for those two we’d need to NMU (or have them upload or us take the packages over); most are in NEW of course, but that’s no problem. Now we can tackle packaging Movim itself — I guess we’ll see whether those other packages actually work then ☺

We also had a chance to fix bugs in other packages, like guacamole-client and musescore.

In the meantime we’ve also had the chance to socialise, discuss, meet, etc. other Debian Developers and associates and enjoy the wonderful food and superb coffee of the “Cantina” at the venue; let me hereby express heartfelt thanks to the MiniDebConf organisation for this good location pick!

Update, later this night: we took over the remaining two packages with permission from their previous team and uploader, and have already started with actually packaging Movim, discovering untold gruesome things in the upstream of the two webfonts it bundles.

I’m currently working on an mksh(1) and bc(1) script that takes a pitch standard (e.g. “A₄ = 440 Hz” or “C₄ = 256 Hz”) and a config file describing a temperament (e.g. the usual equal temperament, or Pythagorean untempered pure fifths (with the wolf), or “just” intonation, Werckmeister Ⅲ, Vallotti or Bach/Lehman 1722 (to name a few; these are all temperaments that handle enharmonics the same or, for Pythagorean in out case, ignore the fact they’re unplayable). Temperaments are rule-based, like in ttuner. Well, I’m not quite there yet, but I’m already able to display the value for MuseScore to adjust its pitch standard (it can only take A₄-based values), a frequency table, and a list and table of cent deltas (useful for using or comparing with other tuners). Of course, right now, the cent deltas are all 0 because, well, they are equal temperament against equal temperament (as baseline), but I can calculate that with arbitrary and very high precision!

For outputting, I wanted to make the tables align nicely; column(1), which I normally use, was out because it always left-aligns, so I used string padding in Korn Shell — except I’m also a Unicode BMP fan, so I had F♯ and B♭ in my table headings, which were for some reason correctly right-aligned (for when the table values were integers) but not padded right when aligning with the decimal dot. So I worked around it, but also investigated.

Turns out that the desired length was used as second snprintf(3) argument, instead of, as in the right-align case, the buffer size. This worked only until multibyte characters happened. A fun bug, which only took about three minutes to find, and is covered by a new check in the testsuite even. Thought I’d share.

Feedback on and improvements for the tuner, once it’ll be done, are, of course, also welcome. I plan to port the algorithm (once I’ve got it down in a programming language I know well) to QML for inclusion in the tuner MuseScore plugin, even. Check here, for now, for my work in progress… it’s quite big already despite doing basically nothing. Foundation laid (or so…).

I’m currently working on consolidating mirsolutions.de (as my former business is long defunct) and “The MirOS Project” (that as of 2018 is also back to being my own one-man show / hobby) into www.mirbsd.org as my hobby / personal sorta homepage to not need any vhosts and simplify EU-DSGVO conformity. (I’ve also reduced logging.) Please excuse upheavals, as well as the continued presence of old, obsoleted or outdated content that may even be, as of now, completely wrong; I’ll fix it as time permits.

Today I’ve released another new CVS snapshot of the FixedMisc [MirOS] font; as usual, the tarball contains the font in BDF form, with no conflict with the system Fixed [Misc] font; sources for use (compilation, editing) with bdfctool(1) are in CVS.

The unveiling of the three new CPU bug classes, collected in the two brandbugs “Meltdown” and “Spectre”, has mostly shocked the BSDs; I’ve got it on some authority that even FreeBSD was not informed ahead of time, left alone the others. Thanks to laffer1 from MidnightBSD for a couple of heads-up warnings into our direction!

Here’s what I could gather until now (please do correct me if I’m wrong):

Meltdown is specific to Intel® CPUs with out-of-order execution, that is, all P6-class (Pentium Pro/MMX, Pentium Ⅱ, but not Pentium Ⅰ/MMX) or newer (except old Atom) CPUs. It appears to allow user processes to read kernel memory, but not across VMs, nor to attack a hypervisor. A variant for ARM exists but AMD’s x86 CPUs are supposedly safe. The KAISER/FUCKWIT/UASS/KPTI patches for Linux fix this, at huge performance cost on x86, not so much on ARM, and no cost for unaffected CPU models (runtime detected).

Spectre affects x86, ARM, POWER CPUs and possibly others. I’ve not yet found information on whether it is also limited to CPUs with out-of-order executions, but it seems likely. SPARC CPUs might be safe; Solaris/SPARC64 is safe due to the way its memory addressing works. If the OOO execution assumption is true, 80486 and P5 class x86 CPUs are also safe. This one does allow cross-VM and hypervisor attacks, so if the bare metal CPU is vulnerable, SOL. There does not yet seem to be a generic fix; some hint at having to patch the compiler and recompile everything with a workaround that has a performance cost, even if the CPU is not affected, or was fixed with a microcode update. AMD’s x86 CPUs are partially hit, one of the variants does not work on them.

“CERT recommends throwing away your CPU and buying an non-vulnerable one” (thanks to El Reg), but nobody states which CPUs are not vulnerable.

At the present time, we suggest any MirBSD/i386 instances that run on any CPU other than an 80486 or P5-class (Pentium Ⅰ or a non-PPro MMX) to be restricted to single user or trusted user access only, and no untrusted software including ECMAscript to be run on them.

Watch this space for updates. Oh, and, if you know what you’re (and I’m) talking about, please, again, do provide me with information necessary to provide those updates, both to MirBSD and to this space. Thank you.

mksh R56 was released with experimental fixes for the “history no longer persisted when HISTFILE near-full” and interactive shell cannot wait on coprocess by PID issues (I hope they do not introduce any regressioins) and otherwise as a bugfix release. You might wish to know the $EDITOR selection mechanism in dot.mkshrc changed. Some more alias characters are allowed again, and POSIX character classes (for ASCII, and EBCDIC, only) appeared by popular vote.

mksh now has a FAQ; enjoy. Do feel free to contribute (answers, too, of course).

The jupp text editor has also received a new release; asides from being much smaller, and updated (mksh too, btw) to Unicode 10, and some segfault fixes, it features falling back to using /dev/tty if stdin or stdout is not a terminal (for use on GNU with find | xargs jupp, since they don’t have our xargs(1) -o option yet), a new command to exit nonzero (sometimes, utilities invoking the generic visual editor need this), and “presentation mode”.

Presentation mode, crediting Natureshadow, is basically putting your slides as (UTF-8, with fancy stuff inside) plaintext files into one directory, with sorting names (so e.g. zero-padded slide numbers as filenames), presenting them with jupp * in a fullscreen xterm. You’d hit F6 to switch to one-file view first, then present by using F8 to go forward (F7 to go backward), and, for demonstrations, F9 to pipe the entire slide through an external command (could be just “sh”) offering the previous one as default. Simple yet powerful; I imagine Sven Guckes would love it, were he not such a vim user.

The new release is offered as source tarball (as usual) and in distribution packages, but also, again, a Win32 version as PKZIP archive (right-click on setup.inf and hit I̲nstall to install it). Note that this comes with its own (thankfully local) version of the Cygwin32 library (compatible down to Windows 95, apparently), so if you have Cygwin installed yourself you’re better off compiling it there and using your own version instead.

I’ve also released a new DOS version of 2.8 with no code patches but an updated jupprc; the binary (self-extracting LHarc archive) this time comes with all resource files, not just jupp’s.

Today, the jupprc drop-in file for JOE 3.7 got a matching update (and some fixes for bugs discovered during that) and I added a new one for JOE 4.4 (the former being in Debian wheezy, the latter in jessie, stretch and buster/sid). It’s a bit rudimentary (the new shell window functionality is absent) but, mostly, gives the desired jupp feeling, more so than just using stock jstar would.

• source tarballs
• Win32 binaries
• DOS binaries
• drop-in jupprc for JOE 2.8, or to update jupp 2.8
• drop-in jupprc for JOE 3.7
• drop-in jupprc for JOE 4.4

CVS’ ability to commit to multiple branches of a file at the same time, therefore grouping the commit (by commitid at least, unsure if cvsps et al. can be persuaded to recognise it). If you don’t know what cvs(GNU) is: it is a proper (although not distributed) version control system and the best for centralised tasks. (For decentral tasks, abusing git as pseudo-VCS has won by popularity vote; take this as a comparison.)

If desired, I can make these new versions available in my “WTF” APT repository on request. (Debian buster/sid users: please change “https” to “http” there, the site is only available with TLSv1.0 as it doesn’t require bank-level security.)

I’d welcome it very much if people using an OS which does not yet carry either to package it there. Message me when one more is added, too ☺

In unrelated news I uploaded MuseScore 2.1 to Debian unstable, mostly because the maintainers are busy (though I could comaintain it if needed, I’d just need help with the C++ and CMake details). Bonus side effect is that I can now build 2.2~ test versions with patches of mine added I plan to produce to fix some issues (and submit upstream) ☻

In other news, I’m working on a new i386+sparc MirBSD snapshot more than ever. Mostly to get everything old out from under my feet before tackling the LibreSSL import (to get TLSv1.2 support, due to the aforementioned idio…decision). I’ve yet to see whether our G++ port works on sparc, and I’ve yet to create ports for libGLU and xlock which used to be in the base X system but had to go away for being written in an unmaintainable language (plus a system is only reliable if it has only one libstdc++), but it’ll be a good stepping stone (plus mfny asked for a sparc snapshot on IRC). I was considering distributing ISOs at FrOSCon but, with an installed user base in the single digits (likely), you can imagine how useful that’d be. (Fun side idea: distribute ISOs with a boot menu where you can choose not only MirBSD installer or live system but also “minimal Debian system directly booting into the MirBSD live system running under qemu-kvm”. But I’ve got not enough spare time right now.)

As already mentioned I planned creating a new snapshot. Well, it will be out shortly, albeit in a hurried manner and not with everything I had planned for it, and with lagging sparc (as if that were new, though…). A hurried mksh release will there be as well. The reason for this is the top #1 known issue:

• Debian OpenSSL now excludes TLS < 1.2 from communication
  ⇒ there will be some followup release with LibreSSL, I think
• There’s still no port for libGLU and xlock
• We didn’t import lzlib into base yet, nor recent fixes to pax(1) from OpenBSD necessary
• The new Unicode property code is not written yet (although I fixed the data shipped so it matches, at least)
• I didn’t test g++ from ports on sparc yet, we’ll see how that goes

That being said, you’ll be able to work with what I’ve got, like in olden times when MirBSD was defined as “the contents of my /usr/src and /usr/ports” and be assured that, besides working on things like MuseScore in the meantime, I’m on it.

An unrelated minor update to another recent post; apparently I managed to make the GitHub Legal people aware enough of the problems that they are working on fixing their ToS; I admit there’s been an update since August 1ˢᵗ/2ⁿᵈ which I haven’t yet gotten around to reading at all.

wtf rocks; Eugen is working on an iOS äpp and already has a beta version which just needs bugfixing.

I was planning to do an mksh R56 release and then a full MirBSD snapshot (i386, sparc — due to actual user request — and possibly even a Live CD or at least baselive) but this got stones on my way.

I’m not quite finished with what I originally had planned for R56 (basically, the Debian postfix package’s maintainer scripts started using character classes in bracket expressions, and this required not only careful planning and design but also quite some rewriting and thinking, fixing other bugs, reading the specs, and considering EBCDIC) which led to me asking the EBCDIC porter some things again, which led to trying to merge his outstanding patches and make R56 the Mainframe Korn Shell release (also mksh ;-) but we’re not quite there yet.

The MirBSD snapshot was planned to be started from CVS as of Beltane (Walpurgis) 2017 except the latest and greatest mksh is also kinda a requirement, and CVE fixes are tricking in, to add insult to injury for stuff I had just updated. I’d also love to have the latest sendmail and lynx in it but that’ll have to wait.

I’ll also do a new CVS snapshot tarball at the same time, so keep your eyes open for the new rolling MirBSD snapshot.

Somehow, spring weather does not agree with me anyway. I didn’t get much done, nor much good sleep. Private life’s looking up but also more busy. I do manage to still help out others even in code I don’t really understand… must be good karma. Vacation would be nice… but then, I know I wouldn’t get all out of it.

Someone from the FSF’s licencing department posted an official-looking thing saying they don’t believe GitHub’s new ToS to be problematic with copyleft. Well, my lawyer (not my personal one, nor for The MirOS Project, but related to another association, informally) does agree with my reading of the new ToS, and I can point out at least a clause in the GPLv1 (I really don’t have time right now) which says contrary (but does this mean the FSF generally waives the restrictions of the GPL for anything on GitHub?). I’ll eMail GitHub Legal directly and will try to continue getting this fixed (as soon as I have enough time for it) as I’ll otherwise be forced to force GitHub to remove stuff from me (but with someone else as original author) under GPL, such as… tinyirc and e3.

My dbconfig-common Debian packaging example got a rather hefty upgrade because dbconfig-common (unlike any other DB schema framework I know of) doesn’t apply the upgrades on a fresh install (and doesn’t automatically put the upgrades into a transaction either) but only upgrades between Debian package versions (which can be funny with backports, but AFAICT that part is handled correctly). I now append the upgrades to the initial-version-as-seen-in-the-source to generate the initial-version-as-shipped-in-the-binary-package (optionally, only if it’s named .in) removing all transaction stuff from the upgrade files and wrapping the whole shit in BEGIN; and COMMIT; after merging. (This should at least not break nōn-PostgreSQL databases and… well, database-like-ish things I cannot test for obvious (SQLite is illegal, at least in Germany, but potentially worldwide, and then PostgreSQL is the only remaining Open Source database left ;) reasons.)

Update: Yes, this does mean that maintainers of databases and webservers should send me patches to make this work with not-PostgreSQL (new install/name.in, upgrade files) and not-Apache-2.2/2.4 (new debian/*/*.conf snippets) to make this packaging example even more generally usable.

Natureshadow already forked this and made a Python/Flask package from it, so I’ll prod him to provide a similarly versatile hello-python-world example package.

Please use the correct (perma)link to bookmark this article, not the page listing all wlog entries of the last decade. Thank you.</update>

Some updates inline and at the bottom.

The new Terms of Service of GitHub became effective today, which is quite problematic — there was a review phase, but my reviews pointing out the problems were not answered, and, while the language is somewhat changed from the draft, they became effective immediately.

Now, the new ToS are not so bad that one immediately must stop using their service for disagreement, but it’s important that certain content may no longer legally be pushed to GitHub. I’ll try to explain which is affected, and why.

I’m mostly working my way backwards through section D, as that’s where the problems I identified lie, and because this is from easier to harder.

Note that using a private repository does not help, as the same terms apply.

Anything requiring attribution (e.g. CC-BY, but also BSD, …)

Section D.7 requires the person uploading content to waive any and all attribution rights. Ostensibly “to allow basic functions like search to work”, which I can even believe, but, for a work the uploader did not create completely by themselves, they can’t grant this licence.

The CC licences are notably bad because they don’t permit sublicencing, but even so, anything requiring attribution can, in almost all cases, not “written or otherwise, created or uploaded by our Users”. This is fact, and the exceptions are few.

Anything putting conditions on the right to “use, display and perform” the work and, worse, “reproduce” (all Copyleft)

Section D.5 requires the uploader to grant all other GitHub users…

• the right to “use, display and perform” the work (with no further restrictions attached to it) — while this (likely — I didn’t check) does not exclude the GPL, many others (I believe CC-*-SA) are affected, and…
• the right to “reproduce your Content solely on GitHub as permitted through GitHub's functionality”, with no further restructions attached; this is a killer for, I believe, any and all licences falling into the “copyleft” category.

Note that section D.4 is similar, but granting the licence to GitHub (and their successors); while this is worded much more friendly than in the draft, this fact only makes it harder to see if it affects works in a similar way. But that doesn’t matter since D.5 is clear enough. (This doesn’t mean it’s not a problem, just that I don’t want to go there and analyse D.4 as D.5 points out the same problems but is easier.)

This means that any and all content under copyleft licences is also no longer welcome on GitHub.

Anything requiring integrity of the author’s source (e.g. LPPL)

Some licences are famous for requiring people to keep the original intact while permitting patches to be piled on top; this is actually permissible for Open Source, even though annoying, and the most common LaTeX licence is rather close to that. Section D.3 says any (partial) content can be removed — though keeping a PKZIP archive of the original is a likely workaround.

Affected licences

Anything copyleft (GPL, AGPL, LGPL, CC-*-SA) or requiring attribution (CC-BY-*, but also 4-clause BSD, Apache 2 with NOTICE text file, …) are affected. BSD-style licences without advertising clause (MIT/Expat, MirOS, etc.) are probably not affected… if GitHub doesn’t go too far and dissociates excerpts from their context and legal info, but then nobody would be able to distribute it, so that’d be useless.

But what if I just fork something under such a licence?

Only “continuing to use GitHub” constitutes accepting the new terms. This means that repositories from people who last used GitHub before March 2017 are excluded.

Even then, the new terms likely only apply to content uploaded in March 2017 or later (note that git commit dates are unreliable, you have to actually check whether the contribution dates March 2017 or later).

And then, most people are likely unaware of the new terms. If they upload content they themselves don’t have the appropriate rights (waivers to attribution and copyleft/share-alike clauses), it’s plain illegal and also makes your upload of them or a derivate thereof no more legal.

Granted, people who, in full knowledge of the new ToS, share any “User-Generated Content” with GitHub on or after 1ˢᵗ March, 2017, and actually have the appropriate rights to do that, can do that; and if you encounter such a repository, you can fork, modify and upload that iff you also waive attribution and copyleft/share-alike rights for your portion of the upload. But — especially in the beginning — these will be few and far between (even more so taking into account that GitHub is, legally spoken, a mess, and they don’t even care about hosting only OSS / Free works).

Conclusion (Fazit)

I’ll be starting to remove any such content of mine, such as the source code mirrors of jupp, which is under the GNU GPLv1, now and will be requesting people who forked such repositories on GitHub to also remove them. This is not something I like to do but something I am required to do in order to comply with the licence granted to me by my upstream. Anything you’ve found contributed by me in the meantime is up for review; ping me if I forgot something. (mksh is likely safe, even if I hereby remind you that the attribution requirement of the BSD-style licences still applies outside of GitHub.)

(Pet peeve: why can’t I “adopt a licence” with British spelling? They seem to require oversea barbarian spelling.)

The others

Atlassian Bitbucket has similar terms (even worse actually; I looked at them to see whether I could mirror mksh there, and turns out, I can’t if I don’t want to lose most of what few rights I retain when publishing under a permissive licence). Gitlab seems to not have such, but requires you to indemnify them… YMMV. I think I’ll self-host the removed content.

And now?

I’m in contact with someone from GitHub Legal (not explicitly in the official capacity though) and will try to explain the sheer magnitude of the problem and ways to solve this (leaving the technical issues to technical solutions and requiring legal solutions only where strictly necessary), but for now, the ToS are enacted (another point of my criticism of this move) and thus, the aforementioned works must go off GitHub right now.

That’s not to say they may not come back later once this all has been addressed, if it will be addressed to allow that. The new ToS do have some good; for example, the old ToS said “you allow every GitHub user to fork your repositories” without ever specifying what that means. It’s just that the people over at GitHub need to understand that, both legally and technically¹, any and all OSS licences² grant enough to run a hosting platform already³, and separate explicit grants are only needed if a repository contains content not under an OSI/OKFN/Copyfree/FSF/DFSG-free licence. I have been told that “these are important issues” and been thanked for my feedback; we’ll see what comes from this.

① maybe with a little more effort on the coders’ side³

② All licences on one of those lists or conformant to the DFSG, OSD or OKD should do⁴.

③ e.g. when displaying search results, add a note “this is an excerpt, click HERE to get to the original work in its context, with licence and attribution” where “HERE” is a backlink to the file in the repository

④ It is understood those organisations never un-approve any licence that rightfully conforms to those definitions (also in cases like a grant saying “just use any OSS² licence” which is occasionally used)

Update: In the meantime, joeyh has written not one but two insightful articles (although I disagree in some details; the new licence is only to GitHub users (D.5) and GitHub (D.4) and only within their system, so, while uploaders would violate the ToS (they cannot grant the licence) and (probably) the upstream-granted copyleft licence, this would not mean that everyone else wasn’t bound by the copyleft licence in, well, enough cases to count (yes it’s possible to construct situations in which this hurts the copyleft fraction, but no, they’re nowhere near 100%).