Developers’ Weblog

Sponsored by
HostEurope Logo

Developers’ Weblog

All 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35

tl;dr: Full JSON encoder/emitter and decoder/parser in Pure PHP now available as part of FusionForge/Evolvis under GPLv2. Do not use PHP’s built-in code, it’s broken. Read on for details and links.

I’ve pimped the minijson code in EvolvisForge to be a full-blown JSON encoder and decoder (lexer/parser) now. You wouldn’t believe just how much is broken in PHP (its own json_encode handles floats wrong in most locales, and mb_check_encoding doesn’t check the encoding of a multibyte string… at least, unlike Python, PHP manages to get 8bit okay though).

Anyway, thought you want to know. If you ever need a Pure PHP JSON encoder/decoder, you know where to look. It’s 100% my code (although written during dayjob, so the exploitation rights are with tarent GmbH). Current licence is GPLv2+ or AGPLv3+. If you spot any bugs, you know where to report them. I think it should be good though. (Do note that JSON is case-sensitive, so “NULL”, “True” and “\N” or “\U20AC” are not valid, “null”, “true”, “\n” and “\u20AC” or “\u20ac” are.)

I plan to store the “art_cust123” information in the user_prefs table in JSON now, instead of ‘|’-separated values, to avoid problems like these we had with broken database format and subsequent corruption of values, by using a dictionary (JSON Object) instead.

The ECMA 262 standard (ECMAscript and JSON) is freely available, though. JSON is also additionally specified in RFC 4627 which differs slightly (see my notes for details, mostly the goal element).

Update: fixed links

The blogs have been moved from *.blogs.evolvis.org to *.blog.tarent.de to provide proper SSL certificates. This also means that the use of blogs for people who are not employees of tarent GmbH, tarent AG or one of its subsidiaries is currently not available. (If that should become necessary, please contact us.

The feed has moved to http://evolvisforge.blog.tarent.de/feed/ for this blog (similarily to the others), but just in case, you’ll get redirected (although only to the https version, which Planetplanet doesn’t seem to like).

Planet Evolvis will be set up anew using Planet Venus software shortly.

All in all, the Greater Evolvis Platform is undergoing updates and improvements.

in-target: E: Kaputte Pakete

11.04.2011 by tg@
Tags: bug debian rant

*buntu Hardy kann zur Zeit nicht installiert werden (der Kernel (in main) dependet auf Pakete aus restrictet, das ist aber zum Installationszeitpunkt nicht aktiv und sowieso unfrei; und wieso ist eigentlich das hardy-updates Repo im d-i eingeschaltet und nicht erst hinterher?).

Lustiger aber: „Einige Pakete konnten nicht installiert werden. Das kann bedeuten, dass[sic!] Sie eine unmögliche Situation angefordert haben oder dass[sic!], wenn Sie die Unstable-Distribution verwenden, […]“

gecko2s Kommentar dazu nur, daß unstable bei *buntu stable heiße. Ich habs dann auf LTS korrigiert (ist nicht das erste Mal – und sowieso, wieso tauschen die in einer stabilen Version PostgreSQL-Majorversionen aus?) und dabei haben wir’s belassen: Debian unstable = *buntu LTS.

Naja, wie wir das letztens Simon gesagt haben (Upgrade innerhalb einer Version von *buntu auf einem Server hat grub durch grub2 ausgetauscht): Mit Debian wär’ das nicht passiert!

Various joys

26.03.2011 by tg@
Tags: debian

I’m online again. (In case you didn’t notice, duh…) Seems as if we (the Telco/ISP guy and me) just needed to look at it hard enough for it to go away – first he could dial in, using my account data, which I probably should change now, then herc with ppp(8) and pppoe(8) was working (although at about 50 KiB/s down, he showed me 508 KiB/s – a rate I had never achieved – with his WiXP), then I took my notebook, which worked with pppoe(4). Now herc’s working again. (Maybe altq(9) can explain the slowdown? Hm, from debian.netcologne.de I get 500 so it looks okay.)
But eurynome isn’t, oh the joy. Luckily, gecko2 who administers its host system just woke up.

Things we do want to see: the Telco/ISP guy accepting that I run MirBSD on a P-233MMX box with Hercules graphics card and a 9″ monitor with no comment other than considering its age (and that it usually runs 24/7) as partial cause for the bug. Thanks, Netcologne!

Things we don’t want to see:
Mar 26 10:40:02 blau /bsd: signal 11 received by (screen:16857) UID(2999) EUID(2999), parent (screen:19111) UID(2999) EUID(2999)
“Suddenly the Dungeon collapses!! - You die...” (luckily, I get it about once a year only)

ObCoffeeSpices: Marrakech (Cumin, Allspice, Cumin Aroma) – though, due to its relative strength compared to the others, the only coffee spice I have left. And another hint: pre-warming the coffee cup with hot water, so it doesn’t cool down too fast with the amounts of milk I put in, rocks.

I just wore the Squeeze Release (FOSDEM, Spacefun) T-Shirt to the bakery and got asked by a neighbour: “Oh, a Debian fan?” “Developer, even” – now imagine the typical “informed interested guy” talk for a conference booth of your OS of choice here. How proud he was to get his wife and himself Windows®-free at home; how he likes to tinker a bit (if he’s got any time left), which has become harder with Windows; how his time constraints have him at OpenSuSE currently but asked how squeeze is; and the usual complaints at places like $ork where they have to use Windows® and MSIE (apparently you can’t centrally manage Firefox, eh, good someone tells me, because that’s what we do…). Wow. Anyway, it’s spring, so people, wear your shirts. (Hrm, what do I make of the fact that this is my only Debian shirt – although I’m thinking how to get Tartan Trousers if money were no issue – and nobody had ever commented on my various BSD, FOSDEM, FrOSCon, etc. wear…)

Debugging PPPoE

24.03.2011 by tg@

⸘Did you know…

	sudo tail -f /var/log/messages &
	sudo tcpdump -ovvlns1500 -eine3 &
	sudo ifconfig pppoe0 debug up
 

… still no network ☹ but at least I got some more information, and the L-Technicians will have a look at it tomorrow as well.

’M back.

21.03.2011 by tg@

Two DNF out of four geocaches, well… one was too muggled, the other was no longer there, judging from the previous visitors’ log entries. Cached with natureshadow and bought his book on how not to cycle across Germany.

CLT was a blast, and it’s refreshing to attend an event without having to drive a booth of our own. Talked to lots of people. Since the boss was paying, even did some mingling in that area.

My ADSL line has been hiccupping ☹

Yawn.

19.03.2011 by tg@
Tags: debian event geocache grml

Will drive to Chemnitz now. Maybe meet me there. No booth, just visiting to meet everyone again, rather spontaneous.

Rhonda suggested I document how to use the LLS (Launchpad Login Service – their implementation of an OpenID provider) as Delegate, which basically means, you can put something up on your webpage, which can be a simple static (X)HTML page like mine (a /index.htm is especially nice, a /~user/index.htm works too), and use its URI and not https://launchpad.net/~me to login. For example, this often hides the LLS from view e.g. in blog comments, such as those where Canonical is being criticised ☺ – but it’s also yours, easier to type and to change if you switch service providers.

The basic idea is to go to your Launchpad user page and view its page source. Look for openid relation links in the header – on Rhonda’s the value we’re looking for is “cyLQbcp”, and you see it several times.

Now you put this on your web page:

	<!-- begin: OpenID delegation to LP -->
	 <link rel="openid.server" href="https://login.launchpad.net/+openid" />
	 <link rel="openid.delegate" href="https://login.launchpad.net/+id/cyLQbcp" />
	 <link rel="openid2.provider" href="https://login.launchpad.net/+openid" />
	 <link rel="openid2.local_id" href="https://login.launchpad.net/+id/cyLQbcp" />
	<!-- end: OpenID delegation -->
 

Of course, insert your, not Rhonda’s, ID. Do note that we don’t copy the X-XRDS-Location tag (that breaks things for some unknown reason), but otherwise, what we insert on our page is pretty much a copy of the info on the user page (maybe it’s a Delegate page, too?).

As usual, try at your own risk, bug Canonical if it breaks. It works with AO3, Gerrit Code Review, and others though (interestingly enough, better in Lynx than GUI browsers because I stay logged in across Lynx sessions (and just have to confirm sending “my information” to the accessing site), whereas I have to re-login to the LLS in every GUI browser session).

As with the LLS generally, “to access a site which is not recognised” is expected and worked on with low urgency (mostly cosmetical, I think).

A few days ago, somebody sent me an e-mail with a few questions about the Go programming language, of which I am a coauthor and contributor. The questions concerned my general impression, the experience I had in it, the future and some good areas of application. Here is my reply.

In my experience, when people try Go, there are two phases: On a first glance, it does not seem very special. However, many that do try it quickly become hooked.

I myself was primarily programming in C when I started using Go. I had written some code in Limbo, which an be seen as the antecessor of Go, so trying it seemed natural. I quickly discovered that it is a very well-designed language, whose constructs make perfect sense and are easy to grasp. The standard library is also very well done and makes many common programming jobs easy.

I decided to write a project I did for work in Go. However, the target was a Windows system, so I became involved in the Windows port. My first contributions were to the path package. It is a nice experience working with such bright and professional people as the Go developers. While they can be sometimes a bit terse in code reviews, they are very talented people. I found myself thinking a long time about a single sentence in a review before finally “getting it”.

Speaking of which, the code review process is very efficient for the development of the language.

In the future, I think Go will get a much wider adoption, and be usable also for writing “native” graphical user interface programs. There is a WinAPI port for creating Windows GUIs, and there is a very good Gtk+ adapter for Unix-like systems. I can also see it being used a lot for network servers, such as HTTP application servers and so on.

In these two areas, parallelism is a key point. In a GUI application, you want the interface to remain reactive during calculations and I/O, which is very easy to do with multiple goroutines. In a network application, you also have to process many requests in parallel. Go has the potential to scale significantly better than other languages. Maybe will even be one of the answers to the problem of programming for ever more parallel machines, as almost all new computers have multiple CPU cores.

I also think that there will be more optimization so the language will get much faster in the future. It is already faster than Python in most cases (easy, since it is a compiled language) but just as easy for many.

I am now a NetBSD developer!

11.03.2011 by bsiegert@
Tags: pkgsrc

Since 2011-03-09, I am now a NetBSD developer. My allocated work area is MirBSD support for pkgsrc. This is now much simpler, as I have direct commit access to the repo.

My sponsors are Guillaume Lasmayous (glc) and Alistair G. Crooks (agc). The whole thing would never have happened if Marc Ballmer had not started the new-developer process during FOSDEM 2011. A big “Thank you” goes out to these three persons in particular as well as to all those that warmly welcomed me on the NetBSD developer mailing list.

mksh-current has just gained an experimental recursive parser for command substitutions, fixing RedHat BZ#496791 and decades-old complaints about the pdksh codebase, compared to AT&T ksh93. (GNU bash could also do the example, but not some other things mksh(1) parses fine now.)

This means that things like the following work now.

	# POSIX, should “always” work
	echo $(case 1 in (1) echo yes;; (2) echo no;; esac)
	# POSIX optional, works now in mksh, works in GNU bash
	echo $(case 1 in 1) echo yes;; 2) echo no;; esac)
	# GNU bash seems to choke on comments ending with backslash
	# a comment with " ' \
	x=$(
	echo yes
	# a comment with " ' \
	)
	# No non-recursive COMSUB parser can pass all of the above
	# tests and these below at the same time (some extensions)
	echo $(typeset -i10 x=16#20; echo $x)
	echo $(typeset -Uui16 x=16#$(id -u)
	) .
	echo $(c=1; d=1
	typeset -Uui16 a=36#foo; c=2
	typeset -Uui16 b=36 #foo; d=2
	echo $a $b $c $d)
	# the ‘#’ is especially tricky, that’s why the above cases
 

Next on my TODO is the complete rewrite of the read built-in command, as well as its documentation. I think that the (reduced) goals for mksh R40 will have been met by then, except porting to LynxOS and MPE, but we’re working on it, and re-testing Syllable and Plan 9). Of course, a release implies testing on a lot of the supposedly supported platforms, so it won’t be out “immediately”. Though, associative arrays have been removed from the R40 goals, so that I can at least get a new release out. Note that Debian and OpenSuSE Buildservice users have been provided with somewhat well-tested mksh-current snapshots for a while already, and Gentoo users can use the “live ebuild”; there’s always compiling from source too…

(Free)BSD vs. Linux

03.03.2011 by tg@
Tags: debian event ill mksh rant

Warning: this is a rant against BSD (specifically FreeBSD®, but don’t let me get started on DragonFly, who think it’s wise to drop all shells except ash from the base system and rely on pkgsrc® – yay let’s compile a dozen packages just to get a shell with tab completion, not to mention boxen with no network access – for the task – although others seem to go into that direction too…; you know, there’s BSD, and then there’s FreeBSD…) – don’t like, don’t read.

If you want to change something in the BSD world, you gotta fork your own BSD – no other way around the thickheads. Ok, back then, I ran into a particularily thick one, but others tend to not be much better. Users share the thickness. If you want to change something in the GNU/Linux world, just make a package, have someone upload it, prod (or pay, Hanno got a Radler) people to do it, or just upload it yourself.

At the BSD booth at FOSDEM, despite me bringing the Windows® Mobile 6 Professional devive, strictly for Geocaching mind you, Macintosh boxen had a share of more than 50% – I didn’t manage to tip the scale. At the Debian booth, almost everyone had a “I want to buy a new laptop some day, but it just keeps on working and doesn’t break” pre-Lenovo IBM laptop. No hyping of Google either. (Last year’s CLT saw BSD people advocating pro-Schily – the guy with the broken encoding in his name – shockingly.)

Honestly, tcsh, FreeBSD® people? Sorry. While I agree that there is merit on having the same script and interactive shell, as someone has pointed out (copy-paste examples into the command line), there’s those zsh users who use mksh or GNU bash for scripting. Or just POSIX shell. And that’s with an interactive shell which can be used for scripting. On the other hand, the C shell (both csh and tcsh) cannot.

And what’s with pretending the accent gravis is non-combining, called “backtick” (such a thing does not exist); and advocating it? Sorry, if your csh/tcsh doesn’t handle the POSIX $(…) you should just drop it. (By the way, there is a convention that example command lines are prefixed with for csh and for sh (or but we write $ sudo  instead, these days). Use it. Or leave it. If you have examples that substitute another process’ output, be specific.) It’s funny to see how one person tries to defuse my arguments against csh by telling me “it’s just an interactive shell”, while the other argues that people copy-paste between them, to which that was my response. Read the thread!

And please, get your facts right. “I would prefer that the standard shell be at least Bourne-compatible.” You don’t want Bourne (“^” instead of “|” for pipes), you want POSIX. That GNU bash is called the Bourne-Again Shell in one of their usual semi-bad puns doesn’t help the global perception of such things any. Also, the root shell and /bin/sh are disjunct.

(Plus, why change the root shell, use sudo(8), plain and simple.)

ObNote: in jupp (should I package that for Debian, btw? rather upload, packages are ready…) the ‘`’ key is used as præfix for Ctrl-X (`X) or to directly enter numerical (decimal, octal, sedecimal/hexadecadic) ASCII, 8-bit or Unicode codepoints. Yay!
And even the FSF has seen the light; for a few releases already, GCC uses “'…'” instead of “`…'” for quoting in messages, even without locales. Great job there! (LC_MESSAGES=en_GB.UTF-8 usually works, too, though.)

ObDisclaimer: I have an (yes, Google…) Alert on the word “mksh”, so I know when it’s being discussed. This obviously includes certain fora. Also, I’m a shell implementer and bound to know a certain amount of details. Plus, mksh’s build script runs with pretty much any Bourne/POSIX/Z Shell which has functions and not too many bugs. I wrote it. Go figure. No lowly trolling.

FWIW, mksh(1) has the cat(1) builtin both because Android has no cat(1), and as speed hack. Almost all other shells have worse speed hacks, like a printf(1) builtin. And recently, builtins have become direct-callable, so this actually reduces the overall system footprint. (Its inclusion also provides for some other possibilities, internally.) And as two final side notes, if you haven’t seen this: determine which shell we are run under (CVS) and I still offer a prompt conversion service (send me any GNU bash or oksh $PS1 and I’ll send that to you in mksh(1) syntax – optionally with adjustments/improvements, like cwd uses only up to 1/3 of screen width).

All 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35

MirOS Logo