Planet SLUG

Dave Airlie: two X servers one graphics card.

Thu, 18 Mar 2010 01:59:57 +0000

So sane multi-seat handling was something I wanted to make KMS do at some point and designed for but never quite implemented.

So in an attempt to maybe get help out people who are interesting in this I've gotten two seats on a single card working here to a demoable level.

http://people.freedesktop.org/~airlied/multiseat/

contains a kernel patch + libdrm patch.

The kernel patch pretty much contains 3 pieces:

(a) ability to create "render" device nodes with an attached list of output resources it controls (crtcs/encoders/connectors).
(b) hardcoded render node setup for my X1900 - two parts - core drm creates 3 devices nodes, radeon driver assigns hardcoded
resources to the nodes - in this case render node 0 gets a crtc + DVI + encoders, and node 1 gets the other crtc/DVI/encoders, and
render node 2 gets no outputs.
(c) drm mapping fixups for multiple device nodes - this is something we should probably cleanup independently of this patch.

the libdrm patch just contains support to use an env var to pick the device path.

With this xorg.conf and the two startx wrappers I can run two X servers separately.

TODO:
(a) define a kernel/user interface to set seats and nodes up. The DRM control node is there specifically for this purpose but I never got around to specifying this interface. It basically needs a few methods:
1. Create new render node with output configuration.
2. Remove render node.
These would have to rely on their being no users of the render or legacy device nodes in advance. The kernel would
also have to get the driver to validate the output configuration. The output configuration would be a list of IDs for crtcs/encoders/connectors.

(b) maybe add a drm device path to xorg.conf so each card section can specify one, would help get away from BusID also.

(c) make a sane userspace interface to use it all - I suspect you'd need something in gdm/ConsoleKit to configure this sort of
thing, you'd have to construct per-card multi-seat profiles with a list of the outputs and stuff you want on each seat etc.

At this point I'm just trying to flesh out my backlog of projects and figure out how long they will take to do properly, feel free if someone is interested in picking this up and running with it.

Sonia Hamilton: Australian BJJ Competitions

Wed, 17 Mar 2010 23:54:06 +0000

A calendar of Australian BJJ Competitions I setup in Google Calendar.

If I’ve missed any, please email me or add a comment.

Michael Fox: VMware Workstation 7.1 Beta

Wed, 17 Mar 2010 05:26:38 +0000

VMware Workstation 7.1 Beta announced. More details can be found at the link below;

http://communities.vmware.com/community/beta/ws

Robert Collins: LCA 2010 videos are showing up

Wed, 17 Mar 2010 04:08:14 +0000

Not all the videos are there yet, but they are starting to show up . Yay. See http://mirror.internode.on.net/pub/linux.conf.au/2010/index.html or your local LA mirror.

Michael Fox: Fusion 3.1 Beta now available

Wed, 17 Mar 2010 01:46:25 +0000

Looks like VMware has pushed out VMware Fusion 3.1 Beta. You can grab a copy from the link below;

http://communities.vmware.com/community/beta/fusion

Mark Greenaway

Wed, 17 Mar 2010 00:17:03 +0000

If you're doing any sort of applied statistics at all, you'll soon be knee deep in diagnostic plots, data and code. Not long after that, you'll be wanting a second monitor.

James Purser: Inspiring and Depressing at the same time

Tue, 16 Mar 2010 21:54:05 +0000

Last night I finished watching a series called "From the earth to the moon". I think it was the first of Tom Hanks' forays into docu-drama (coming after the movie Apollo 13).

As you might guess from the title the series follows the events leading up to the first landing on the moon and afterwards to the final Apollo mission in 1972. As a series it is an excellent examination of the people involved in the American Space effort, from Astronauts to their wives, to the contractors and everyone else involved.

What it did for me was to bring home how far short we have fallen from the promise offered by the Apollo missions themselves. If the momentum had been maintained, Apollo could have been the stepping stone to the next wave of human exploration. By now we wouldn't be marvelling at the staying power of a couple of Rovers on Mars, but instead we'd be marvelling at the fact that the human species had set foot on another planet, not just our celestial partner, but a whole other world.

Today we are told that it's better to send probes and rovers to other worlds. Less of a risk, cheaper and so on. Feh I say. The USSR sent a probe to the moon, then the US sent two men to the moon, which is more inspiring? Which is more likely to stir the blood?

So yes, "From the earth to the moon" is an inspiring series, it details what can be done when there is the will to move forward and achieve something great. However it's also depressing to see where the evaporation of that will has left us.

Nick Jenkins: Recovering from a corrupted MySQL install due to a dying hard disk

Tue, 16 Mar 2010 12:05:18 +0000

Might as well write these steps down in case I ever need them again:

Background: A test box started making a faint high-pitched squealing sound, and then powering off. Happened semi-randomly, but most commonly during periods of hard disk access (such as boot-up). Fixed the hardware problem by replacing the Power Supply Unit ($25 from MSY).

Then the above PSU semi-random-poweroff problem in turn caused the box’s 5-year-old hard disk to start playing up (age, plus having the power repeatedly die mid-write probably doesn’t help any).

Steps for moving to a new hard disk:

Buy a new hard disk of equal or larger size.
Install it in one of the USB external single hard disk enclosures that supports both an SATA or an IDE hard disk ($23 at MSY).
Download, burn, and boot from System Rescue CD (v1.4.0 is the current latest).
After it boots, plug in USB drive.
See which disk has which device name: fdisk -l
Recover from the old to the new disk: ddrescue -b 2M /dev/hda /dev/sda ./ddres.txt
The above copied about 17 Mb per second, and claimed zero disk errors were found.
Poweroff, and swapped the old disk out of the machine and the new disk in.
Boot the new disk, was dropped into a shell during boot due to finding file system errors carried over from the old disk. Run fsck on the affected partion: fsck /dev/hda8 -y
Reviewing the SMART warnings in the syslog from the old disk seemed to indicate that it probably was dying, confirming that swapping the disks was the correct course of action.

Then found the previous HDD corruption had in turn corrupted a MySQL database (aren’t cascading failures great?). This manifested itself as at least 10 different MySQL errors/warnings/problems:

Error in /var/log/syslog when starting mysqld: Failed to open log (file ‘/var/log/mysql/mysql-bin.000348′, errno 2) The cheat was to delete the last line, the one referencing the /var/log/mysql/mysql-bin.000348 file, from the /var/log/mysql/mysql-bin.index file. Note that in this case I knew that the last log file contained no updates that mattered, so it really was no loss.
vim /var/log/mysql/mysql-bin.index

MySQL server would no longer start, instead giving a “Fatal error: Can’t open and lock privilege tables: Can’t find file: ‘host’ (errno: 2)” message in the logs. For me, the problem was that the /var/lib/mysql/mysql/host.MYI file was missing. What fixed it: Repair the host table.
mysqld_safe --skip-grant-tables & mysql mysql> use mysql mysql> REPAIR TABLE host USE_FRM; mysql> exit

Try to reset the host table with useful starting data, to fix this error when starting mysql client: error: ‘Access denied for user ‘debian-sys-maint’@'localhost’ (using password: YES)’
mysql_fix_privilege_tables mysqladmin shutdown

To fix these errors in /var/log/syslog :
[ERROR] /usr/sbin/mysqld:Fatal error: Can’t open and lock privilege tables: Table ‘./mysql/db’ is marked as crashed and should be repaired
[ERROR] /usr/sbin/mysqld: Table ‘./mysql/db’ is marked as crashed and should be repaired
cd /var/lib/mysql/mysql myisamchk db myisamchk *.MYI

To fix these warnings in the syslog:
myisamchk: warning: Table is marked as crashed
MyISAM-table ‘db.MYI’ is usable but should be fixed
myisamchk -r db

To fix this error in /var/log/syslog :
[ERROR] /usr/sbin/mysqld: Incorrect information in file: ‘./mysql/tables_priv.frm’
This did not work: repair table tables_priv USE_FRM;
Cheated: just copied /var/lib/mysql/mysql/tables_priv.* from another working machine.
chmod -x,o-r,g+w tables_priv.* chown mysql.mysql tables_priv.*

Recurrence of this error:
ERROR 1045 (28000): Access denied for user ‘debian-sys-maint’@'localhost’ (using password: YES)
And a new one:
Access denied for user ‘root’@'localhost’ (using password: NO)
… and at this point a “desc user;” showed that the user table file must have had a doubly-claimed inode with another table during the fsck, as it was a completely different user table schema from that found on another machine.
Cheated again: just copied /var/lib/mysql/mysql/user.* over from another working machine.
chown mysql.mysql user.* /etc/init.d/mysql start

Then to fix: ERROR 1045 (28000): Access denied for user ‘debian-sys-maint’@'localhost’ (using password: YES)
cat /etc/mysql/debian.cnf
Copy the “password” field’s value for the “debian-sys-maint” user to the clipboard.
mysql mysql> use mysql mysql> GRANT ALL PRIVILEGES ON *.* TO 'debian-sys-maint'@'localhost' IDENTIFIED BY 'insert_password_copied_above_from_clipboard' WITH GRANT OPTION; mysql> exit

To fix this warning in /var/log/syslog: WARNING: mysqlcheck has found corrupt tables
Force a check of all tables:
mysqlcheck -A

Fix for this error when granting privileges: ERROR 126 (HY000): Incorrect key file for table ‘./mysql/db.MYI’; try to repair it
mysql mysql> GRANT ALL PRIVILEGES ON dbname.* to 'dbuser'@'localhost' IDENTIFIED BY "fakefake"; ERROR 126 (HY000): Incorrect key file for table './mysql/db.MYI'; try to repair it mysql> use mysql mysql> REPAIR TABLE db USE_FRM; +----------+--------+----------+----------------------------------------------------+ | Table | Op | Msg_type | Msg_text | +----------+--------+----------+----------------------------------------------------+ | mysql.db | repair | info | Wrong bytesec: 255- 37- 32 at 0; Skipped | | mysql.db | repair | info | Found block that points outside data file at 424 | .... | mysql.db | repair | info | Found block that points outside data file at 24960 | | mysql.db | repair | status | OK | +----------+--------+----------+----------------------------------------------------+ 151 rows in set (0.01 sec) mysql> GRANT ALL PRIVILEGES ON dbname.* to 'dbuser'@'localhost' IDENTIFIED BY "fakefake"; Query OK, 0 rows affected (0.00 sec) mysql> exit mysqladmin shutdown /etc/init.d/mysql start

To fix these errors in the syslog on mysqld startup:
/etc/mysql/debian-start[4592]: ERROR 1017 (HY000) at line 116: Can’t find file: ‘columns_priv’ (errno: 2)
/etc/mysql/debian-start[4592]: ERROR 1017 (HY000) at line 516: Can’t find file: ‘proc’ (errno: 2)
mysql mysql> use mysql mysql> REPAIR TABLE proc USE_FRM; mysql> REPAIR TABLE columns_priv USE_FRM; mysql> exit

… and after all the above, the box powers on, and stays on, the disk errors are gone, the mysqld service starts cleanly, and from a quick cursory glance, the data still looks okay.

Ian Wienand: Handling hostnames, UDP and IPv6 in Python

Tue, 16 Mar 2010 04:54:00 +0000

So, you have some application where you want the user to specify a remote host/port, and you want to support IPv4 and IPv6.

For literal addresses, things are fairly simple. IPv4 addresses are simple, and RFC2732 has things covered by putting the IPv6 address within square brackets.

It gets more interesting as to what you should do with hostnames. The problem is that getaddrinfo can return you multiple addresses, but without extra disambiguation from the user it is very difficult to know which one to choose. RFC4472 discusses this, but there does not appear to be any good solution.

Possibly you can do something like ping/ping6 and have a separate program name or configuration flag to choose IPv6. This comes at a cost of transparency.

The glibc implementation of getaddrinfo() puts considerable effort into deciding if you have an IPv6 interface up and running before it will return an IPv6 address. It will even recognise link-local addresses and sort addresses more likely to work to the front of the returned list as described here. However, there is still a small possibility that the IPv6 interface doesn't actually work, and so the library will sort the IPv6 address as first in the returned list when maybe it shouldn't be.

If you are using TCP, you can connect to each address in turn to find one that works. With UDP, however, the connect essentially does nothing.

So I believe probably the best way to handle hostnames for UDP connections, at least on Linux/glibc, is to trust getaddrinfo to return the sanest values first, try a connect on the socket anyway just for extra security and then essentially hope it works. Below is some example code to do that (literal address splitter bit stolen from Python's httplib).

import socket

DEFAULT_PORT = 123

host = '[fe80::21c:a0ff:fb27:7196]:567'

# the port will be anything after the last :
p = host.rfind(":")

# ipv6 literals should have a closing brace
b = host.rfind("]")

# if the last : is outside the [addr] part (or if we don't have []'s
if (p > b):
    try:
        port = int(host[p+1:])
    except ValueError:
        print "Non-numeric port"
        raise
    host = host[:p]
else:
    port = DEFAULT_PORT

# now strip off ipv6 []'s if there are any
if host and host[0] == '[' and host[-1] == ']':
    host = host[1:-1]

print "host = <%s>, port = <%d>" % (host, port)

the_socket = None

res = socket.getaddrinfo(host, port, socket.AF_UNSPEC, socket.SOCK_DGRAM)

# go through all the returned values, and choose the ipv6 one if
# we see it.
for r in res:
    af,socktype,proto,cname,sa = r

    try:
        the_socket = socket.socket(af, socktype, proto)
        the_socket.connect(sa)
    except socket.error, e:
        # connect failed!  try the next one
        continue

    break

if the_socket == None:
    raise socket.error, "Could not get address!"

# ready to send!
the_socket.send("hi!")

Erik de Castro Lopo: GHC 6.12.1 in Debian Testing.

Sun, 14 Mar 2010 20:02:01 +0000

Joachim Breitner recently announced on the Debian Haskell mailing list that version 6.12.1 of the Glasgow/Glorious Haskell compiler was about to transition from Debian unstable to Debian testing. That has now happened. This means there is a very good chance it will be part of the next stable release of Debian.

A big thanks is due to Kari Pahula, the Debian maintainer for GHC who managed to get this version of GHC working on a bunch of CPU architectures not officially supported by the upstream GHC maintainers. Deserving of equal attention are Joachim Breitner and Marco Túlio Gontijo e Silva who did a large amount of real quality work to improve the way Haskell libraries are packaged in Debian.

The big change recently was drastic improvements in the way library dependencies are tracked across packages which will make it much easier to write tools to automatically check for broken dependency chains. Packaging Haskell libraries for Debian is now a relatively trivial and fool proof exercise. Packaging a library which is on Hackage can take as little as 5 minutes.

With the current version of GHC in Debian and a large and growing collection of Haskell libraries, writing Haskell code on Debian using nothing but Debian packages is now a pleasure. Ubuntu and other Debian and Ubuntu derived distributions of course also benefit from this work.

Michael Fox: iPhone 3GS

Sun, 14 Mar 2010 09:04:23 +0000

Picked up an iPhone 3GS to replace that of my iPhone 3G that I already have had for sometime. The existing phone was purchased outright and was locked with Telstra. It was on a Telstra cap, and the amount it cost me was certainly not competitive.

I managed to get a new phone on a plan that includes more for less then I was paying with Telstra using my own phone. Go figure.

iPhone 3GS seems a bit different so far, in fact better then previous model. Had considered to get something else, but kept coming back to the iPod which can be used for audio/video playback. Both handy features for a phone device all in one unit.

Erik de Castro Lopo: FP-Syd #22.

Sat, 13 Mar 2010 02:01:09 +0000

On Thursday February 25th we held the first meeting for 2010 of the Sydney Functional Programming group. The meeting was held at Google's Sydney offices and we had 17 people show up to hear our two presenters.

First up we had your correspondent (Erik de Castro Lopo), giving a presentation titled "Hacking DDC" on my bug fixing work on Ben Lippmeier's DDC compiler. I explained a little about what DDC and Disciple were; a Haskell like language with some interesting extensions to the type system. I then suggested that anyone curious as to why these extensions were interesting should read the first chapter of Ben's PhD thesis "Type Inference and Optimisation for an Impure World". I then went on how using Darcs for the revision control made it easy to use one branch per bug or feature I'm working on, specifically, it allowed me to work on one until I got stuck and then move on to another without the debugging of the first interfering with the second.

Our second presenter for the evening was Tim Docker who gave us an explanation of a Domain Specific Language (DSL) for handling dates in financial systems. Code written in his DSL looked a lot like Ocaml, but the implementation was in C++.

A big thanks to Tim for presenting and Google for providing the meeting venue and the snacks.

Dave Airlie: GPU offloading - PRIME - proof of concept

Fri, 12 Mar 2010 06:16:59 +0000

THIS IS A PROOF OF CONCEPT - its not going to be upstream unless someone else dedicates their life to it, (btw anyone know anyone in ASUS?)

So NVIDIA unveiled their optimus GPU selection solution for Windows 7, so I decided to see what it would take to implement something similar under DRI. I've named it PRIME for obvious reasons.

Goals:
1. Allow a second GPU to render 3D apps onto the screen of the first, pickable from the client side.
2. Just target the rendering side, I'm assuming the GPU power up/down is similiar to what was done for the older switching method.

Restrictions + limitations:
1. Must have compositing manager running
2. Must have second screen configured for slave card (doesn't need to be used)

Test system:
Intel 945 IGP + radeon r200 PCI card - yes this won't be a speed demon.

Terms:
Master: the IGP displaying the output - intel
Slave: the GPU rendering the app - radeon r200 in this case.

Step 1: kernel support

http://git.kernel.org/?p=linux/kernel/git/airlied/drm-testing.git;a=shortlog;h=refs/heads/drm-prime-test
http://cgit.freedesktop.org/~airlied/drm/log/?h=prime-test

The kernel requirements were simple, we needed a way to share a memory managed object between two kernel device drivers.
The kernel has a GEM namespace per device, however this isn't good enough to share with other devices, so I introduced a new PRIME namespace with two ioctls. One ioctl allows the master device to associate a device buffer handle with a name in the prime namespace, and the other allows the slave device to associate a prime namespace handle with a buffer. When the master creates a prime buffer the kernel associates the list of pages with the handle, and when the slave looks up the same handle it retrieves the list of pages and fakes up a TTM buffer populated with those pages as backing store. I've added the concept of slave object to TTM to allow for this.

The drm repo contains the API wrappers + intel + radeon pieces to call the association functions for buffer objects.

Step two: DRI2 Protocol
http://people.freedesktop.org/~airlied/prime/0001-dri2proto-add-prime-token.patch
http://people.freedesktop.org/~airlied/prime/0001-prime-support-for-mesa.patch

From the X server point of view a recent change to the DRI2 layer allowed for multiple device driver names to be associated with a DRI2 end point. The client can request either a DRI or VDPAU device name currently. I firstly extended the DRI2 protocol, to add a new buffer type, called PRIME, and added a hack to mesa's glx loader to request the prime driver if an environment variable was specified.

Step 3: X server DRI2 module + drivers

http://people.freedesktop.org/~airlied/prime/0001-intel-add-prime-master-support.patch
http://cgit.freedesktop.org/~airlied/xf86-video-ati/log/?h=prime-test
http://people.freedesktop.org/~airlied/prime/0001-dri2-prime-hackfest.patch

This was the messiest bit and still requires a lot of change. First up I added an interface for the drivers to register as PRIME master and slaves. Intel driver registers as master, radeon as slave for my demo. We store these in an array. When a client connects and requests prime driver, we mark the drawable and redirect the dri2 buffer creation requests to the slave screen driver. Also the drm authentication is sent to both kernel drms. It then hooks the swapbuffers command where it does a region copy, and redirects this to the slave driver, and damages the pixmap in the master driver. Now the "interesting" part, my original implementation simply grabbed the window pixmap at the dri2 create buffers time, however there is an ordering issue with compositing, this pixmap is pre-composite redirection so isn't actually the pixmap you want to tell the kernel to bind to both gpus. This turned out to function badly, I could see gears all stretched over the front buffer.

So a quick coke + chocolate break later, I had enough sugar to bash out the hack that now exists. DRI2 calls the slave driver copy region callback, which checks if the drawable pixmap is on the same screen, if its not, it checks if we've marked the pixmap as a prime pixmap (i.e. one that belongs to the master). It is, it swaps in the slaves copy, otherwise it callsback into DRI2. This callback calls the Intel driver to make the buffer object backing the pixmap, shareable, and returns the handle,then calls into radeon with the handle to create a new pixmap pointing at the shared buffer object. Once all that is done, radeon copies the back buffer to the shared front pixmap, we return and damage is posted and the compositor grabs the window pixmap and displays it.

So does it work?
On my blistering fast test system with X + xcompmgr running glxgears was going at 150fps from the r200 PCI card. Hopefully I can get some time on a faster system or one of the dual laptops.

Caveats:
- When a window manager is running the gears get all corrupted, this looks like the clipping and/or stride matching between
the drivers isn't correct. I suspect something with reparenting and decorations, I'm not enough of an X guru to understand this yet, hopefully one of the other hackers can fill me in. Also before it gets reparented and redirected a frame can land on the real front buffer, again clipping should take care of this, but isn't working yet. I need to workout how clipping and that stuff works in X/DRI2. - talk to ppl about clipping then JDI.
- Once a client has connected as a prime, we don't tear it down properly, so later clients can end marked as prime. - work out some sort of resources to turn stuff off
- Reference counting on the pages in the kernel is iffy, currently i915 ups the page list refcount but never drops it. solution JDI
- hardcoded /dev/dri paths in dri2 for slave device - solution JDI
- radeon driver could in theory be a prime master - solution JDI
- nouveau could support prime master/slave also. - solution nouveau guys JDI
- requires an ugly second screen in xorg.conf to load the slave driver. Can we have a 0 sized screen or maybe a rootless second screen. - solution : rearchitect X server to allow drivers without screens (6m-1yr work)
- pageflipping needs to be hacked off in intel driver. - work out and then JDI

Where is the video?
Once I get it working with a window manager on a useful machine I might do a video of two gears going.

Where now?
Well this is a purely academic exercise so far, after a week of kernel fighting I decided to do something new and cool. To make this as good as Windows we need to seriously re-architect the X server + drivers. At the moment you can't load an X driver without having a screen to attach it to, I don't really want a screen for the slave driver, however I still have to have one all setup and doing nothing and hopefully not getting in the way. We'd need to separate screen + drivers a lot better. Having some sort of dynamic screens would probably fall out of this work if someone decides to actually do it.

The kernel bits aren't as ugly as I thought but I'm not sure if upstreaming them is a good idea without the others bits. The refcounting definitely needs work also the cleanup when clients exit.

DRI2 needs some more changes, I might try and flesh it out a bit more and then talk to krh about a sane interface.

I'm probably going to get forced task switch quite soon, so I might just get to having this running on a W500 or T500, before dropping it for 6 months, so if anyone wants a neat project to play with and has the hw feel free to try and take this on.

ASUS feel free to send me one of the real optimus laptops and I'll get nouveau guys hooked up and try and RE the nvidia DMA engine.

Mark Greenaway: This is just amazingly silly

Thu, 11 Mar 2010 23:55:49 +0000

Assemblyman seeking to ban all salt in restaurant cooking

Erik de Castro Lopo: Intel Embedded Graphics Driver Fail.

Thu, 11 Mar 2010 10:01:36 +0000

In my day job I do Linux embedded work and as people in the embedded world know, Linux is a pretty commonly used embedded OS. Today I was evaluating a new board and found it had an Intel graphics chip that was not properly detected by Ubuntu 9.10. The ever trusty lspci said this:


  00:02.0 VGA compatible controller: Intel Corporation System Controller Hub
        	(SCH Poulsbo) Graphics Controller (rev 07)

We all know that Intel employs a bunch of well known Xorg developers, so this shouldn't be a problem, right?

Unfortunately, it is a problem. Intel's offering for this chipset is the Intel® Embedded Graphics Drivers web page where they offer a 124 megabyte download (registration required). After registration you get to choose which driver pack you want and which OS you are downloading it for. Ubuntu was not on the list and neither was Debian. I chose Fedora 10 (released in 2008) as that was the most recent one.

Now, you can image my surprise when the driver download for Fedora Linux contained just four files:


  Archive:  /tmp/IEGD_10_3_GOLD.zip
      testing: UsersGuide_10_3_1525.pdf   OK
      testing: IEGD_10_3_GOLD_1525.exe    OK
      testing: IEGD_SU_10_3_GOLD.pdf      OK
      testing: RELNOTES_10_3_1525.txt     OK
  No errors detected in compressed data of /tmp/IEGD_10_3_GOLD.zip.

Yep, thats right, the driver download for Fedora Linux contains two PDF files, a text file and an executable installer for Windows.

Being the curious (and paranoid) type I decided to explore this further, by running the installer under WINE in a chroot. After the installer you get left with several metric craploads of Java Jar files, and another windows executable iegd-ced.exe that supposedly configures this nightmare. I ran it (again, under WINE in a chroot) but it didn't seem to do anything sensible or worthwhile so I looked around amongst the other installed files and found IEGD_10_3_Linux.tgz.

Inside that tarball there are a bunch of Xorg library binaries (for several different versions of Xorg), a large chunk of source code that gets compiled into the Linux kernel and even better yet, a couple of Microsoft Visual Studio project files. WTF?

Unbe-fscking-lievable. Needless to say, I will avoid any hardware which uses this chipset and any other hardware that requires binary only kernel blobs packaged this badly. Doing so makes my life easier.

The people at Intel who thought this was a good idea must have their own personal mother-lode of stupid.

Mark Greenaway

Thu, 11 Mar 2010 05:48:04 +0000

Time to learn Javascript properly.

Simon Rumble: Sneaky trick to de-obfuscate Omniture JavaScript plugins

Wed, 10 Mar 2010 23:06:06 +0000

My current job involves working extensively with Omniture products. The company has an annoying habit of secrecy, with documentation only available on request for many aspects of their products.

They also attempt to obfuscate their JavaScript, despite the fact that a determined viewer should be able to work it out eventually. I'm told this is so that people aren't tempted to play with the code. The obvious methods of deobfuscation are pretty tedious, and because Omniture don't use standard (minify et al) methods of obfuscation so it seems a little more difficult. Fortunately I lucked onto a better approach.

You'll need Firebug, and if you don't have that already you should anyway. Go to a page that already has the Omniture "plugin" (function) you want. Open the Firebug console and run alert(s.functionNameYouWant) and run it. You'll be shown a nicely-formatted anonymous function, which will be much easier to read than the line noise you'll see in the actual s_code.js file.

In my case I'm after Cross-Visit Participation, and that's used on the Omniture site itself (though an older version than the latest available from Omniture which has a very useful additional feature).

Contact me

Simon Rumble: UI fail from Exetel

Mon, 08 Mar 2010 01:12:12 +0000

Work is providing me a mobile, so I went to cancel my phone with Exetel. Unfortunately this is the UI you see. So first of all, you can helpfully cancel it in the past. But then the button is labelled "Cancel". So does that means clicking it will cancel my service, or cancel the request to disconnect?

The resulting page is even more confusing. Does that mean my "Cancel" was successful? Or do I now need to "Submit" to make it happen? Terribly confused.

Contact me

Michael Fox: Traxxas Rustler VXL damage

Sat, 06 Mar 2010 23:17:11 +0000

Ran the Traxxas Rustler VXL yesterday and managed to have a wall jump in front of me.

The damage this time was a shock coming apart. Good news is it wasn’t damaged to the point I couldn’t fix it. Pulled apart, added new oil. Rebuild shock. Reinstalled back on car.

All good for another use.

Michael Fox: Amazon books arrive

Sat, 06 Mar 2010 01:51:47 +0000

The amazon book order arrived yesterday, just had to go pick up from local post office as I had a card left for me. As I wasn’t home when the package arrived.

Books shipped in good condition. Very happy with the service. Now I just need to read and study the content.

Michael Fox: Transition to Wordpress MU

Fri, 05 Mar 2010 19:38:28 +0000

Transition to Wordpress MU seems to have gone fairly well. The product seems to perform exactly like the normal Wordpress. Although one exception would appear that embedded media doesn’t work unless you install a plugin. I’ve since done that and it seems to now work as expected.

With that said, I think my blog will live in the new location. As I can now manage Wordpress MU and it’s updated for all associated blogs hosted off the users.heimic.net vhost.

Ian Wienand: RFC3164 smells

Fri, 05 Mar 2010 12:23:00 +0000

From RFC3164, which is otherwise about syslog formats:

6. Security Considerations

An odor may be considered to be a message that does not require any acknowledgement. People tend to avoid bad odors but are drawn to odors that they associate with good food. The acknowledgement of the receipt of the odor or scent is not required and indeed it may be the height of discretion to totally ignore some odors. On the other hand, it is usually considered good civility to acknowledge the prowess of the cook merely from the ambiance wafting from the kitchen. Similarly, various species have been found to utilize odors to attract mates. One species of moth uses this scent to find each other. However, it has been found that bolas spiders can mimic the odor of the female moths of this species. This scent will then attract male moths, which will follow it with the expectation of finding a mate. Instead, when they arrive at the source of the scent, they will be eaten [8]. This is a case of a false message being sent out with inimical intent.

...

Along the lines of the analogy, computer event messages may be sent accidentally, erroneously and even maliciously.

This smells more like "I bet nobody ever really reads this RFC, let's put some stuff in the middle to see if they do!".

James Purser: Dear TV Networks - Sigh

Fri, 05 Mar 2010 00:01:20 +0000

Okay, Freeview (the group that represents all of the major Free to Air Networks, including commercial and government owned) has announced that it's going to be launching a new Electronic Programme Guide service in June, complete with new hardware and something they are calling an Online Catchup Service.

Apart from the Online Catchup Service (which I will be talking about shortly) the biggest selling point that Freeview seems to be pushing for this new service is the ability to record based not only on time blocks, but by genre and programme as well.

This is what 250 million dollars buys you these days?

From the article and the reading around that I've done, it appears that the Free To Air Television industry (somewhat bizzarley aided by the ABC, an organisation that has proven time and again that it knows where the future lies) has decided that the best way to tackle the threat of extinction is to re-arrange the deck chairs on the titanic. People have had the ability to record programmes based on genre and programme titles, episode titles and so on for a few years now. If they haven't been building their own PVR's via projects like MythTV (my personal favourite), they've been using off the shelf solutions such as TiVO. It's not new people, it's old and people have been doing it for a long time now.

Hell I was there when digital came to a certain regional television network, I helped setup their first EPG generator and I can tell you that the Standard of the day, derived from the European DVB-T EPG stuff required genre information to be sent out.

Oh yes, there is something new about this particular EPG service. It's not going to be standard. In order for you to take advantage of it and the attached "Online Catchup Service", you are going to need to buy new hardware. Your set top box isn't going to be able to use any of the new features, and certainly your plasma/led/lcd tv with built in HD tuner isn't going to be able to utilise the new features being offered. Nope, you're going to have to go out and spend another couple of hundred bucks so that you can make sure you don't miss that 6 month old episode of Big Bang Theory or which ever is your Television poison.

Re the "Online Catchup Service" that appears to be a service that allows you to catchup with programmes that you've missed, online. Though I'm not quite sure if you're going to need to "catchup" given that you're going to have teh cool new "recording by genre and programme" features of the new EPG Service.

Sigh.

Sonia Hamilton: BJJ for women

Thu, 04 Mar 2010 01:52:38 +0000

I’ve been doing BJJ (Brazilian Jiu Jitsu) for years. It’s a great sport and form of self-defence, but in Australia it’s dominated by men – probably putting a lot of women off starting.

This is unfortunate, as I think BJJ is one of the best forms of self defence women can do; statistically violence against women is less likely to be the “pub punchup” that guys get in and more likely to be a grab/sexual assault (either in public or in the home). In these situations, reflexively knowing how to escape from someone’s grips or fight on the ground with someone stronger and heavier on top of you is invaluable, and just by training with a stinky, sweaty guy on top of you you’re much less to likely to panic and escape the situation (The Bene Gesserit Littany against Fear – Dune).

But BJJ is also a great sport! You get a great workout (especially abs), it’s exciting and you need to think a lot (BJJ is often called “physical chess”). You share the pain and triumphs, the sweat and exhilaration, the interstate and overseas trips with your team mates, and form close friendships.

I’m now training at The Dojo in Bondi Junction with Daniel Sainty. Amongst the guys there’s 4 other women training – Eleni, Kunita, Laura and Sam. They’re all inspirational, but the one who really gets me back to training when my middle-aged joints are creaking is Sam. She’s a petite high school girl, self-described “girly-girl” and geek. She fights against the boys, gives (and receives) a good thrashing, is starting to win competitions, and is now blogging about her training. You go girl!!

BJJ is so effective that the US Army now uses it as the foundation for their Army Combatives course. Not because they want their soldiers to throw down their rifles and start up UFC-style fights with “the baddies”. Rather, they found that teaching a random collection of moves (this kick, this punch, this other kick) was ineffective for learning a “combat mindset”. The sport nature of BJJ allows for “an avenue or the motivation for continued training” as well as internalizing the “concept of a hierarchy of dominant positions”. And knowing how to rear naked choke when someone jumps on you and hangs on to your rifle is kind of handy…

See you on the mat sometime

PS a great website on Women & BJJ – Women << BJJ Girl (especially Resources)- thanks Slideyfoot.

Sonia Hamilton: Serial Port Access on OSX to Cisco devices

Mon, 01 Mar 2010 06:07:57 +0000

How to connect to a Cisco device (or other serial devices) using OSX and a USB to Serial converter (eg ATEN). Googling, everyone says to use zterm, but I couldn’t get it to work (and zterm looks and feels like a toy).

This probably isn’t the “Mac way” of doing things, but </whatever>…

Install PL2303 USB to Serial Driver for Mac OS X, restart

sudo port selfupdate
sudo port upgrade outdated
sudo port install minicom

Work out tty of USB-Serial converter:

ls /dev/tty* > pre

Plug in USB-Serial converter.

ls /dev/tty* > pst
% diff pre pst
3a4
> /dev/tty.PL2303-00002006

Link up to make life easier:

cd /dev
sudo ln -s tty.PL2303-00002006 ttyusb

Setup minicom, and use colour:

sudo minicom -s -c on

Setup defaults to:

/dev/ttyusb (ctrl-A, O, Serial Port Setup)
8N1
9600
no hardware or software flow control
init string empty (so garbage doesn’t appear on screen when you connect) – ctrl-A, O, Init string
save settings as default (ctrl-A, O, Save setup as dfl)

Always use colour:

add export MINICOM=”-c on” to shell rc file (~/.bashrc, .~/zshrc, etc)

Connect (sudo minicom), hit enter a few times, and you should be on the device.

Bruce Badger: For home DNS & DHCP, use dnsmasq

Sun, 28 Feb 2010 13:48:16 +0000

I find that I have many devices in my home requiring IP addresses and names; laptops, netbooks, phones, nas, music player ... the list goes on. My DSL modem/router will run DHCP so devices can get IP addresses. If I then want to refer to a device by name (e.g. 192.168.0.24 aka "nero") I can just add the name and IP to the hosts file on my workstation ... and (yawn) on every other computer I also want to see that device by name.

Unfortunately the DHCP server in the router is a bit free and easy with IP addresses, so if I restart a device it can get a different IP address which means that IP addresses move from device to device over time, which in turn means the names in my hosts files are all messed up. Darn ... and potentially harmful.

Of course all I really want to do it to write down once that *this* box should be given *this* IP address and made available with *this* name

... and the small utility dnsmasq lets me do just that. The following line says that the device with the MAC address 00:1b:62:01:e4:d5 is always to be given the IP address 192.168.0.24 and is always to be known as nero:

dhcp-host=00:1b:62:01:e4:d5,192.168.0.24,nero

This one line configures both a DHCP server and a DNS server (light-weight implementations internal to dnsmasq), so any machine on the LAN can now use the name "nero" in place of the IP address even if the IP address did change, which would only happen if the configuration line above were changed.

For the low-down on dnsmask see the project home page and the articles here and here.

A very brief how-to (assuming a Debian-like server) would go something like ...

1. Identify a machine on your LAN which runs 24/7 (I use a FitPC running Ubuntu) and install dnsmasq on it

2. Decide on a domain for your LAN, e.g. home.localdomain

3. Identify the devices on your network which will have static IP addresses. I have two, the DSL modem/router and the FitPC running dnsmasq. For these devices make entries in the /etc/hosts file of the machine running dnsmasq, e.g.:

192.168.0.1 gateway gateway.home.localdomain
192.168.0.100 server server.home.localdomain

4. Now work on the dnsmasq config file (/etc/dnsmasq.conf). First take a copy of the default file because it has lots of useful information in there. Then edit the file with details of your network and devices. You can see an example based on what I did for my home network below, but do refer to the docs and the original config file to understand the detail.

5. Stop any other DHCP server you may have running (e.g. the one in the DSL modem/router)

6. On the machine running dnsmasq: sudo /etc/init.d/dnsmasq restart

7. Restart the network interfaces on your devices (or just reboot the computer, phone or whatever)

8. Test that everything is as you expect. e.g. does ping nero work? Does ping nero.home.localdomain work?

9. Done

The example file:

domain-needed
bogus-priv
expand-hosts
domain=home.localdomain
dhcp-range=192.168.1.100,192.168.1.150,1h

dhcp-host=00:1b:62:01:e4:d5,192.168.0.24,nero
dhcp-host=00:1c:f0:06:25:ca,192.168.0.25,see
dhcp-host=00:23:76:cd:a7:9a,192.168.0.26,rome
dhcp-host=00:14:a6:2d:a7:9a,192.168.0.27,burn

dhcp-option=option:router,192.168.0.1
dhcp-option=option:domain-name,home.localdomain

Mark Greenaway: Hard hack: Behringer HellBabe wah

Sun, 28 Feb 2010 06:50:32 +0000

A while ago I bought one of these. It has good build quality, and a good sound, but suffers from a fatal design flaw: the rocker pedal doesn't have enough travel in it. So it didn't get much use.

Enterprising people on the Internet weren't going to be deterred by something as trifling as this. If you don't mind voiding the warranty, you can follow these detailed instructions to fix the mechanical problem. I substituted a small paring knife for a scalpel, and dumb luck for precision, but everything worked perfectly and now I have a beautiful sounding wah pedal for much less than you'd pay for a brand name one.

A word of caution though, you may lose an afternoon playing with the result :)

James Purser: Never take a photo of a Yeast Ring

Sat, 27 Feb 2010 08:49:37 +0000

Today while out with the family, we decided to purchase a treat for the kids. So we headed for the nearby donut place to purchase the required sickeningly sweet treat.

Turns out they didn't have enough of the particular sweet my son wanted, so he picked what looked like an iced donut with chocolate flakes on it. Except it wasn't called a donut, rather it was called a Yeast Ring.

This naturally caught my eye as being a somewhat risky name to give a ring of dough, so after the kids had consumed their various sweets, I thought I would get a photo of the Yeasty Rings to follow up the tweet I had already made.

Not thinking much of it, I went up to the counter and used my camera to take a couple of what turned out to be uselessly blurred photos. Gave it up as a lost effort and then went off to complete the rest of the tasks we had at the time.

45 minutes later as we exited the shopping centre a security guard approached me and asked me if I had taken any photos around the donut place earlier on. Specifically he asked if I had taken photos of a young girl who had apparently been standing close to the Yeast Rings display. Apparently the girls mother had registered a complaint, accusing me of taking photos of her girl. The guard didn't mention why this would be a problem, but the obvious implication is that I was taking photos for nefarious purposes.

It took me but a minute to explain to the guard that no, I hadn't taken photos of the girl, I hadn't even seen the girl standing there, focused as I was, on the Yeast Rings. The guard explained that they had to follow up the complaint both to calm the apparently distressed mother and for reasons of terrorism (my jaw dropped I'm sure at this one). However while I was finishing up with the guard, a police car pulled up in a nearby spot, and I was informed by the guard that I would need to talk to the officer.

Right.

Remaining calm at all times I walked over to the officer and once again went through the process of showing him that the photos I had taken were of the donut silmacra rather than the girl. The officer was pretty good about it, requesting to see through the rest of the photos, which I allowed, and then we parted ways without a request for details or anything else.

All in all the security staff and police were polite and cordial in the way that they treated me. I have since discovered however that they were actually trailing myself and my family for at least half an hour before we were approached.

Sigh.

Below are the two photos that I took:

and

Update (01/03/2010):

I have just received confirmation that the security personal have managed to get in contact with the woman who made the original complaint and explain that there was no substance to her complaint - something that had been playing on mind since Saturday afternoon. While this wraps up the event (and believe me I feel quite relieved), it doesn't excuse the manner in which it was dealt with. I will be writing a further letter to the centre regarding this sort of situation and ways it could have been handled better.

Ian Wienand: DIG Jazz Applet, V3

Fri, 26 Feb 2010 13:44:00 +0000

The ABC overhauled DIG Jazz (now I think it's just called "ABC Jazz") and upgraded from the oh-so 2008 XML playlist to a much more web-cool JSON one.

Hence Version 3 (source) of the applet. Now with improved HTML escaping and different colors.

Check out The Dilworths while you're there!

Dave Airlie: GPU switching update

Fri, 26 Feb 2010 05:04:56 +0000

Okay I've been busy elsewhere but dragged myself back to try and finish this for upstream

v10 of the patch is up
http://people.freedesktop.org/~airlied/vgaswitcheroo/0001-vga_switcheroo-initial-implementation-v10.patch

changes are mainly that mjg59 was right about keeping ugly things in the drivers.

adding ATRM support to get the ROMs on ATI hybrid for the discrete card was actually a pain with the previous code design,
so I moved lots of it around again, and now the discrete ROM can be retrieved via the ATRM method.

I've tested it on the W500 and it works as well as before, which means still the 3rd or 4th switch fails and locks the machine up,
I need to debug this further.

The refactored code should hopefully make it easier to fill in the nvidia/nvidia and intel/nvidia blanks for mjg59.

Update 1: v11 is now up
http://people.freedesktop.org/~airlied/vgaswitcheroo/0001-vga_switcheroo-initial-implementation-v11.patch
It should fix the failure to switch to IGD the 2nd time hopefully.

Update 2: v13 is now up, it blindly implements nvidia DSM changing, but I've no idea if it works. Hopefully someone can test it and give me some feedback. Its nearly all guesswork from work mjg59 did.

James Purser: Online Content Ombudsman

Fri, 26 Feb 2010 01:58:29 +0000

Hey here's an idea to combat the rash of idiotic facebook page "hacks" (leaving your facebook page open to all is not a hack by the way). Let's set up an Online Content Ombudsman, who shall have the power to... umm... complain to overseas companies.

Um.

Originally suggested by South Australian independent Senator Nick Xenaphon, it's been picked up by the Government to show how "pro-active" they are when it comes to dealing "Teh Evil Interwebs".

Considering that companies like Facebook and MySpace have absolutely zero Australian presence, I don't see how any Ombudsman can have any affect on their operations, unless, and this is a very scary, scary thought. The Ombudsman has the power to attach web sites to the Filter. Anything less than threatening to ban the site within the country is going to have all the persuasive power of a damp tissue.

I would love to be able to sit down with Kevin Rudd and Senator Nick, away from the cameras and explain to them how the internet works. If not me, then please can someone else do it?

Sonia Hamilton: Problem: braindead screensaver timeouts

Fri, 26 Feb 2010 00:06:39 +0000

Problem: You work in an organisation with locked down desktops that have stupidly short screensaver timeouts. And you’re connected via multiple rdp’s to a process that you want to keep running all day, but the timeouts keep halting the process.

Solution: an optical mouse on top of an analog watch (that has a second hand):

Take that Group Policy!

(Tip of the Hat to Stu).

Ian Wienand: Building a quiet, cool media/house server

Thu, 25 Feb 2010 02:56:00 +0000

After getting sick of having to underclock my existing home server to get it to remain up for any period of time, along with the horrendous noise, I finally found the time and budget to rebuild.

My goals were:

Quiet enough to have in the living room
Cool enough to live inside the TV cabinet with little ventilation
Large, redundant storage
Powerful enough to re-encode video
As power efficient as possible

In the end I went with

Antec NSK 1380
Asus AT3N7A-I motherboard with a dual-core Atom 330 CPU and Nvidia chipset
4GB of RAM (for which I am still waiting for the mail-in rebate, as usual!)
1 Western Digital Green 1TB SATA disk
1 Samsung Ecogreen 1TB SATA disk
30GB Patriot SATA SSD
Recycled DVD writer + PCI IDE card

The case is really awesome. Very easy to access, and the fan is extremley quiet. It very cleverly holds 3 full-size hard-disks; two mounted vertically on either side, and one horizontally in the middle. The final space is for a DVD -- after that it's pretty cramped inside! It makes claims it is a very efficient power supply.

It has a very bright blue LED on the front, and the face-plate over the DVD looks nice but the button doesn't quite reach the eject button on my drive, so it's software eject only. All over, definitely recommend.

The motherboard is fairly good. One annoying thing is that it only has 3 SATA ports -- I think it's reasonable to want to have a primary drive, two mirrored large disks plus a DVD for a nice little media server. It also has no parallel-IDE, which is reasonable these days. However, if you wanted to install a wireless card you'd be out of luck if you also wanted to put in another SATA card, as it only has one PCI slot. There are plenty of USB ports for a USB wireless card, however. It also comes with two SATA cables, which isn't mentioned anywhere I could see (hopefully this saves you a trip back to Fry's to return your extra cables :).

The CPU fan is a little on the loud side, as mentioned in some other forums. It is also located right where the power supply cables come down in this case, making for a fairly tight fit.

Here's a lspci for those interested in such things

00:00.0 Host bridge: nVidia Corporation MCP79 Host Bridge (rev b1)
00:00.1 RAM memory: nVidia Corporation MCP79 Memory Controller (rev b1)
00:03.0 ISA bridge: nVidia Corporation MCP79 LPC Bridge (rev b2)
00:03.1 RAM memory: nVidia Corporation MCP79 Memory Controller (rev b1)
00:03.2 SMBus: nVidia Corporation MCP79 SMBus (rev b1)
00:03.3 RAM memory: nVidia Corporation MCP79 Memory Controller (rev b1)
00:03.5 Co-processor: nVidia Corporation MCP79 Co-processor (rev b1)
00:04.0 USB Controller: nVidia Corporation MCP79 OHCI USB 1.1 Controller (rev b1)
00:04.1 USB Controller: nVidia Corporation MCP79 EHCI USB 2.0 Controller (rev b1)
00:06.0 USB Controller: nVidia Corporation MCP79 OHCI USB 1.1 Controller (rev b1)
00:06.1 USB Controller: nVidia Corporation MCP79 EHCI USB 2.0 Controller (rev b1)
00:08.0 Audio device: nVidia Corporation MCP79 High Definition Audio (rev b1)
00:09.0 PCI bridge: nVidia Corporation MCP79 PCI Bridge (rev b1)
00:0b.0 IDE interface: nVidia Corporation MCP79 SATA Controller (rev b1)
00:10.0 PCI bridge: nVidia Corporation MCP79 PCI Express Bridge (rev b1)
00:15.0 PCI bridge: nVidia Corporation MCP79 PCI Express Bridge (rev b1)
01:05.0 RAID bus controller: Silicon Image, Inc. PCI0680 Ultra ATA-133 Host Controller (rev 02)
02:00.0 VGA compatible controller: nVidia Corporation ION VGA (rev b1)
03:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL8111/8168B PCI Express Gigabit Ethernet controller (rev 03)

I know there are issues with the controller of the Patriot SSD, which would probably worry me for a general purpose machine. However for the primary disk of a server machine I'm not too fussed. The silence is golden and cool-running and low power consumption really helps too. I wouldn't really say it seems that blazing fast. It comes with a handy mount so it fits in a full-size hard-disk slot. Hard to beat for the price.

One concession I made to avoid excessive writes was mounting /tmp on tmpfs; this is where it's nice to have 4GB of RAM. Handbrake seems to do a lot of work in /tmp for example.

The two green hard-disks (software mirrored) are also inaudible. I'm hoping different brands should at least not fail at the same time. I'm not sure if they're auto-spinning down, but you can't really tell if they're on or not, even under load.

Even when running flat-out re-encoding a DVD in the enclosed cabinet with minimal airflow the CPU temperature hasn't gone above 60C; normal CPU temp is around 40C. Performance is adequate -- running PlayOn in a VMware workstation XP VM almost works.

It's kind of hard to take photos inside a case, but here's an attempt:

Debian unstable installed without issues (except for some weird bug with the boot hanging for 60 seconds due to the RTL driver). It is also very strange installing from USB to an SSD — no noise!

James Purser: What do I and Will Smith have in common?

Wed, 24 Feb 2010 23:26:46 +0000

We've both played Enemies of the State.

Seriously, according to this article the IIPA (International Intellectual Property Alliance) has requested that the US Trade Representative place countries that mandate, or (and here's the kicker) even suggest the use of Open Source software be placed on a special list of countries that have lax intellectual property regimes (read Pirate Havens).

Note that this is not just a case of sneaking in the Open Source thing amongst a number of other reasons, the IIPA is explicitly claiming that even thinking about using Open Source software in Government is tantamount to destroying the global software industry, letting the Goths into Rome and scheduling a 7 day, 24 hour run of Home and Away on every single channel.

So according to the IIPA, I as a purveyor of Open Source solutions am leading to the delinquency of governments world wide, using my awesome "Powahs! (tm)" to drive poor innocent multi-nationals to the wall.

Sigh.

Jamie Wilkinson: Monitoring Cron Jobs

Wed, 24 Feb 2010 12:27:00 +0000

(This article comes from one I helped edit and publish inside work, so I can't take any credit for the ideas expressed within, though I do vehemently and violently subscribe to the sentiment! Thanks to Alan Sundell for originally educating me.)

When you set (or don't set at all) MAILTO in a crontab fragment, typically it's because you want to be notified if your job fails -- failure in this case if and only if the job only prints to stdout/stderr if there is an exceptional condition... However not all jobs print only on exceptional conditions, many use stderr for logging, and email is just not a great solution to this problem, especially at scale.

Problems with alerting by email from `cron`.

Why is it a bad idea to rely on cron mail?

We all get so much mail from cron that few are in the habit of reading it anymore.
If your server is broken, the mail submission agent may be broken too.
You may handle your cron mail, but you may be on holiday when it arrives.
crond can crash.
You will get mutiple emails for the same failure.
Your cron mail will get delivered to every one of your mailboxes, eating up storage.
You cannot suppress cron mail notifications.
Your cron mail has no concept of dependencies.
You will get notified of temporary failures when you only care about persistent ones.

If a cronjob running successfully is critical to operation, then it seems that what you really need is some kind of monitoring system that addresses all of these things, and can send alerts to some oncall rotation that determines who is responsible for handling alerts.

A potential solution

Here's an idea that might help with that.

Direct the output of your job to some log file for debugging, in the event of persistent failure. Note the truncate:
```
MAILTO=""
*/1 * * * * root cronjob > /var/log/cronjob.log 2>&1
```
(If you decide to appent, not overwrite the log each execution, then make sure you logrotate that file.)
At the end of cronjob, update a status file, like so:
```
scriptname=$(basename $0)
date +%s > /var/tmp/cronjob-last-success-timestamp
```
Ensure that your job exits on error before reaching the last line!
Collect the content of that file regularly with your monitoring system; scrape it with the nagios host agent, pump it into collectd, whatever you hip open source cats are using these days.
Configure your monitoring system to send a notification on the timestamp having not been updated in some time period.
```
if cronjob-last-success-timestamp
  <
(time() - 30m)
  then alert
```
Profit!

Now you only generate an alert if the cron job hasn't succeeded in the last 30 minutes (a threshold you can adjust to match your monitoring scrape intervals and service SLAs), and with a sufficiently mature monitoring system you can now express dependencies, suppress the notification, and send it to an oncall rotation, and so on!

Most significantly, we have converted a system that always reported failure, into a system that is based around checking for success -- a failsafe.

Jeremy Visser: Multicast versus IPTV

Wed, 24 Feb 2010 07:56:04 +0000

I’ve been hearing a lot of buzz about IPTV lately. It’s something that is exciting, useful, and inevitable.

As I understand it, IPTV is defined to be a live television service delivered over multicast (usually RTP) over a broadband Internet connection. Multicast has an advantage over normal traffic (“unicast”) because it avoids the duplication of traffic. Only one copy of the data is ever sent.

TPG currently offer 19 channels over an IPTV service, and TransACT allegedly offer over 50. iiNet is “on the brink of IPTV launch”. There are rumours that Internode are investigating something similar too.

That last paragraph should have triggered alarm bells in your head. It did for me, but maybe I’m just a cynic. Let me explain.

I asked myself: why are the ISPs providing the IPTV service? It sounds wrong to me. I’ve tried to come up with an answer, and my conclusion is that it is because of two reasons: (1) infrastructure (multicast is virtually non-existant in most production networks) and (2) content lock-in.

Having the ISPs provide the infrastructure for the IPTV service (marketing, encoding, distribution, etc.) means the ISP will likely sign a contract to exclusive usage rights with the TV partner for each channel they will provide. If iiNet sign an exclusive deal with Sky News, they sure as heck ain’t going to let Internode or TPG also put the content on their networks.

The infrastructure would also be duplicated. Each ISP needs their own TV receiving equipment (likely a few satellite dishes on top of their data centre), encoding equipment, marketing, private IP multicast network, and whatnot.

With regards to content lock-in, this will also mean that the ISP you choose to use will determine what IPTV channels you are able to watch. Conversely, what IPTV channels you want to watch will determine which ISP you will sign up with.

That sounds all rosy, until you consider that I am not restricted on what websites I can visit based on what ISP I am with. I can watch YouTube or ABC iView from any ISP in Australia. Sure, some ISPs offer better services than others, faster speeds, or more reliable connections, but it’s just the one Internet. This is because the ISP is not in the content provider role. They are merely the gateway to the Internet — the Internet service provider (now where have I heard that term before?).

The way IPTV looks like it is heading is shifting the ISP onto the content provider role. As I’ve already said, this means the service is delivered over the ISP’s private network, available exclusive to that ISP’s customers, and not over the Internet. That’s a Bad Thing™ in terms of net neutrality.

If an ISP “rebroadcasts” content from an IPTV channel that another ISP has signed an exclusive deal on, that would be a copyright violation.

But that still doesn’t explain the title of my post: “Multicast versus IPTV”. The above isn’t actually the crux of what I wanted to get at (apologies to the reader if you’ve managed to read this far in great detail).

I’m supportive of multicast in general. Combined with IPv6 (each /64 subnet has a corresponding multicast subnet), multicast opens up the doorway for anybody to serve high definition audio and video to the Internet at large without the need for exorbitant amounts of bandwidth up their sleeve.

But that’s public multicast. These “IPTV” solutions will be, and currently are, deployed on private multicast networks. On the plus site, private multicast networks mean it is easier for the ISP to deploy the infrastructure — it’s much easier to only have to worry about your own network, than also have to worry about interfacing with others. But it also means less incentive to deploy a public multicast infrastructure, and less incentive for content providers (such as TV networks) to provide their services over multicast themselves. It’s a chicken-and-egg problem.

We don’t need ISPs to “provide IPTV services”. We need a decent multicast network (preferably IPv6) that all Australians can access, so that TV stations themselves can provide the multicast streams with the knowledge that a large percentage of Internet subscribers are able to access their content, and that channel availability discrimination does not occur based on what ISP you choose to sign up with.

That’s the crux of what I’m getting at. I’m sure someone will come along and say “but but but…they have a right to sign exclusive contracts and deliver only over private networks”. My response to that would be that just because you have the right, doesn’t make it right. Nor left.

Thoughts?

Michael Fox: Amazon certainly is cheaper

Tue, 23 Feb 2010 08:43:51 +0000

I am about to embark on the reading of 2 x Microsoft Press books to attempt two Microsoft certification exams. More details on these as time goes on, but for now the story is about the book prices.

Locally the books cost about over $100 for one. Looking on Amazon the books I wanted had list prices of $44 USD and $37 USD. Oddly enough I ended up getting both with postage for about $127 AUD.

It’s an easy decision, buy from Amazon, get both books and have to wait up to a month.

Will post back my experience, I’ve heard others say Amazon has been a pleasure experience to purchase from.

Silvia Pfeiffer: HTML5 Media and Accessibility presentation

Tue, 23 Feb 2010 03:05:51 +0000

Today, I was invited to give a talk at my old workplace CSIRO about the HTML5 media elements and accessibility.

A lot of the things that have gone into Ogg and that are now being worked on in the W3C in different working groups – including the Media Fragments and HTML5 WGs – were also of concern in the Annodex project that I worked on while at CSIRO. So I was rather excited to be able to report back about the current status in HTML5 and where we’re at with accessibility features.

Check out the presentation here. It contains a good collection of links to exciting demos of what is possible with the new HTML5 media elements when combined with other HTML features.

I tried something now with this presentation: I wrote it in a tool called S5, which makes use only of HTML features for the presentation. It was quite a bit slower than I expected, e.g. reloading a page always included having to navigate to that page. Also, it’s not easily possible to do drawings, unless you are willing to code them all up in HTML. But otherwise I have found it very useful for, in particular, including all the used URLs and video element demos directly in the slides. I was inspired with using this tool by Chris Double’s slides from LCA about implementing HTML 5 video in Firefox.

Jeremy Visser: Enable caching in collectd!

Mon, 22 Feb 2010 21:32:33 +0000

If you decide to run collectd on your system, don’t do what I did: I ran it for 8 months without caching enabled. Basically, that means that not only was collectd monitoring load, it was generating a huge amount of load itself — way more than anything else running on the system.

Case in point:

The “no caching” bit is what it’s been running like for 8 months, using this config in /etc/collectd/collectd.conf:

<Plugin rrdtool>
        DataDir "/var/lib/collectd/rrd"
</Plugin>

To more than half my idle CPU usage, and get my load averages back down to near 0.00, all I had to do was add:

<Plugin rrdtool>
        DataDir "/var/lib/collectd/rrd"
       CacheTimeout 120
       CacheFlush 900
</Plugin>

I don’t get why they don’t ship it like that by default. Enable caching. Your server will thank you. Thanks to the folks on #slug who held my hand while fixing this.

Simon Rumble: Josephine's Cheese: Terrible packaging

Sun, 21 Feb 2010 23:17:17 +0000

Josephine's Cheese, given your site has no contact information, and doesn't even display the product I bought, I'll have to howl into the aether and assume you'll find it when I SEO your arse (given Google doesn't even see your image-only site, that won't be hard).

So I bought your "Traditional Goat's Cheese in Ash" last week. It was delicious. Unfortunately your packaging is abysmal. After very carefully peeling off the label, you're presented with this tube of squished cheese. If you take care from this point, and happen to have scissors or a sharp knife handy, you can just about get a single blob of cheese out ready to serve. More likely you'll end up with more like a tube of cheese toothpaste squirted out onto your serving location.

Great cheese. I won't buy it again in this packaging. Buy your competitors' products and learn.

Contact me

Jeremy Visser: SCLUG site redesigned

Sun, 21 Feb 2010 00:36:37 +0000

The South Coast Linux Users Group (SCLUG) website has had a lick of paint by yours truly.

The old site didn’t look bad (in particular, I liked the penguin header image), but was getting a little long in the tooth, had a few issues (such as broken category styles, and images being linked to the wrong domain), and didn’t really reflect much about Wollongong, which is where we are based.

So a little Inkscaping later, I came up with this:

Check it out. No, like, seriously. Check it out. A bit more polish to come, but the idea is there.

Mark Greenaway: Food security

Sat, 20 Feb 2010 23:29:31 +0000

Science magazine has an issue on feeding the future. I've linked it because I just love some of the article names. I can't decide whether I prefer the article about holding back the plague of rats, or the one that suggests we eat less meat and more insects. Mmmm, crickets!

Silvia Pfeiffer: Google’s challenges of freeing VP8

Sat, 20 Feb 2010 07:57:39 +0000

Since On2 Technology’s stockholders have approved the merger with Google, there are now first requests to Google to open up VP8.

I am sure Google is thinking about it. But … what does “it” mean?

Freeing VP8
Simply open sourcing it and making it available under a free license doesn’t help. That just provides open source code for a codec where relevant patents are held by a commercial entity and any other entity using it would still need to be afraid of using that technology, even if it’s use is free.

So, Google has to make the patents that relate to VP8 available under an irrevocable, royalty-free license for the VP8 open source base, but also for any independent implementations of VP8. This at least guarantees to any commercial entity that Google will not pursue them over VP8 related patents.

Now, this doesn’t mean that there are no submarine or unknown patents that VP8 infringes on. So, Google needs to also undertake an intensive patent search on VP8 to be able to at least convince themselves that their technology is not infringing on anyone else’s. For others to gain that confidence, Google would then further have to indemnify anyone who is making use of VP8 for any potential patent infringement.

I believe – from what I have seen in the discussions at the W3C – it would only be that last step that will make companies such as Apple have the confidence to adopt a “free” codec.

An alternative to providing indemnification is the standardisation of VP8 through an accepted video standardisation body. That would probably need to be ISO/MPEG or SMPTE, because that’s where other video standards have emerged and there are a sufficient number of video codec patent holders involved that a royalty-free publication of the standard will hold a sufficient number of patent holders “under control”. However, such a standardisation process takes a long time. For HTML5, it may be too late.

Technology Challenges
Also, let’s not forget that VP8 is just a video codec. A video codec alone does not encode a video. There is a need for an audio codec and a encapsulation format. In the interest of staying all open, Google would need to pick Vorbis as the audio codec to go with VP8. Then there would be the need to put Vorbis and VP8 in a container together – this could be Ogg or MPEG or QuickTime’s MOOV. So, apart from all the legal challenges, there are also technology challenges that need to be mastered.

It’s not simple to introduce a “free codec” and it will take time!

Google and Theora
There is actually something that Google should do before they start on the path of making VP8 available “for free”: They should formulate a new license agreement with Xiph (and the world) over VP3 and Theora. Right now, the existing license that was provided by On2 Technologies to Theora (link is to an early version of On2’s open source license of VP3) was only for the codebase of VP3 and any modifications of it, but doesn’t in an obvious way apply to an independent re-implementations of VP3/Theora. The new agreement between Google and Xiph should be about the patents and not about the source code. (UPDATE: The actual agreement with Xiph apparently also covers re-implementations – see comments below.)

That would put Theora in a better position to be universally acceptable as a baseline codec for HTML5. It would allow, e.g. Apple to make their own implementation of Theora – which is probably what they would want for ipods and iphones. Since Firefox, Chrome, and Opera already support Ogg Theora in their browsers using the on2 licensed codebase, they must have decided that the risk of submarine patents is low. So, presumably, Apple can come to the same conclusion.

Free codecs roadmap
I see this as the easiest path towards getting a universally acceptable free codec. Over time then, as VP8 develops into a free codec, it could become the successor of Theora on a path to higher quality video. And later still, when the Internet will handle large resolution video, we can move on to the BBC’s Dirac/VC2 codec. It’s where the future is. The present is more likely here and now in Theora.

ADDITION:
Please note the comments from Monty from Xiph and from Dan, ex-On2, about the intent that VP3 was to be completely put into the hands of the community. Also, Monty notes that in order to implement VP3, you do not actually need any On2 patents. So, there is probably not a need for Google to refresh that commitment. Though it might be good to reconfirm that commitment.

Erik de Castro Lopo: Conroy : Is He a Duck?

Fri, 19 Feb 2010 22:01:12 +0000

The Australian Federal Minister for Communications, Stephen Conroy, may not actually be corrupt; I certainly have no evidence that he is, but a number of recent incidents sure look like corruption to me. For instance:

Conroy's recommendation of a former ALP Staffer for a highly paid senior position (government relations) with the National Broadband Network an organisation operating in the area of Conroy's ministry.
Conroy playing golf with billionaire media mogul James Packer on the same day the government announced a $250 million licence fee rebate for free-to-air television stations (one of which is owned by Packer).
Conroy skiing with media mogul Kerry Stokes in Colorado a month before the $250 million licence fee rebate was announced. James Packer and Kerry Stokes are the owners of 2 of the 3 main non-government free-to-air television stations.

The real irony is that under Conroy's proposed scheme to filter the internet in Australia comments like this blog entry may end up being censored. The problem with Conroy's filter is not that it filters porn, but rather that the list of what is being filtered is secret and hence could easily include web sites which contain comments which the government or the Minister for Communications want silenced.

Jeremy Visser: IPv6–enabled

Fri, 19 Feb 2010 04:36:08 +0000

This blog is officially IPv6–enabled, as of a few days ago.

The server-side IPv6 connectivity is admittedly powered by 6to4, which is not quite the real deal, but given that the nearest 6to4 gateway is 0.8msec away, I couldn’t very well pass the opportunity to stress test my new Linode.

I’m a pretty firm believer in the fact that IPv6 adoption is absolutely essential to the continued health and function of the Internet. I’ve been IPv6 tunneling from my house for years, had native IPv6 at my house since November, and though I’m certainly not the first to do so, it’s time to IPv6–enable my blog.

Silvia Pfeiffer: Accessibility support in Ogg and liboggplay

Fri, 19 Feb 2010 01:57:34 +0000

At the recent FOMS/LCA in Wellington, New Zealand, we talked a lot about how Ogg could support accessibility. Technically, this means support for multiple text tracks (subtitles/captions), multiple audio tracks (audio descriptions parallel to main audio track), and multiple video tracks (sign language video parallel to main video track).

Creating multitrack Ogg files
The creation of multitrack Ogg files is already possible using one of the muxing applications, e.g. oggz-merge. For example, I have my own little collection of multitrack Ogg files at http://annodex.net/~silvia/itext/elephants_dream/multitrack/. But then you are stranded with files that no player will play back.

Multitrack Ogg in Players
As Ogg is now being used in multiple Web browsers in the new HTML5 media formats, there are in particular requirements for accessibility support for the hard-of-hearing and vision-impaired. Either multitrack Ogg needs to become more of a common case, or the association of external media files that provide synchronised accessibility data (captions, audio descriptions, sign language) to the main media file needs to become a standard in HTML5.

As it turn out, both these approaches are being considered and worked on in the W3C. Accessibility data that are audio or video tracks will in the near future have to come out of the media resource itself, but captions and other text tracks will also be available from external associated elements.

The availability of internal accessibility tracks in Ogg is a new use case – something Ogg has been ready to do, but has not gone into common usage. MPEG files on the other hand have for a long time been used with internal accessibility tracks and thus frameworks and players are in place to decode such tracks and do something sensible with them. This is not so much the case for Ogg.

For example, a current VLC build installed on Windows will display captions, because Ogg Kate support is activated. A current VLC build on any other platform, however, has Ogg Kate support deactivated in the build, so captions won’t display. This will hopefully change soon, but we have to look also beyond players and into media frameworks – in particular those that are being used by the browser vendors to provide Ogg support.

Multitrack Ogg in Browsers
Hopefully gstreamer (which is what Opera uses for Ogg support) and ffmpeg (which is what Chrome uses for Ogg support) will expose all available tracks to the browser so they can expose them to the user for turning on and off. Incidentally, a multitrack media JavaScript API is in development in the W3C HTML5 Accessibility Task Force for allowing such control.

The current version of Firefox uses liboggplay for Ogg support, but liboggplay’s multitrack support has been sketchy this far. So, Viktor Gal – the liboggplay maintainer – and I sat down at FOMS/LCA to discuss this and Viktor developed some patches to make the demo player in the liboggplay package, the glut-player, support the accessibility use cases.

I applied Viktor’s patch to my local copy of liboggplay and I am very excited to show you the screencast of glut-player playing back a video file with an audio description track and an English caption track all in sync:

elephants_dream_with_audiodescriptions_and_captions

Further developments
There are still important questions open: for example, how will a player know that an audio description track is to be played together with the main audio track, but a dub track (e.g. a German dub for an English video) is to be played as an alternative. Such metadata for the tracks is something that Ogg is still missing, but that Ogg can be extended with fairly easily through the use of the Skeleton track. It is something the Xiph community is now working on.

Summary
This is great progress towards accessibility support in Ogg and therefore in Web browsers. And there is more to come soon.

James Purser: How much Australian Content is generated online

Thu, 18 Feb 2010 12:22:25 +0000

I was pondering the supposed reason behind the recent fee rebate the Government is giving to the TV networks, that Australian Content needs to be developed and played on our air waves, when a question arose.

How much "Australian Content" is generated online? If you took the podcasts, blog posts, music and videos generated by Australians for online consumption, how would they stack up against the "Traditional Media"?

Does anyone know if any studies have been done on this?

Oh and I've added the spam module and turned off comment moderation, let's see how this goes.

Silvia Pfeiffer: How to display seeked position for HTML5 video

Thu, 18 Feb 2010 04:29:25 +0000

Recently, I was asked for some help on coding with an HTML5 video element and its events. In particular the question was: how do I display the time position that somebody seeked to in a video?

Here is a code snipped that shows how to use the seeked event:

<video onseeked="writeVideoTime(this.currentTime);" src="video.ogv" controls></video> <p>position:</p><div id="videotime"></div> <script type="text/javascript"> // get video element var video = document.getElementsByTagName("video")[0]; function writeVideoTime(t) { document.getElementById("videotime").innerHTML=t; } </script>

Other events that can be used in a similar way are:

loadstart: UA requests the media data from the server
progress: UA is fetching media data from the server
suspend: UA is on purpose idling on the server connection mid-fetching
abort: UA aborts fetching media data from the server
error: UA aborts fetching media because of a network error
emptied: UA runs out of network buffered media data (I think)
stalled: UA is waiting for media data from the server
play: playback has begun after play() method returns
pause: playback has been paused after pause() method returns
loadedmetadata: UA has received all its setup information for the media resource, duration and dimensions and is ready to play
loadeddata: UA can render the media data at the current playback position for the first time
waiting: playback has stopped because the next frame is not available yet
playing: playback has started
canplay: playback can resume, but at risk of buffer underrun
canplaythrough: playback can resume without estimated risk of buffer underrun
seeking: seeking attribute changed to true (may be too short to catch)
seeked: seeking attribute changed to false
timeupdate: current playback position changed enough to report on it
ended: playback stopped at media resource end; ended attribute is true
ratechange: defaultPlaybackRate or playbackRate attribute have just changed
durationchange: duration attribute has changed
volumechange:volume attribute or the muted attribute has changed

Please refer to the actual event list in the specification for more details and more accurate information on the events.

Sonia Hamilton: I love handbrakecli (and dd_rescue)

Wed, 17 Feb 2010 01:38:08 +0000

I watch lots of (martial arts) instructional videos. Something I’ve always wanted to do is “mashup” my own videos ie take a few chapters from one video, a few chapters from another video. Now I know how – handbreakcli:

First, rip the vobs off the dvd. You can work directly off the dvd with handbreak, but working from the hard-disk is often easier. I use dd_rescue for unencrypted dvds. (For scratched dvds, I’ve been using motorbike visor cleaner to remove the scratches, and myrescue).

for i in /media/cdrom/VIDEO_TS/* ; do
  j=`basename $i`
  dd_rescue $i $j
done

Use handbrakecli to scan the disk to find the titles and chapters:

./handbrake -i cd1/VIDEO_TS -t 0

Then convert to mp4’s split on chapters (in this case I’m taking chapters 2-18 from title 2, as individual files):

for i in `seq 2 18` ; do
  ./handbrake -i cd1/VIDEO_TS/ -t 2 -c $i -o cd1.t2.c${i}.mp4
done

Thanks to EngageMedia for their great articles. Another good site is videohelp.com

Robert Collins: Yay Dell-with-Ubuntu down under

Mon, 15 Feb 2010 06:12:02 +0000

Dell has been offering Ubuntu on selected models for a while. I had however nearly given up hope on being able to buy one, because they hadn’t started doing that in Australia. I am very glad to see this has changed though – check out their notebook page. Not all models yet, but a reasonable number have Ubuntu as an option.

Yay!

Mark Greenaway: Problems only nerds have

Mon, 15 Feb 2010 04:03:20 +0000

I'm so tired from the weekend, I read a couple of subscripts as if they were just normal script. So ln(x_k)/n became k*ln(x_k)/n in my summation over k. That didn't work very well.

Michael Fox: Rain, rain and more rain.

Sun, 14 Feb 2010 08:08:08 +0000

Sydney has been getting a fair bit of rain of late. Each weekend we have managed to get a bit of rain too. Which means being unable to do much outside.

On the upside, the grass seems to be growing so fast it’s hard to believe. Problem is, you mow the lawn and then within days of doing so it’s needing another mow. All we need now is for the rain to ease up a bit so that those lawns can get another trip.

I’d love to be able to run the remote control car a little more, haven’t had the chance to do so due to the weather. Managed a short go late today and then we got a serious storm in the late afternoon.

I wish I got a video clip of the water tank overflowing. The rain was very heavy and the tank was overflowing with a very high flow rate of water. Pretty impressive to see, but shame we didn’t have another tank to collect all that water. Oh well.

Managed to get a replacement air compressor today and a friend gave me a tyre pressure attachment, so I can pump up my car tyres when I want. No more need to visit the garage to use the tyre pump. Sweet.

James Purser: Two Ministers and a Shadow

Sun, 14 Feb 2010 00:20:01 +0000

Let me ask you a question.

Which is worse:

A minister who oversees an infrastructure programme that relies upon State standards bodies to police the work (building codes being State responsibilities). Under this programme there have been a number of deaths (3) and faulty installations (houses burning down) due to participants not working to minimum standards.

A minister who meets with the head of one of the very large and wealthy companies directly affected by his ministry (whilst on holidays, so not part of his official duties). Soon after this meeting, said minister announces between $250 and $500 million dollars in rebates for the industry (depending on how well said industry performs gross revenue wise).

To be honest I'm not sure which one is worse.

Peter Garrett has been copping a shelacking this week over the three deaths and numerous house fires that have occured as a result of the shoddy work practices of some of those taking part in the Home Insulation Programme (yes it is HIP to be square). Tony Abbott went so far as to accuse Garrett of "industrial manslaughter", a charge normally brought against company directors who knowingly encourage work practices that lead to death in the workplace.

However what no one seems to have been doing is asking the State bodies responsible for policing building and construction practices. It is their direct responsibility to ensure that their rules and regulations are followed, not the federal governments. So while Garrett can legitimately be hauled over the coals for not including regular and random checks and audits (which should have been considered over and above the State requirements), ultimately the responsibility for these tragic events lie with the businesses and State regulatory bodies.

If you really want to have a go at Garrett then the green loans (with no checks to see if the money is being spent on green anything) would be a much better place to look.

Let's move on to Senator Conroy shall we. Last week he announced that he would be giving the Television networks in Australia a cut in their license fees of 33% for 2010 and 50% in 2011. Given that license fees are calculated at 9% of Gross Revenue, this could mean anything between $250 million and $500 million will be gouged out of the budget for the next two years. The justification given at the time was to help preserve Australian Content, however there isn't any such condition in the rebate and Australian TV Networks are already obliged under law to carry 55% Australian content (which is why we're seeing a huge explosion in "Real TV" type programming).

Now it turns out that Conroy and Channel 7 head Kerry Stokes ran into each other a month before the announcement was made. On a skiing trip in Colorado no less. Both Stokes and Conroy are refusing to say what was discussed at this meeting. For all we know they were discussing the latest in Apre Ski fashions, or possibly their chances in that nights Chalet karaoke night competition. However as both a Minister and a seasoned political operator (I don't like the man but he's managed to stay around this long), Conroy should have known better than to meet with the head of one of the major players affected by his portfolio like that.

Oh and we can't leave this week in politics without a word about everyones favourite National Party member. Barnaby has been in fine form this week, going from confusing his millyuns with his billyuns (and for those who are trying to cover for him, he's CPA, he should know the difference), to trying to spook the market by declaring that Australia is on the brink of defaulting on its foreign debt. Well done Mr Joyce. Oh and if you want to have a look at what a country that is staring default in the face looks like? Have a look at Greece.

Mark Greenaway: Flexibility/weird muscle imbalances

Sat, 13 Feb 2010 09:57:35 +0000

As I've been doing more martial arts/active stuff, my body awareness has been increasing. Recently I've realised that when I try to stretch my left leg, I reflexively tilt my spine to the right to reduce the difficulty of the stretch, which lessens the effectiveness of the stretch (and the discomfort!). I think I got some sort of injury in my childhood and learned to compensate, and I've probably been doing it ever since. So now that I'm more aware of it, I can start to fix it.

Silvia Pfeiffer: Audio Track Accessibility for HTML5

Fri, 12 Feb 2010 10:44:43 +0000

I have talked a lot about synchronising multiple tracks of audio and video content recently. The reason was mainly that I foresee a need for more than two parallel audio and video tracks, such as audio descriptions for the vision-impaired or dub tracks for internationalisation, as well as sign language tracks for the hard-of-hearing.

It is almost impossible to introduce a good scheme to deliver the right video composition to a target audience. Common people will prefer bare a/v, vision-impaired would probably prefer only audio plus audio descriptions (but will probably take the video), and the hard-of-hearing will prefer video plus captions and possibly a sign language track . While it is possible to dynamically create files that contain such tracks on a server and then deliver the right composition, implementation of such a server method has not been very successful in the last years and it would likely take many years to roll out such new infrastructure.

So, the only other option we have is to synchronise completely separate media resource together as they are selected by the audience.

It is this need that this HTML5 accessibility demo is about: Check out the demo of multiple media resource synchronisation.

I created a Ogg video with only a video track (10m53s750). Then I created an audio track that is the original English audio track (10m53s696). Then I used a Spanish dub track that I found through BlenderNation as an alternative audio track (10m58s337). Lastly, I created an audio description track in the original language (10m53s706). This creates a video track with three optional audio tracks.

I took away all native controls from these elements when using the HTML5 audio and video tag and ran my own stop/play and seeking approaches, which handled all media elements in one go.

I was mostly interested in the quality of this experience. Would the different media files stay mostly in sync? They are normally decoded in different threads, so how big would the drift be?

The resulting page is the basis for such experiments with synchronisation.

The page prints the current playback position in all of the media files at a constant interval of 500ms. Note that when you pause and then play again, I am re-synching the audio tracks with the video track, but not when you just let the files play through.

I have let the files play through on my rather busy Macbook and have achieved the following interesting drift over the course of about 9 minutes:

You will see that the video was the slowest, only doing roughly 540s, while the Spanish dub did 560s in the same time.

To fix such drifts, you can always include regular re-synchronisation points into the video playback. For example, you could set a timeout on the playback to re-sync every 500ms. Within such a short time, it is almost impossible to notice a drift. Don’t re-load the video, because it will lead to visual artifacts. But do use the video’s currentTime to re-set the others. (UPDATE: Actually, it depends on your situation, which track is the best choice as the main timeline. See also comments below.)

It is a workable way of associating random numbers of media tracks with videos, in particular in situations where the creation of merged files cannot easily be included in a workflow.

Robert Collins: Using UEC instead of EC2

Thu, 11 Feb 2010 21:14:40 +0000

So, we wanted to move a Hudson CI server at Canonical from using chroots to VM’s (for better isolation and security), and there is this great product Ubuntu Enterprise Cloud (UEC – basically Eucalyptus). To do this I needed to make some changes to the Hudson EC2 plugin – and thats where the fun starts. While I focus on getting Hudson up and running with UEC in this post, folk generally interested in the differences between UEC and EC2, or getting a single-machine UEC instance up for testing should also find this useful.

Firstly, getting a test UEC instance installed was a little tricky – I only had one machine to deploy it on, and this is an unusual configuration. Nicely though, it all worked, once a few initial bugs and misconfiguration items got fixed up. I wrote up the crux of the outcome on the Ubuntu community help wiki. See ‘1 Physical system’. The particular trap to watch out for seems to be that this configuration is not well tested, so the installation scripts have a hard time getting it right. I haven’t tried to make it play nice with Network Manager in the loop, but I’m pretty sure that that can be done via interface aliasing or something similar.

Secondly I needed to find out what was different between EC2 and UEC (Note that I was running on Karmic (Ubuntu 9.10) – so things could be different in Lucid). I couldn’t find a simple description of this, so this list may be incomplete:

UEC runs an old version of the EC2 API. This is because it hasn’t implemented everything in the new API versions yet.
UEC defaults to port 8773, not port 80 (for both the EC2 and S3 API’s)
The EC2 and S3 API’s are rooted differently: at AWS they are at /, for UEC they are at /services/Eucalyptus and /services/Walrus
UEC doesn’t supply a SSL API port as far as I can tell.
DescribeImages has something wonky with it.

So the next step then is to modify the Hudson EC2 plugin to support these differences. Fortunately it is in Java, and the Java community has already updated the various libraries (jets3t and typica) to support UEC – I just needed to write a UI for the differences and pass the info down the various code paths. Kohsuke has let me land this now even though it has an average UI (in rev 27366), and I’m going to make the UI better now by consolidating all the little aspects into a couple of URL’s. Folk comfortable with building their own .hpi can get this now by svn updating and rebuilding the ec2 plugin. We’ve also filed another bug asking for a single API call to establish the endpoints, so that its even easier for users to set this up.

Finally, and this isn’t a UEC difference, I needed to modify the Hudson EC2 plugin to work with the ubuntu user rather than root, as Ubuntu AMI’s ship with root disabled (as all Ubuntu installs do). I chose to have Hudson reenable root, rather than making everything work without root, because the current code paths assume they can scp things as root, so this was less disruptive.

With all that done, its now possible to configure up a Hudson instance testing via UEC nodes. Here’s how:

Install UEC and make sure you can run up instances using euca-run-instances, ssh into them and that networking works for you. Make sure you have installed at least one image (EMI aka AMI) to run tests on. I used the vanilla in-store UEC Karmic images.
Install Hudson and the EC2 plugin (you’ll need to build your own until a new release (1.6) is made).
Go to /configure and near the bottom click on ‘Add a new cloud’ and choose Amazon EC2.
Look in ~/.euca/eucarc, or in the zip file that the UEC admin web page lets you download, to get at your credentials. Fill in the Access Key and Secret Access key fields accordingly. You can put in the private key (UEC holds onto the public half) that you want to use, or (once the connection is fully setup) use the “Generate Key’ button to have a dedicated Hudson key created. I like to use one that I can ssh into to look at a live node – YMMV. (Or you could add a user and as many keys as you want in the init script – more on that in a second).
Click on Advanced, this will give you a bunch of details like ‘EC2 Endpoint hostname’. Fill these out.
Sensible values for a default UEC install are: 8773 for both ports, /services/Eucalyptus and /services/Walrus for the base URLs, and SSL turned off. (Note that the online help tells you this as well).
Set an instance cap, unless you truely have unlimited machines. E.g. 5, to run 5 VMs at a time.
Click on ‘Test Connection’ – it should pretty much instantly say ‘Success’.
Thats the Cloud itself configured, now we configure VM’s that Hudson is willing to start. Click on ‘Add’ right above the ‘List of AMIs to be launched as slaves’ text.
Fill out the AMI with your EMI – e.g. emi-E027107D is the Ubuntu 9.10 image I used.
for remote FS root, just put /hudson or something, unless you have a preseeded area (e.g. with a shared bzr repo or something) inside your image.
For description describe the intent of the image – e.g. ‘DB test environment’
For the labels put one or more tags that you will use to tell test jobs they should run on this instance. They can be the same as labels on physical machines – it will act as an overflow buffer. If no physical machines exist, a VM will be spawned when needed. For testing I put ‘euca’

For the init script, its a little more complex. You need to configure up java so that hudson itself can run:

cat >> /etc/apt/sources.list << EOF
deb http://archive.ubuntu.com/ubuntu/ karmic multiverse
deb http://archive.ubuntu.com/ubuntu/ karmic-updates multiverse
deb http://archive.ubuntu.com/ubuntu/ karmic-security multiverse
EOF
export http_proxy=http://192.168.1.1:8080/
export DEBIAN_FRONTEND=noninteractive
apt-get update
echo "buildd shared/accepted-sun-dlj-v1-1 boolean true" | debconf-set-selections
apt-get install -y -f sun-java6-jre

Note that I have included my local HTTP proxy there – just remove that line if you don’t have one.

Click on Advanced, to get at the less-common options.
For remote user, put ‘ubuntu’, and for root command prefix put ’sudo’.
For number of executors, you are essentially choosing the number of CPU’s that the instance will request. E.g. putting 20 will ask for an extra-large high-cpu model machine when it deploys. This will then show up as 20 workers on the same machine.
Click save
Now, when you add a job a new option in the job configuration will appear – ‘tie this job to a node’. Select one of the label(s) you put in for the AMI, and running the job will cause that instance to start up if its not already available.

Note that Hudson will try to use java from s3 if you don’t install it, but that won’t work right for a few reasons – I’ll be filing an issue in the Hudson tracker about it, as thats a bit of unusual structure in the existing code that I’m happier leaving well enough alone .

Matthew Palmer: Using Virgin Mobile Broadband with wvdial

Wed, 10 Feb 2010 04:05:00 +0000

File Under: Stuff I'm guaranteed to forget if I don't write it down

I just went a bit wild and decided to get myself some mobile broadband love -- I'm finding that I crave a bit of light 'net when I'm out and about, and an impending change in circumstances will mean that I'm out and about a lot more than I am currently.

I have:

Hardware: Huawei E160e USB dongle, USB ID 12d1:1003
Provider: Virgin Mobile Australia Prepaid
OS: Debian GNU/Linux 5.0 with an upgraded 2.6.32 kernel

My wvdial.conf stanza looks like:

[Dialer virgin]
Init1 = ATZ
Init2 = ATQ0 V1 E1 S0=0 &C1 &D2 +FCLASS=0
Init3 = AT+CGDCONT=1,"IP" "VirginBroadband"
Stupid Mode = 1
ISDN = 0
Phone = *99#
Modem = /dev/ttyUSB0
Modem Type = USB Modem
Username = irrelevant
Dial Command = ATDT
Password = irrelevant
Baud = 460800

I also had to disable CHAP (by putting -chap in /etc/ppp/options, although I suspect the same would have worked in /etc/ppp/peers/wvdial).

After that, a simple sudo wvdial virgin and I was away.

Scarily enough, there was actually some useful information on the Virgin support site regarding the settings, which helped me confirm that I was on the right track. I have no doubt that the universe will find some alternate means of punishing me down the line.

James Purser: Friendly Fire on the Open Internet Front

Wed, 10 Feb 2010 02:57:16 +0000

You know something, there are a lot of us working to try and educate and inform our family, friends and communities about the dangers and inadequacies of the proposed mandatory ISP filtering scheme. We're actually talking to people, explaining the issues as they stand and answering questions.

We're also trying to get the media to move away from the over-simplistic "It's all about the Child Porn" angle to look at the real problems with the scheme.

All that work however is being hampered by the kind of activities conducted today by a group of people calling themselves Anonymous. In their wisdom, they decided that they needed to take more direct action. So, in the spirit of attacking censorship on the internet they declared they would be attempting a Denial of Service attack on a number of Government web sites, and at the same time conducting a "blackfax" campaign and other activities of a similar type against various Government offices.

In essence they were going to declare to the Australian government that any attempt to bring regulation to the internet would be met with attacks on government infrastructure.

This sort of short sighted ill thought out protest annoys me on a couple of levels. Firstly it cuts the ground out from under those of us who have been working for a long time to try and change this policy. By acting in such an irresponsible manner, Anonymous has given the media a sensational angle which has and will be used to counter the more cogent and reasoned arguments against the filter. Now, along side the infamous "If you're against the filter you must be for Child Porn" we'll have "Opponents against the filter are evil hackers only concerned with getting access to porn".

Secondly, speaking as a system-administrator and IT worker, these sort of attacks annoy the living hell out of me. The create work, not for the people responsible for the policy you are protesting against, but for the IT staff instead. Not only do they have to deal with the immediate effects of the protest (infrastructure issues and so on), but they also have to spend many hours after any such attack ensuring that nothing more sinister than a DoS was conducted against their infrastructure.

So in the end, Anonymous will have brought attention to themselves, created a link between the Open Internet movement and illegal protests and generally caused problems for everyone concerned.

If only they could get involved in the discussion rather than standing outside yelling obscenities.

Mark Greenaway

Wed, 10 Feb 2010 01:16:51 +0000

Dear Judoka of the world,

If you're going to be doing ground fighting/ne waza, please brush your teeth.

Your pal,
Mark

Robert Collins: Is a code of silence evil?

Tue, 09 Feb 2010 09:07:27 +0000

Looking at using google apps for my home email, as I want to be able to have my home machines totally turned off from time to time.

Found this interesting gem in the sign up agreement (which I have not yet agreed to ):

11. PR. Customer agrees not to issue any public announcement regarding the existence or content of this Agreement without Google’s prior written approval. Google may (i) include Customer’s Brand Features in presentations, marketing materials, and customer lists (which includes, without limitation, customer lists posted on Google’s web sites and screen shots of Customer’s implementation of the Service) and (ii) issue a public announcement regarding the existence or content of this Agreement. Upon Customer’s request, Google will furnish Customer with a sample of such usage or announcement.

This is rather asymmetrical: If I agree to the sign up page, I cannot say ‘I am using google apps’, but google can say ‘Robert is using google apps’. While I can appreciate not wanting to be dissed on if something goes wrong, this is very much not open! A couple of implications: Everyone seeking support for google apps in the apps forums is probably in violation of the sign up agreement; we can assume that anyone having a terrible experience has been squelched under this agreement.

Le sigh.

Planet SLUG

Dave Airlie: two X servers one graphics card.

Sonia Hamilton: Australian BJJ Competitions

Michael Fox: VMware Workstation 7.1 Beta

Robert Collins: LCA 2010 videos are showing up

Michael Fox: Fusion 3.1 Beta now available

Mark Greenaway

James Purser: Inspiring and Depressing at the same time

Nick Jenkins: Recovering from a corrupted MySQL install due to a dying hard disk

Ian Wienand: Handling hostnames, UDP and IPv6 in Python

Erik de Castro Lopo: GHC 6.12.1 in Debian Testing.

Michael Fox: iPhone 3GS

Erik de Castro Lopo: FP-Syd #22.

Dave Airlie: GPU offloading - PRIME - proof of concept

Mark Greenaway: This is just amazingly silly

Erik de Castro Lopo: Intel Embedded Graphics Driver Fail.

Mark Greenaway

Simon Rumble: Sneaky trick to de-obfuscate Omniture JavaScript plugins

Simon Rumble: UI fail from Exetel

Michael Fox: Traxxas Rustler VXL damage

Michael Fox: Amazon books arrive

Michael Fox: Transition to Wordpress MU

Ian Wienand: RFC3164 smells

James Purser: Dear TV Networks - Sigh

Sonia Hamilton: BJJ for women

Sonia Hamilton: Serial Port Access on OSX to Cisco devices

Bruce Badger: For home DNS & DHCP, use dnsmasq

Mark Greenaway: Hard hack: Behringer HellBabe wah

James Purser: Never take a photo of a Yeast Ring

Ian Wienand: DIG Jazz Applet, V3

Dave Airlie: GPU switching update

James Purser: Online Content Ombudsman

Sonia Hamilton: Problem: braindead screensaver timeouts

Ian Wienand: Building a quiet, cool media/house server

James Purser: What do I and Will Smith have in common?

Jamie Wilkinson: Monitoring Cron Jobs

Problems with alerting by email from cron.

A potential solution

Jeremy Visser: Multicast versus IPTV

Michael Fox: Amazon certainly is cheaper

Silvia Pfeiffer: HTML5 Media and Accessibility presentation

Jeremy Visser: Enable caching in collectd!

Simon Rumble: Josephine's Cheese: Terrible packaging

Jeremy Visser: SCLUG site redesigned

Mark Greenaway: Food security

Silvia Pfeiffer: Google’s challenges of freeing VP8

Erik de Castro Lopo: Conroy : Is He a Duck?

Jeremy Visser: IPv6–enabled

Silvia Pfeiffer: Accessibility support in Ogg and liboggplay

James Purser: How much Australian Content is generated online

Silvia Pfeiffer: How to display seeked position for HTML5 video

Sonia Hamilton: I love handbrakecli (and dd_rescue)

Robert Collins: Yay Dell-with-Ubuntu down under

Mark Greenaway: Problems only nerds have

Michael Fox: Rain, rain and more rain.

James Purser: Two Ministers and a Shadow

Mark Greenaway: Flexibility/weird muscle imbalances

Silvia Pfeiffer: Audio Track Accessibility for HTML5

Robert Collins: Using UEC instead of EC2

Matthew Palmer: Using Virgin Mobile Broadband with wvdial

James Purser: Friendly Fire on the Open Internet Front

Mark Greenaway

Robert Collins: Is a code of silence evil?

Problems with alerting by email from `cron`.