Airline staff

Daddy, why is that gate so narrow?

So people don’t take their big bags onto the aeroplane themselves, and they check them in to go in the cargo hold.

So the pilot can put them in the aeroplane?

Yes, the pilot needs to put them in the place underneath. Well the pilot has just one special job of flying the plane. Other people do other special jobs like that one. It actually takes a lot of people to help get us and our things to fly from one place to another.


I don’t love flying – and I reckon few people do – but I am still grateful for the people who make it possible. I spend some hours packing, getting the timing and the papers all right, lining up, waiting and then waiting some more. For that, I get to be somewhere else much faster than by other ways. They get a day’s pay, which I guess they consider a fair trade. I don’t know how many flights per week a cargo loader, air steward or pilot might do, or how often they meet that tiny number of passengers who make things difficult, or what they have to deal with behind the scenes. So, thank you. I have time to sit and feel grateful, because the kids are settled.

(This is actually from November 2015, but the words haven’t dated. I’m still grateful to sit still when the kids are settled, and to anyone who helps that happen!)

Thank you, honest strangers

Life has been heavy going for the past year, more so than ever before. It’s had ups and downs, and yesterday was largely a down – I forgot too many things and failed to communicate with people properly so that only made things worse. Anyway, they were all very good about it and I’m lucky to have great caring people around me helping make it all work.

Today was better. I had time to get things done which will mercifully remain done. Washing, cleaning the chook cage, getting the kids to brush their teeth… have to be done again and again, but it’s good to make some progress on the backlog.

Anyway, to the title of the post: I dealt with two unfamiliar situations today, and both of them turned out more smoothly than I dared hope: Selling a low-value car, and shipping a kitchen appliance across the country to a Gumtree buyer with just-in-time (significant) payment of postage. Both of these have scam potential, certainly evidenced by a couple of the messages I’d received from potential buyers (or scammers). I reckon the best response is to treat everyone as if they’re honest, but take protective steps along the way, like taking photos of goods, explicitly discussing how things are happening, and recording relevant information. There’s no sense in being paranoid and putting people off side.

To cut a long story short, one person handed me cash, let me photograph their drivers licence and took away the car; another sent me the full agreed payment plus enough to cover relevant costs, and I no longer have these things cluttering up my life. To both of you, thank you. We drank champagne tonight for a few reasons, and you helped.

Enable Java WS on Centos 7

Sometimes when you’re making changes to systems it feels wrong. Insecure, hacky, manual and frustrating. But then you move on, hoping you don’t have to do it again. Well, here’s how I got to use the IPMI (iLO, BMC, iDRAC, etc) web interface of some old servers from my Centos 7 server:

Access the IPMI web interface

ws $ ssh -X server
server $ firefox $some_ip

Login, browse to the ‘remote control’ section (they’re all pretty similar), click launch. It pops up a prompt asking me what I would like to use, to launch jviewer.jnlp.

Install and configure Java

I found a guide which says to install and configure Java; java-1.8.0-openjdk was already installed out of the box, so it was just a matter of configuring it:

server # update-alternatives --config java

There are 2 programs which provide 'java'.

  Selection    Command
-----------------------------------------------
*  1           java-1.8.0-openjdk.x86_64 (/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.121-0.b13.el7_3.x86_64/jre/bin/java)
 + 2           /usr/java/jre1.8.0_121/bin/java

Enter to keep the current selection[+], or type selection number: 2

Configure Firefox to launch .jnlp files with javaws

Firefox doesn’t know how to run javaws, so it needs to be told, via these instructions.

server $ vim .mozilla/firefox/vgenq8rj.default/mimeTypes.rdf

Mangle Java security settings

Java (rightly) complains about security settings. It’s only for internal boxes on a particular network, but BeyondCorp thinking still makes me cringe. Open the Java Control Panel:

ws $ ssh -X server
server $ /usr/java/jre1.8.0_121/bin/ControlPanel

In Security, Exception Site List, I added the URLs of the servers I need to manage. It works. I feel dirty. I suspect I could install an older version of Java to skip this step, and feel just a bit dirtier.

Taming a Mac, Lubuntu style

I’ve supported and administered Mac servers and desktops before, but never leapt on the tech worker bandwagon of actually using one for my own work. At the new office, I was handed a Macbook Pro, so I figured it was time to learn.

Learning when to use the various meta keys is a job for my fingers to keep practising – I’m getting better, and hopefully it won’t break my brain when I head home to my Lubuntu desktops.

Some web searching led me to fiddle with Automator to help fill some gaps, but the resulting keyboard shortcuts didn’t always respond, so I’ve settled on some programs to help:

  • Slate for launching, switching, resizing and placing programs in the desktop
  • Mission Control gives me multiple desktops and shortcuts to switch between them
  • HyperSwitch gives me a more comfortable alt-tab (programs within the current desktop) than the full-blown command-tab (all programs)
  • Quicksilver promises more features than Spotlight, and so far delivers except for System Preferences contents

The three programs I’ve installed have more features than I’m using, and I may explore them later.

I’m content with how it’s all working, and I am hesitant to throw it all away and install Lubuntu. That’s partly because of the time spent doing so, and partly because I’m not certain about how well it would work. Hardware compatibility, external screen resolution, power management… things seem to work just fine in *buntu, but I’m quite sure Apple have written things to work excellently in OS X. I want to spend time using the computer, not twiddling with it. I was initially dubious, but my OS X is now Good Enough(TM) for me to get on with some work.

VMWare upgrade 5.1 -> 6.0 notes

In 2013, the fine folks at Interconnekt sold us a fine Dell PowerEdge T420 server to run VMWare and a bunch of virtual machines. VMWare came installed on a dual SD module, and this came in handy later. We quickly discovered that VMWare ESXi 5.1 has a RAM limit of 32GB, so we removed the other 32GB and sailed on.

Three years on, and we have bumped up against that limit, our VMs exhausting available supply. We could pare down our use on some machines, but knowing there is an upgrade available and some DIMMs in the cupboard, I’m upgrading.

Dell recommend their customised version of VMWare, so I downloaded that through the Drivers & Downloads part of their website. I was stumped finding the link for a while, but eventually found I had to ‘change OS’ from Windows to VMWare, and found it under Enterprise Solutions. There is nothing to suggest skipping 5.5 is a bad idea, so I grabbed 6.0 and burned it to CD.

Backup the VMs

I installed ghettoVCB and did a test backup run of all machines:

$ wget -O ghettoVCB.vib https://github.com/lamw/ghettoVCB/blob/master/vghetto-ghettoVCB.vib?raw=true
$ scp ghettoVCB.vib root@server:
esxi # esxcli software vib install -v /ghettoVCB.vib
 [DependencyError]
 VIB virtuallyGhetto_bootbank_ghettoVCB_1.0.0-0.0.0 violates extensibility rule checks: [u'(line 23: col 0) Element vib failed to validate content']
 VIB virtuallyGhetto_bootbank_ghettoVCB_1.0.0-0.0.0's acceptance level is community, which is not compliant with the ImageProfile acceptance level partner
 To change the host acceptance level, use the 'esxcli software acceptance set' command.
 Please refer to the log file for more details.
esxi # esxcli software vib install -v /ghettoVCB.vib -f
Installation Result
   Message: Operation finished successfully.
   Reboot Required: false
   VIBs Installed: virtuallyGhetto_bootbank_ghettoVCB_1.0.0-0.0.0
   VIBs Removed: 
   VIBs Skipped: 

GhettoVCB creates a snapshot, does a backup of that snapshot, then deletes the snapshot. It fails if a snapshot already exists.

esxi # vim /etc/ghettovcb/ghettoVCB.conf
esxi # /opt/ghettovcb/bin/ghettoVCB.sh -a -c /etc/ghettovcb/ghettoVCB.conf -l /ghettoVCB-`date +%Y%m%d`.log

Some VMs had old unused snapshots, and some were in an old format, probably from when they were copied across from previous versions of ESXi. Therefore those backups failed. Right click, upgrade VM worked. Delete old snapshots. A successful test backup took 24 hours to our NAS – slow, but that’s okay in this case.

I didn’t want to manage changes in the event of failure, so I shutdown all VMs before running the actual backup. 24 hours later, I was ready to upgrade ESXi and install more RAM.

Upgrade ESXi to 6.0.0 Update 2

I put ESXi in maintenance mode so it wouldn’t automatically boot VMs on next boot. I first wanted to see that all is well. Shutdown ESXi, physically removed all HDDs and network cable from NAS – for paranoia’s sake, boot up with the CD I previously burned. Select upgrade and the dual SD module. Dammit, the upgrade failed because there’s a CommunitySupported VIB installed – ghettoVCB! Boot up, remove it:

esxi # esxcli software vib remove --vibname=ghettoVCB

Shutdown and boot the CD again. The upgrade was simple as can be, and for interest I timed it: 8 minutes of processing time. ESXi boots, complains about missing HDDs, and works just fine.

I’d heard of another benefit that I am now going to realise: I don’t have to boot my Windows VM to run vSphere Client any more! I might still for some things, because web clients sometimes suck, but not having to is great.

Install 4 x 8GB 2Rx4 1333MHz RAM DIMMs

Now, on to the RAM upgrade. The machine is nicely laid out, with easy access to the DIMMs. Naturally I want to put two of my DIMMs into each bank of slots, but where? There are two used in each out of six. The information panel on the case was helpful but not conclusive. Over to the Owner’s Manual.

“Populate all sockets with white release tabs first and then black.”
“Advanced ECC mode extends SDDC from x4 DRAM based DIMMs to both x4 and x8 DRAMs”
“Memory Optimized (Independent Channel) Mode … supports SDDC only for memory modules that use x4 device width and does not impose any specific slot population requirements”

A bit of discussion led me to believe that I should just place them closes to the CPU and with x4 DIMMs it didn’t make a lot of difference. It worked. Sadly I left out the big plastic baffle from the case when I reassembled – not a huge problem, but I need to schedule another outage to reinstall it.

Job done. I’m happy.

FreeBSD and pg_upgrade

Thank you kreynolds, your post makes my current job look dead easy. I want to upgrade PostgreSQL from 9.3.12 to 9.5.4, and after checking through all of the release notes between the two versions, I have narrowed the relevant ones down to the two biggest releases: 9.4 and 9.5 (thanks in part to our devs not using the darkest corners of PostgreSQL features… yet).

A side note: PostgreSQL is a shining example of how to do release notes. All in one place, linked properly between versions, available back as far as the eye can see, to pre-1.0 in the 1990s!

Another win for documentation: the FreeBSD handbook. Curated by members of the project, a simple URL, and quality information.

I could use pg_dumpall > backup.sql, upgrade the package, then psql -f backup.sql postgres, or do it in place faster with pg_upgrade. FreeBSD doesn’t simply allow both old and new packages to be installed together, so enter jails.

# jailroot=/usr/tmp/pg_upgrade
# bsdinstall jail $jailroot
# pkg -r $jailroot install postgresql93-server

In the dialog boxes, choose a nearby mirror and no extra components. Wait for the installation to complete.

# su pgsql -c 'pg_dumpall -c | bzip2' > /usr/tmp/pgdump.sql.bz2
# service puppet stop
# service postgresql onestop
# pkg delete postgresql93-client postgresql93-contrib postgresql93-server
# mv /usr/local/pgsql/data{,.93}
# pkg install postgresql95-client postgresql95-contrib postgresql95-server
# service postgresql oneinitdb
# pg_controldata -D /usr/local/pgsql/data.93

“Latest checkpoint location” needs to be the same on master and replication slaves. pg_upgrade(1) has good information on the 16 step process.

# su -l pgsql -c "pg_upgrade -b $jailroot/usr/local/bin -B /usr/local/bin -d /usr/local/pgsql/data.93 -D /usr/local/pgsql/data -j 16 -k --check"
Checking for reg* system OID user data types       fatal
Your installation contains one of the reg* data types in user tables.
These data types reference system OIDs that are not preserved by
pg_upgrade, so this cluster cannot currently be upgraded. You can
remove the problem tables and restart the upgrade. A list of the problem
columns is in the file:
    tables_using_reg.txt
[root@flora ~]# wc -l ~pgsql/tables_using_reg.txt 
 38 /usr/local/pgsql/tables_using_reg.txt

Bother. I can’t use pg_upgrade. Oh well, I’ll talk to the devs for next time, and get on with dump and restore.