Tag Archives: Computing

Perl Testing: A Developer’s Notebook

By Chromatic and Ian Langworth

Work’s library got this in for me by request; I’ve done a lot of work on a module (Storage::Nexsan::NMP) that I hope to release to CPAN soon, and I’d never got my head round software testing. All the books and articles, and even courses I’d been on stressed the importance of writing them, and then showed simple applications I couldn’t see the point of. Having now written a module (my first for, what, 8 years?), and broken it while adding new functionality, this book made a lot more sense to me, especially as it lays out when and what to test.

Ignore any reviews you see saying its old, or out of date; its the only book I’ve found that actually walks you through what and how of testing from the beginning, rather than assuming you know about testing and want to know how to do it in perl.

Thanks to this book, I now have, including some ideas on how to write functional tests for a module that needs a Nexsan storage unit to connect to for all its functions!

Now I just need the time to write some tests and play around… I’m resisting buying the print and eBook bundle right now, as work have this paper copy and access to the fantastic Safari Books Online, but…..

UKUUG Spring 2011 Conference

I was lucky to be able to attend this year, as work had agreed to send me on it, and hadn’t reneged when I’d handed in my notice, although I did end up volunteering to pay for the travel and take holiday, so its only some impact for them..

Most impressed with the trains from Peterborough to Leeds – clean, on time and with free Wifi (for 15 mins, then chargeable), and power sockets at every seat.  Makes me wonder what the hell the cambridge operators are doing when the Leeds line can obviously muster this..  Also most impressed with Google maps/Navigation on my HTC Wildfire.  It directed me walking from the train station to event, and hotel, flawlessly.  Pity the battery didn’t hold out all day, though.  I really should be able to listen to a couple of hours of music, walk for 30 mins with the GPS on and make a couple of phone calls without using up all the juice.

Conference was good and increasingly attracting the DevOps crowd, which is a very good thing – preventing the conference from fading into obscurity and obsolescence.  Inspiring talks from Matthew Macdonald-Wallace and Adrian Kennard in particular, although I enjoyed the talks on Ceph, Git and DNSSEC also, and have some new projects to investigate as a result.

Thanks to Google for the excellent stationary swag, and for paying for dinner at the excellent Leeds Armoury.

Good fun, interesting talks and people and not too much intrusion into work for those attending via that.  Bloody good value.

Tech tips from India trip

So after my recent trip to Pune, a few observations in no particular order or importance;

Almost everywhere I went in Pune & Mumbai had plug sockets that took Indian and UK plugs, which was nice.

This Wiki has an excellent guide to buying PAYG sims with Data, although I didn’t take advantage of it in the end.  Its odd how much you get used to having pervasive, all you can eat data (India not having ubiquitous 3G coverage, yet).  It will be interesting to see how the UK market changes as all the operators go for restrictive caps and per-use charging (I thought they’d learned; the smartphones only took off when people weren’t terrified at the prices being charged per MB).

Having a real GPS makes a difference; I took my Garmin eTrex this time and it did a much better job fixing onto satellites on the Pune – Mumbai road.  Also the USB chargeable batteries I picked up in Morrisons for £10 lasted the entire 4 hour drive (but not the return trip – about 5 hours in total!)

I’ve proved that my Acer Aspire One with extended battery works very well as a movie viewer on airplanes – big enough to view, small enough not to get in the way even with a meal on the tray (yes, even in economy).  A combination of DVD ripping and MythBox means plenty of viewing.

I’ve never regretted the purchase of the noise cancelling headphones, although I think they make better in-ear ones these days..

Most hotels Internet connections appear to be wise to the ‘bring a wireless router and connect it up to the connection and put all your pc’s and smartphones behind that’ approach.  The wiser ones just provide it free, to as many devices as you have, of course.

The Art of Capacity Planning

By John Allspaw

I bought this about the same time as we set-up a Nagios & Munin system (and by we I mean Gatekeeper), but didn’t get round to reading it until just recently.  As I kinda suspected, it mainly emphasises getting good information and knowing what you want the system to do, but explained very clearly and logically.  There was also some good information on automated build systems, something I’m looking into recently.

I was looking for more information on the kind of statistics maths you need, but apart from some frustrating hints about using excel and linear regression (unusually with no links to find out more info), it wasn’t expanded on.

Ultimately, well written, a good introduction but I’d have liked more hard info on calculating metrics, which is ultimately what I bought it for..

Learning Nagios 3.0

by Wojciech Kocjan

I bought this after Gatekeeper set-up our Nagios system at work, and we needed to take over the maintenance.  It took a while to get round to reading it (fortunately, I have a lot of reading time at the moment, as long as I can do so with one hand..), and I’d figured out the basics thanks to Madhatter’s tips, but this takes you through the entire setting up and extending it, and I feel confident I can maintain and extend the system now – its got me exited as to what we can do with the system.

The book covers the usual setup and rollout, as well as monitoring external servers.  Pretty comprehensive, but I was surprised that there was no mention of integrating it with Munin, which does a good job of providing the external inputs that otherwise can be laborious to script, and with Munin providing such a nice graphing interface..

My only commends would be that his language of choice is TCL, which I’d argue is not widespread, but as he also has examples in Python, Java and Perl, it makes little difference – you can see how the ideas work whatever language its written in.  The author also straddles the line between howto and manual quite well, although some of the instructions (building nagios from scratch, etc), seem superflous given how much about Linux you need to know to implement it.  They don’t detract though, so I’m not complaining.

Basically, if you want to get up to speed quickly, this is a good book, and you don’t end feeling that now you need to go and read the more indepth books on the subject.

My MythTV Setup

About three weeks ago, my DVD recorder died.  I was pretty grumpy, as it cost ~£100, even if it was a ‘cheapie’ from ASDA, and it will take millennia to decompose.  Also, I was playing CD’s via it and it had the irritating habit of switching the audio to whatever tv channel it was tuned to whenever the CD finished – pretty jarring.  So rather than buy a new DVD recorder and CD player to integrate into my home AV setup, I thought I’d build myself a MythTV box, which would also allow me to pause live tv, use a live programme guide, etc.  I approached this with some trepidation, since MythTV is rumoured to be a pig to setup.

Short version of this; got it working, fantastic promise, but woeful documentation and still much promise unrealised.  But what it can do, oh boy!

My set-up;

Samsung SyncMaster T200HD (already bought previously from the nice people at RicherSounds.co.uk)

Yamaha Amp (Nine years old now with only RCA and optical in – anything new I’d buy would have HDMI interface these days)

ASRock IONSTAR Atom 330 Nettop PC (White, as the black was out of stock and not likely to come back in stock at the same price, according to the nice man @LinITX)

Hauppauge Nova-T Stick

RF Wireless Mini Keyboard with Touchpad

I bought the Nova-T first, because it was £25 in PCWorld, and if I couldn’t get it working I hadn’t shelled out too much.

First of all, I plugged it into my Acer Aspire One netbook, and installed the Myth Backend and Myth Frontend packages.  After about three hours mucking around I got the setup to  view live Tv and another couple of hours got recording working.  The most difficult thing I found was not the hardware but MythTV itself – the Nova-T was recognised by the 9:04 Ubuntu desktop install I had on the netbook.  Random parts of the IR remote that came with the USB stick worked, but not completely enough to be useful.  This was the last time the remote worked, and I’ve since given up on it.

Having proved the concept, I went shopping for the bits.  I wanted something that was as likely to work together, and fit together as easily as possible, but also small and quiet.  LinITX have been great on this in the past, so that was my first port of call, as all the stuff is most likely to work with Linux.  I ended up with the ASRock because it came with everything built in – including HDMI, and optical out, which was something of a surprise.  The downside was that there was no room in the box for a PCI style card (the usual way to do the TV tuner card, as the USB ones were historically flaky – this advice now seems superseded)- that’s OK, its one of the reasons I bought the USB one, but its one more thing that can get disconnected.

The box, when it arrived, is tiny – even smaller than the Shuttle PC’s.  Its not quite entirely silent, but its certainly no louder than my DVD recorder was if left on standby, and no complaints from A, the cats or Fred!

I installed Mythbuntu 9.10 – risky given it was days after the release, but the OS went on smoothly, autoconfiguring a number of useful things, such as VNC, SSH, Samba access and prompting me that restricted source drivers were available (I’m sorry, I went for them.  Can’t say it made that much of a difference, though), and kicking off the MythTV setup.

Setting up a working MythTV session is a lot of guesswork and headscratching, not everything is logged to the /var/log/mythtv/mythbackend file (or similar files in that dir), there are no manual pages for the Myth programs (but are for their perl scripts that pull in the listing data – bizarre), and the myth setup forces you to do all the work in the graphical frontend – which promptly didn’t scroll the output, so for interactive pages, it appeared to have hung.  Major bug and PITA – I had to google for ways to turn off the full-screen aspect of the applications to regain mouse control on the ‘scroll bars’.  You might be able to run these scripts manually – but good luck finding out what they are, or where, or how to invoke them.  Its all possible, but you need to read the output, and the logs, and the commands with –help to get an idea of what is going on.

Myth backends are basically three parts; the TV tuner selection (mostly painless if you can see it with dmesg or lspci/lsusb etc), allowing you to have multiple tuners (nice touch, elegant, but introduces a lot of complexity), the channel selection (what channels are available, and wheer you get the listings information), and tying the two together.  Yes, the last is a manual step, and the most likely to go wrong, in my experience.

The second, listings data is what caused me the largest amount of problems.  The MythTV box was up and running within a couple of hours, but it took another week of late night hacking to get the listings data to both appear on screen, and in the schedule database (this worked fine in 9.04, perhaps bugs were introduced in the 9.10 version?).  The idea is you get the listings data from either over the air info, or another xml source.  I ended up needing to configure mine to get both – as with just the EIT (over the air), there was no schedule info for recordings, and with just the XML (from the Radio Times, thank you Auntie Beeb!), there was no listings info on the interface.  Most bizzare.

Useful command to read the help for here is mythfilldatabase – use the ‘-v all’ flag, and watch the output closely.  Also, check your ~/.mythtv directory contents.  If your Mythfilldatabase is failing every night, try running it with the –update flag, and again, watching the output closely (there’s a lot).  I had to delete the ‘community channel’ (whatever that is) from my channels file as the xmltv perl modules have a bug in the Mythbuntu 9.10 release that errored out on this (which caused the whole process to fail with non-obvious errors).  Developers!  I should not have to consider running strace on your app!  Use syslog, FFS!

Websites & tutorials that were helpful were;

MythTV Ubuntu Installation Guide (This is what got me started and pointed me at helpful docs – thanks!)

‘Ethics Gradient’s guie t MythTV DVB setup (gotta love another Iain Banks fan!)

‘Mythic Beasts’ MythTV Setup

The Ubuntu Forums (I never actually posted a question, but felt I could have done, other people had had similar problems).

Don’t bother with the MythTV site – the documentation used a custom HTML presentation that takes up a third of the screen with useless borders, and gives you no diagnostic info, and makes lots of assumptions as to your knowledge.

So what’s cool about it?

Well, having a box that records live TV, can have multiple tuners, and records TV programs in a format I can copy off and watch on the laptop, with a pretty GUI that A. can use, that also plays DVD’s is cool in and of itself.

Its open source, so at least I *can* troubleshoot it.

There are a number of cool keys to control aspects of playback if you’re using a keyboard. Most people use some kind of media centre remote, but the keyboard works find for us now.  I have a bluetooth remote on order that should relegate the keyboard to advanced functions only (don’t recommend the keyboard I bought actually, the RF interface is lossy/subject to interference – last thing you need when trying to type terminal commands and not having every key press register..).

Built in ad detection and a keypress to skip – works too!

Records both TV and Radio.

MythMusic plugin – can import to Ogg, mp3 or FLAC – seriously thinking about re-importing all my worthwhile music as FLAC.  playlist functionality takes a little getting used to, but better with MythWeb (see below).  Seriously needs a playlist importing tool.  Album art appears to be broken – I copied across a load of FLAC I’d added art to, that it didn’t see.  The Myth Docs for this are actually pretty good.

MythBackup plugin – save programs to DVD, to play in any DVD player (not tested this).

MythVideo – play any video files you might have around the place (say, DivX..).

Backend/frontend seperation.  You can install just the frontend, point it at an existing backend, and use it over the network, inc streaming.  Not tried this, interested to over VPN..

Remote controllable via telnet interface – there is a Android app for this, amongst others.  Had issues getting it working, so stay tuned (hah!  See what I did there?).

DVD import -  not tested this yet, but should be able to rip DVD’s into the internal video library like you would a CD.

MythWeb – this turns the whole experience from great to awesome.  An HTTP interface that allows you to control the box, book & delete recordings,stream or download recorded progams or music.  Makes searching the listings & booking easier than using the remote, not that that is particularly hard.  Can be password protected (although this means the http user/pass is needed for every streaming file – a problem in a music playlist), so I may make this available via the router for use outside the home..

MythExport -  a plugin that transcodes the files into mp3/ogg, or iPod sized video if requested after the program is recorded and creates an RSS feed for the resulting file.  Having problems getting this working, but sweet if it does – all those Radio 4 programs I miss because BBC doesn’t podcast them yet, I can grab and create my own podcast feed of!

MythVodka – a plugin that is a wrapper around iPlayer.  Again, not got this working (but not tried yet), so…

Its a server in its own right.  So I can run podcastamatic on it, use screen for multiple remote sessions, and generally do anything I can with a Linux desktop.

ADSL ‘iPlate’ via BT

A colleague at work recommended this to me.  It sounded a combination of unlikely and plausible so I thought what the hell.  After BT making it very difficult to order with a credit card, I ended up using Paypal.  If you’re with BT for broadband, they send them to you for free, just for asking.  For the rest of us, its £7.99

The idea is that the original design of the BT socket provides a small current as a ‘bell ringer’ for the days when the bell in the phone was powered by the phone line.  With extensions, particularly, ADSL connections can pickup additional noise from this, and the iplate is an extra filter to fit to the main socket of the connection.

So, does it work?

Before;

Before

Before

After;

After

After

Of course, these measurements were not taken immediately, and there are a number of things that could interfere with this, but even so, its pretty impressive!