Why surveys aren’t very useful
I’m on the train and have just been asked to fill out a survey bfor the train company. It’s a double-page spread of questions about my journey and how satisifed I am with the service provided by Southeastern Trains. There’s no space for qualititative feedback, but I’ve made some of my own (including writing “WTF?” next to one of the questions).
These kinds of surveys are really a fig leaf-like attempt to ignore the important questions. For instance, a number of the questions ask about one’s impression of the quality of information provided at the station and on the train. It gives me the usual five-point spread - do I think they are very good, good, neither good nor bad, poor, very poor (or don’t know). Now, my impression of the the announcements provided on my train service to London is that there is plenty of them and they provide more than enough information. Actually, too much. At every stop, it tells you every stop that you are going to go through. Now, there’s twenty or so stops on my line. Between my station and London, there’s nine stops - sometimes more, sometimes less - meaning I get to hear the station names between my home and London forty-five times, or just over one-and-a-half times a minute. That’s too much information. I don’t want to pine for the good ol’ days, but only a few years ago, on the slam-door trains, these kinds of announcements were not made. Do we really have so many people getting on the train and not knowing where they are going that it justifies aurally spamming people with this information.
The survey also asks about my ‘personal security’ - how at risk I feel either when on the train or at the station. I don’t feel anything about personal security. I’ve mentally trained myself out of my fears, because fear is often highly irrational. I want to be guided by reasoned evidence, not by fear. And I try to take the counter-point to the fear - the inconvenience caused by the solutions to other people’s feelings of fear. I mean, read Bruce Schneier.
Another thing I wanted to pick up on is a section of the survey which asked me which words I feel best describe Southeastern trains? Well, the words “train” and “company” pretty much sum it up. I try not to be one of these people who has a ‘relationship’ to my toothpaste. Similarly, I don’t think of a train company as “community spirited” or “dynamic”. They only enter my thoughts when, say, they fuck up my journey home and cause me to endure stress and inconvenience. When they don’t do that, I try not to think about them at all. Yes, that’s not very loving or community-spirited. But then they do access to my bank account a couple of times a week, so I hope what they lack in feelings of warm cuddliness is made up for in cash.
Another question: how likely am I to recommend making a journey on Southeastern to my friends? Well, since, from where I live, they have basically a near monopoly on ways of getting into central London, I’d have to say they come top of my list. That’s pretty meaningless. My local take-away curry restaurant puts on the front of it’s menus “Voted No 1 Tandoori in [where I live] or surrounding area”. I do like their baltis, but, seriously, this is a bit like me declaring myself the number one vegetarian, atheist Semantic Web blog in all of Sussex. If you can take what you do and put a word “the” in front of it and it still make some sort of sense, then why bother? (I was going to bring up Bertrand Russell and definite descriptions there…)
As for these surveys, I have to say it’s rather weird they are handing them to me on a mid-afternoon train. The number of people on this train who have had a shit experience of the train service is likely not to be tremendously high. If you really want to know what people think about Southeastern trains, why not get the rush hour train home, go on there with a microphone and ask people their honest opinions. Compile that down into a representative tape of what people think and play it in the next boardroom meeting.
Also, asking people to put the post-codes of their homes and offices, as well as their e-mail addresses, then leave the surveys on the train is a very easily exploitable security hack. Read Mr. Schneier’s blog as linked above, and you’ll start spotting them everywhere.
Surveys have some big problems - they get you the answers they want. If Southeastern want to know what people think, they should join the blogosphere, read Twitter (here) and GetSatisfaction.
