Fifteensquared

Never knowingly undersolved.

Which daily cryptic is the hardest?

Posted by nmsindy on May 22nd, 2012

nmsindy.

Every year on 31 December, nmsindy posts an annual review of the Indy puzzles on 15 sq giving average solving times for each setter.      Very often I’ve been asked “How do the other puzzle series compare with this?”      So I decided, out of interest, to tackle them all for one calendar month to see.    I chose March 2012.

I’ll start with the results, and then follow with some detail on the individual series.     Average solving time for all puzzles was 28.9 minutes.         Indy was the hardest (36.4), followed by Guardian (34.6), Times (32.3) and Toughie (30.4).   However these four figures are so close to each other that I’d say the standard of difficulty is pretty much similar.

Next, a little further back was FT (26.3) and Daily Telegraph was way behind that (16.7).     Also tackled the Sundays where Sunday Times came in at 32.3 mins, Sunday Tel at 15.3 and Observer Everyman at 13.8.

Methodology:   nmsindy does not approach puzzles in ‘race-the-clock’ mode but found it of interest to note the setting times when finished and to compare the setters and, now also, the different puzzle series.      With much practice at solving over decades and, as standard vocabulary is used in these puzzles, I can usually finish without consulting anything.   If I am completely stuck, I will go to dicts and other reference sources.   In these circs, I usually find the answer is a word I did not know or had forgotten.

Series:   Most will know this already but I’ll quickly list them out:   Indy gives setters’ pseudonyms and as there is overlap between the Sunday and daily editions I have considered them together.    The hardest puzzle was on 31 March from Bannsider (99 mins) – the easiest a Dac puzzle (13 mins).    Guardian also gives pseudonyms – hardest was Boatman on 22 March (86 mins) and the easiest – and also the easiest in the whole month in any of the papers – was the 5 March puzzle by Rufus (7 mins).      The Times does not indicate who sets the puzzle.   The hardest puzzle of all in any series was the Times on Sat 17 March (103 mins) – the quickest in the Times was 15 mins.     Toughie appears in the Daily Telegraph 4 days a week – Tuesday to Friday with pseudonyms being given.   So there were 18 puzzles during March – the hardest was Notabilis (66 mins) on 30 March and the easiest a Messinae puzzle (14 mins).     The comparison with the Daily Telegraph shows that the paper has achieved, I expect, what they sought when introducing the Toughie ie that it would be a much harder puzzle.

FT, I think, aims at being a little easier on average than the other series and this is borne out – the hardest (71 mins) was a Gaff puzzle (themed to coincide with Crufts dog show) – the easiest (11 mins) was a Sleuth puzzle.       The Daily Telegraph (where nmsindy began his cryptic solving a long time ago) aims to be much easier – the hardest was 43 mins and only three puzzles took more than 30 mins. The fastest time was 10 mins on 3 occasions.    Like the Times, pseudonyms are not given.

The Sundays involved solving 4 puzzles and a feature of them all was consistency in standard from week to week.    Sunday Times turned out to be exactly the same in difficulty as The Times.  Sunday Telegraph is very similar to the Daily Telegraph.    Everyman in the Observer is easiest of all as I guess is its aim as the title of the series would suggest.

Finally I’d like to thank all the setters, crossword editors and bloggers on the various websites whose work was invaluable as I checked the various solutions often quite a while after the puzzles first appeared.

A spreadsheet is available giving full details of all the individual solving times, pseudonyms etc.    If anyone would like this sent to them,  please ask indicating an email address (if I don’t already know it)

57 Responses to “Which daily cryptic is the hardest?”

  1. Paul B says:

    Thanks NMS. Yes, please rush a copy of your spreadsheet to:

    surfcake at tiscali dot co dot uk

    Cheers
    PB

  2. grandpuzzler says:

    Thanks NMS. Please send me a copy to:

    grandpuzzler at yahoo dot com

    Cheers…

  3. RCWhiting says:

    Thanks for that.
    Quite fun to read and obviously fun for you to produce.
    The small sample sizes and very subjective solve-method mean that there is little significance to the results.
    Nevertheless, good luck with any future projects.

  4. Quixote etc says:

    I’d say the sample size is OK and that the results give a good ranking of levels of difficulty — so, yes RCW, the results do have some significance and we should be grateful that there is one solver taking this amount of care — thanks NMS. Of course the method could be refined and the sample enlarged, but the result that the Telegraph is at one end of the spectrum for the average and the Indy at the other will not surprise the new i converts from the Telegraph who have found some of the hardest Indy puzzles far too hard. Actually the Telegraph with its back-pager augmented by its Toughie shows that it is on to a good trick! By the way, very hard and very easy puzzles can skew the averages so it might be useful to look at medians as well as means (and to plot distributions) — a task which I shall not volunteer for!

  5. nmsindy says:

    The medians are Guardian 31 mins, Indy 30.5 mins, Toughie 30 mins, Times 27 mins, FT 24 mins, Telegraph 15 mins. I’ve added these to the spreadsheet.

  6. William says:

    Smashing project nms, thank you.

    I don’t agree with RCW, results like this are always significant so long as the proper interpretations are applied.

    Appreciate your spreadsheet to madacres1 at gmail dot com

    Kind regards.

  7. nmsindy says:

    Have sent the spreadsheet to all who asked, converting the addresses to standard format. If any did not receive it, please mention this.

  8. Paul B says:

    Thanks indeed NMS, for a very entertaining sheet.

  9. Sil van den Hoek says:

    Very interesting data which for which many thanks, NMS.
    Could you please send me a copy too at: [email protected]

  10. RCWhiting says:

    William @6
    GIGO

  11. Sil van den Hoek says:

    Thank you Niall, all Roger.
    Perhaps, it is also interesting (certainly for Guardian puzzles) to compare your solving times with the day of the week.
    While I like all this, I am with RCW that statisticians will probably reject this survey as being, from a technical point of view, biased or not representative.
    But, yes, it tells us (not a random sample from the population of solvers, though :)) something (whatever that is).

  12. nmsindy says:

    Thanks, Sil. No one is claiming that it is more than one solver’s subjective view based on that solver’s strengths and weaknesses. Nonetheless, from the reaction, I’m glad to say it is proving of interest.

    Also, in passing, I might mention that NMS is a statistician and so therefore may have a penchant for producing analyses of this kind…

  13. JollySwagman says:

    Thanks nms – your dedication to the task is commendable.

    The only stat that surprises me is Times and Sunday times equal solve times – I only see them once a week – both in Saturday’s The Australian – about a month behind the UK (whereas everything else here is a month and ten years behind) but I usually find the Sunday Times easier and more fun – even when Anax is on.

    Perhaps you could scribble down marks out of 10 for fun, cluing satisfaction – whatever – as Einstein observed “Not everything that counts can be counted … etc” – something like that.

    Also, as observed by Q at #4, how tightly the times are distributed about the mean (that’s the SD isn’t it?). We have always been told that the higher level of (crossword-) editorial activism at The Times is intended to produce more consistent puzzles – I wonder if the stats bear that out.

    Plus – also agreeing with Q@#4 – the DT’s two-puzzle approach must surely be a winner with readers. No single puzzle is ever going to please the entire spectrum. Anyone with access to press bigwigs and bean-counters should push that line as hard as possible.

    I had a questionnaire pop up on the G site the other day asking all kinds of questions about why I read the paper and which bit I liked. Not a single mention of the puzzles. As a long-term expat, for years I took The G weekly principally for the puzzle. When the internet edition came along no further need.

  14. sidey says:

    Very interesting NMS, if the offer is still open [email protected]

    I too was a bit surprised as Swagman by the Times/ST being on a par. I had relegated the ST to the bottom of my interests pile.

    I also agree that the Graun are fools when it comes to promoting their puzzles. There used to be lively discussions on their now defunct Talkboards (which unfortunately were never properly moderated) which the ghastly CiF system can’t replicate.

  15. AndyB says:

    Most interesting. I tend to the Indy or the Guardian, one only, not quite every day. I’d have said that over the long run the difficulty is about the same, and that’s what the median solving figures are telling us.

    But a couple of subjective observations:

    first, completion time is the most obvious and best measure of difficulty. But I quite often find that the last two or three to go in take well nigh as long as the rest of the crossword. (And quite often, no namnes no pack drill, that’s because it’s an obscure word unkindly clued). Completion time may chiefly be measuring the difficulty of the most difficult clue. It would be interesting – if anyone ever did this again – also to have figures for getting to four left.

    Secondly, while some setters are universally agreed to be more difficult than the average (enigmatist/nimrod, bannsider etc), we probably all have setters who we personally tend to find difficult. I personally seldom make headway with Bonxie in the Guardian; comments on his puzzles here suggest others find him tough but not as realtively tough as I do.

  16. Quixote etc says:

    The two-tier Telegraph idea came from Will Lewis, when he took over as editor of the Telegraph and was dismayed by comments from his Times chums that they found the Telegraph puzzle too easy. I helped crossword editor Kate Fassett put together a team at short notice. WL is now a big noise at News International. The Times had a similar idea of publishing some second-tier esier puzzles and even commissioned some, but the idea was then ditched.

  17. Paul says:

    NMS, well done for doing this. May I too have a copy? Many thanks! [email protected]

  18. Qaos says:

    Thanks NMS. As a mathematician also, I’m fully aware there’s lies, damn lies and statistics. But that doesn’t stop this sort of project being very interesting to read :-).

    So I’d be delighted if you could also send a copy to: [email protected]

  19. anio says:

    Sorry if I’m a bit late but I’d love a copy – I’d like to work out the G’s average without Rufus among other things. Thanks for all your effort! [email protected]

  20. anax says:

    JollySwagman @13

    “…and more fun – even when Anax is on”.

    Oh, that’s nice. Right, I need to think of a suitable return curse. May you, er, er, – oh, I know; I hope you win DIY COW and have to judge for a week.

    There. Said it.

  21. JollySwagman says:

    @Anax #20

    Oops – it’s called a dangling modifier isn’t it? – like “Chest of drawers wanted by lady with Queen Anne legs”

    How’s this:

    but I usually find the Sunday Times more fun and easier – even when Anax is on.

    Maybe to be sure:

    but I usually find the Sunday Times more fun and, even when Anax is on, I usually find it easier.

  22. anax says:

    ;o)

  23. Ian Payn says:

    I’d be interested in seeing the spreadsheet, as well. Thanks for doing this.

    Ian.Payn at CharterChambers.com

  24. Rishi says:

    NMS

    Please email me a copy of your spreadsheet. TIA.

    cgrishi at hotmail dot com

  25. grandpuzzler says:

    NMS – I am following up on my request @2. Haven’t received yet. Since I am in Bellevue, Washington, maybe there is a problem clearing customs.

    Cheers…

  26. nmsindy says:

    grandpuzzler @ #25 – I’ve sent it again now.

    If you still have problems please send an email to niallmacsweeney at hotmail dot com and I will reply to it enclosing the spreadsheet.

  27. grandpuzzler says:

    Customs cleared! Thanks nmsindy for the extra effort.

    Cheers…

  28. JollySwagman says:

    NMS – Would appreciate the spreadsheet.

    jollyswagman at internode dot on dot net

    Thanks

  29. Mick H says:

    Fascinating stuff. Would love a copy to michael dot [email protected].

  30. Bamberger says:

    As a solver at the lower end of the food chain, I’d agree with Everyman being the easiest. As pointed out the others have different setters but I’d agree that on average The Telegraph and FT are next up. The BigDave44 website tells you who the Telegraph setters are -I’m lucky if I can get a handful of Ray T’s clues who usually has the Thursday slot. I don’t find a lot of difference on average between the Grauniad and the Indie though I have given up on Araucaria. I have finished the Times unaided twice but generally even when the solvers on the blog site claim it was easy, I struggle to get more than a handful -probably psychological. I’ve only tried the Telegraph Toughie on two occasions and both times I solved one to avoid the bagel but gave up. I think I unluckily hit on two very hard ones.
    Very interesting thread-you’d have to be very good to be able to do all six in a day.

  31. nmsindy says:

    Thanks, Bamberger, most interesting – I can assure I never did six in a day – the solving exercise was spread over 3 months so about 2 a day on average. Practice is everything, really, in cryptic crosswords. It takes a while – it certainly did in my case – before one can consistently solve puzzles. Studying clues afterwards can be rewarding too as, when all is said and done, there are only a limited number of clueing devices which will be used all the time.

  32. RCWhiting says:

    nms
    “Practice is everything, really, in cryptic crosswords. It takes a while – it certainly did in my case –”
    Indeed, and no intra- or inter-solver consistency measures.
    The former would be difficult since you cannot solve the same puzzle twice!
    The latter would be very interesting, might the effect be greater than that between setters.

  33. Huw Powell says:

    Wonderful little project.

    RCW, I don’t think this is being presented as Science, but as Geek Fun. The real project would involve a few scores of dedicated solvers, all agreeing to focus on at least one paper’s puzzle every day for at least a few months, recording solving times and “time until gave up & number of unsolved clues”.

    And it still wouldn’t be Science.

    I wouldn’t volunteer because I don’t solve in a “timeable” fashion – I multitask, I often solve in several sit-downs, and the obvious, I don’t finish all the puzzles I start so I couldn’t present a time stamp consistently. I print all the Grauniads, I like them a lot, although I have started skipping the Everyman. I do some FTs on the side, and yes, they are easier. I once finished about 75% of an Araucaria Bank Holiday puzzle. I also do the Puzzlecrypt monthlies and the Harvard bimonthlies, and recently fell in love again with The Atlantic’s old team at, of all places, the Wall Street Journal. Cut my teeth on these things on the back page of The Nation – which started posting Frank (RIP)’s classics from the old days a while back, which were insanely hard.

    I ramble.

    If the offer is still “live” I’d love to see the data at [email protected]

    All the best,

    Huw

  34. Huw Powell says:

    Oh and also, since the Grauniad struggles to provide a rise of difficulty level over the week – from Rufus on Monday to the Saturday Prize – I’m very curious to see if that is reflected in your results.

  35. sidey says:

    The real project would involve a few scores of dedicated solvers, all agreeing to focus on at least one paper’s puzzle every day for at least a few months

    You need to subscribe to the Times for that level of geekiness. The times for the times site is fairly nerdish too http://times-xwd-times.livejournal.com/

  36. louise says:

    Hi NMS

    I’d love a copy too
    mlcurrie at bigpond dot com

    thanks

  37. Margaret Tait says:

    Fascinating reading – please could I have a copy of the spreadsheet too, if it’s not too late (dmtait36 at btinternet dot com)
    Thanks

  38. Rufus says:

    Have only just come across this – please send a copy to [email protected]. My brief with the crossword editors is to provide puzzles that are straightforward and not too difficult, so I expect to be included under the “easy” section!
    Rufus (Guardian), Dante (FT) and Mondays (Telegraph).
    Many thanks in advance.

  39. Ape says:

    Interesting read, thanks nmsindy. But as someone who is pleased to finish a puzzle I’m curious, do you actually solve all the puzzles every time? You mentioned looking at dictionaries and other resources – are your solution times then a bit skewed by how long you wait before looking up e.g. a rare dog breed no one’s ever heard of?

  40. RCWhiting says:

    Huw @33
    I agree with pretty much all of that (including your discontinuous solving method).
    That is why my first post @3 included “Quite fun to read and obviously fun for you to produce.”
    and “good luck with any future projects”.
    Nevertheless,our society is swamped with nonsensical, misleading and downright dishonest statistics. These are often given spurious credibility by the media. It is never wasted to point this out on occasion.
    Comments like (William) ” results like this are always significant so long as the proper interpretations are applied.” show the effects of this.

  41. eimi says:

    Well, however misleading some may think such statistics, in the absence of anything more scientific I’m always interested to read nms’s annual reports on the Indy crossword and I’d be concerned if his data suggested that the Indy was significantly harder than the Times or the Guardian. What I can’t understand is why anyone would come on here to damn with faint praise what is obviously a personal project and a labour of love that is clearly, from the responses, of interest to many.

  42. Hamilton says:

    I don’t do enough of Guardian, Indy and Telegraph puzzles to really appreciate the figures, but from what I’ve read nmsindy you have set everyone thinking (which has to be a good thing) and it has been a fascinating thread to follow.
    I do attempt the FT every day, and thus, prompted by my colleague Dante/Rufus’s comment I wonder…
    - why if the Monday puzzle brief is to be straightforward and not too hard, do I get on fine with Dante puzzles but really struggle with Crux, who appears there every 2-3 weeks?
    - what would the effect of physical condition be on solving times – take me for example – most mornings I’m firing on all my mental cylinders but occasionally I’m the complete opposite – and the latter is not alcohol-related (!) and I get roughly the same amount of sleep every night?

  43. Paul B says:

    Re #41, the comments referred to are the latest in a long and tiresome stream of mostly either contentious or off-topic remarks. Just a view, of course.

  44. nmsindy says:

    Responding to some points made above particularly ape at #39.

    When doing the annual Indy report, I only give the average solving time of puzzles I’d solved correctly – there are always a small number where I would make an error or not be able to finish no matter what I consulted! For instance in 2011, I failed on 21 of the 364 or about 6% and this is set out in the report.

    When thinking of how to approach this new exercise, which was aimed as seeing how the series compared with each other, I gave some thought to this aspect before starting. I thought that a disadvantage of the previous approach was that the time spent at unfinished or incorrectly completed puzzles was totally ignored. This could sometimes be quite long as these would tend to be the most difficult puzzles. So the average time spent at a puzzle could be underestimated.

    There again were a small number of puzzles in this latest exercise where I made an error or could not get the last answer or the last couple of answers.

    I took a pragmatic approach to this and added a further 5 mins to the time I had spent at the puzzle for each word that I did not get right and this is the figure shown in the spreadsheet. Did not go into this level of technical detail in the original report as aiming to pick out key points only.

    Re the query on when I go for outside help, this would be after considering (mentally cycling thro) all the possibilities based on the crossing letters I had and not seeing a word that would fit and not being able to put anything together either from the wordplay.

    BTW, IMHO the only purpose of solving is enjoyment tho puzzles should be fair.

  45. Wanderer says:

    Many thanks nmsindy for this fascinating analysis.

    Thank you in particular for including your fastest/slowest solving times in each series. This certainly corresponds to my own experience, that I find some setters in a given series MUCH harder than others. And this in turn is one of the things I most enjoy. I only do puzzles which are blogged on this site (ie Guardian, Indy and FT) and I see it as a major strength of all three series that they include such a wide variety of challenge. Today, May 24, I found Nestor in the Indy by far and away the toughest; yesterday, it was Monk in the FT who held me up for longest. Another day the Guardian takes the honours. So my own (not very scientific) answer to the question at the top of your blog, ‘which daily cryptic is the hardest?’ is: ‘it depends what day it is’. I pass on my thanks to the crossword editors for mixing up the difficulty levels in the way they do.

  46. Richard Wilks says:

    Very interesting stuff. Could you send me the spreadsheet please?
    I’ve not been doing cryptics for more than a few years but get a lot out of it.
    People keep asking which is the hardest where is a good place to begin? etc.

    Now I’ll send ‘em your data.

    Many thanks

    Richard

  47. Pat says:

    Very interesting, and I, too, would love to see the spreadsheet:

    patmark291 at yahoo dot com

  48. Bannsider says:

    Great stuff Niall.

    Both a labour of love and a useful guide for all those editors I’d have thought.

    Plainly my puzzles are not tough enough though …

  49. tallboss says:

    Please could you send me the spreadsheet

    [email protected]

    Thank you

  50. Meic says:

    Just seen this, and I too would like a copy of the spreadsheet

    meicgoodyear at hotmail.com

    I have a daily train journey of 1 hour 10 minutes, which is enough to cope with the i, Guardian, FT and Times on most days as well as to read as much of the i as I can face. I only see the Telegraph if someone orphans a copy on the train, but would agree the Toughie is abot the same as Guardian and i.

    I’d agree the FT is generally easiest (median solve <8 minutes), and the Guardian and i/Indy a little harder (though I find Don Manley's Guardian puzzles harder than his others). Median solve for these 11-12 minutes. Overall times are censored (in the statistical meaning of the term) as unless a puzzle is really super (most of the late Bunthorne's for example) I only give any puzzle about 20 minutes max, and there are usually about 2 a week I cannot do in that time – this week GAFF in the FT, sometimes the Times Saturday, sometimes Boatman. I started with Everyman, but rarely bother now, having been spoilt by AZED and the Mephisto now being available online.

  51. Shuchi says:

    Fascinating study, thanks for taking this on NMS. I haven’t timed myself but of the puzzles I solve, my understanding of their relative difficulty agrees with these results: it is pretty clearly Indy > Guardian > Times > FT for me. I think unfamiliarity with some of the themes in Indy/Guardian contributes to it.

    I’d love to see the spreadsheet. My id is shuchi [at] crosswordunclued.com.

  52. mhl says:

    What an excellent project! Thanks for doing this, nmsindy. I would love to have a copy of the spreadsheet too, if you could email one to mark-[ the name of this site ]@longair.net

    Something else I’m interested in is how “bursty” different people’s crossword solving is – I often get stuck for a couple of minutes, but then after getting one clue, another three or four follow quickly. Other friends of mine very steadily put in a clue every 30 seconds or so. All the newspapers with interactive crossword sites could collect this interesting data, but I’ve no idea if they actually do (or how one could get to analyse it)…

  53. DROPO says:

    I too keep statistics on the difficulty of the FT and Guardian puzzles. I only complete about 1/5 of the Guardians and 2/5 of the FT puzzles. The “easiest” Guardian setters, of those that appear often, are for me Rufus and Orlando. The easiest FT setters are Gurney and Dogberry. I find Bonxie the hardest in the Guardian and Bradman the hardest in the FT. Everyman, I get almost 100%, and Rufus in the Globe and Mail about 4/5 of the time (as opposed to 2/5 of the time in the Guardian, and 3/5 of the time [as Dante] in the FT). I’ve been doing these puzzles a long time, but I am both an American and an idiot, so I don’t often get far on them :)

    I don’t seem to be able to get the Independent crossword – wrong version of Java, or something that I can’t get around :(

  54. jetdoc says:

    Coming to this a bit late, but please send me the spreadsheet:
    jane dot teather at jetdoc dot co dot uk

    Excellent work, nmsindy!

  55. Robin says:

    This is very interesting – if not too late, could you send me a copy too? orbin at tiscali dot co dot uk
    Regards, Robin

  56. Patrick Traill says:

    I too would much appreciate having your spreadsheet, sent to <Christian Name> dot <Surname> at t dash online dot de. Thanks in advance, and congratulations on your effort.
    (Incidentally, since I have to supply an address to post, would it not have been possible to use that — or just post the file for downloading – can this site not do that?)

    I too would have been interested to hear more about the spread in difficulty (or at least in times); I look forward to the data (if not too late!).
    I too feel that I vary a great deal in speed, depending on things like how well I slept; perhaps you could have got (or did get) a little more information by doing them in batches (noting the order!), though quite how one would interpret the results I’m not sure. The curious thing (which is clearer with Sudoku-like puzzles) is that I don’t feel as if I am thinking more slowly, it just takes longer on the watch, as though subjective time slows in proportion.

    I find myself all too readily/lazily distracted by Sudoku and Kakuro or a quick crossword, even though I find cryptic crosswords more satisfying than those.
    When I try one it is the Grauniad (on paper), but I seldom complete it; I know there is a setter who often seems easy to me, though I didn’t think it was Rufus; Araucaria is usually fun, and not always (all) hard.
    I don’t mind quite fairly freestyle cluing, but I do like prepositions to be consistent with the direction of the clue.

    Is it simply your style to refer to yourself sporadically in the third person, or is there some other significance?

  57. John Owen says:

    I’d be grateful get a copy of the spreadsheet. I think I’m in for a dose of humble pie.

    Thank you

    [email protected]

Leave a Reply

Don't forget to scroll down to the Captcha before you click 'Submit Comment'

XHTML: You can use these tags: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>


9 + three =