What follows is the more detailed artist radio comparison on which my Music Ally piece Slave To The Algorithm (Part 1): Five More Things Internet Radio Should Steal From Broadcast was based. If you haven’t read Part 1, it’s worth heading over there first for context, and then jumping back into the detail if you want to go deeper.
This listening exercise was about eliminating confirmation bias and levelling the playing field between the services. I’ve been a heavy user and/or occasional employee of all them over the years, but laboratory conditions are essential for any comparison that claims to be fair and controlled. (For the record, Spotify was my mainstay for on-demand listening going into this exercise, Last.fm for radio. I’ll be switching to another service for radio as a result of this analysis – find out which below.)
I listened to thirty songs of ‘Foo Fighters Radio’ on all the major services, thirty being roughly equivalent to two hours of broadcast radio, which is the minimum output window I would review when working with a new broadcast client.
Why Foo Fighters?
First of all I chose the genre I’m strongest in – alternative rock – in order to take some of the heavy lifting out of judging whether a general audience would consider each song familiar or unfamiliar. I chose Foo Fighters in particular for three reasons. One, they’re known to a general audience, and it’s mainstream listeners who will, in the end, decide whether smart radio emerges from the margins or remains a niche pastime outside the US. Two, I’m a fan, and when you’re staring down the wrong end of twenty hours’ similar-artist listening you need to be. And lastly, they’re an ‘in the middle’ band for their genre; there’s a good stock of popular-but-more-alternative stuff to the left of Foo Fighters, e.g. Queens of the Stoneage, but plenty of room to the right, e.g. Nickelback. I was interested to note which way each service would swing.
The aim was to listen with a radio programmer’s ear. I was listening out for general flow, property clustering and sound clash, artist separation and some version of clock programming. Important point to note here: I made accommodations for the fact that this is (a) artist radio and (b) webcast. Internet and broadcast radio aren’t the same thing – nor should they be – and there’s little point judging the former by the more rigorous standards of the latter. For example, ‘same artist separation’ of ten positions would be considered a cardinal sin by even the most tightly programmed commercial station, but you can afford to be a little more forgiving with personalised streams. (After a few hours listening I opted to accept 8 positions’ artist separation, or 4 positions for Foo Fighters, i.e. the starter artist.)
As I would for a broadcast client, I noted every ‘event’ in the stream – songs and ads – to get a sense of structure. I considered first whether the song was popular on the service itself – in the Top 10 songs by that artist, where this information was available – in order to ascertain whether familiarity was a factor in song selection and placement. Next, I asked whether the song was a radio hit in the territory I was listening from (the UK), to get a sense of whether it would be known to a general audience.
Then I noted whether I knew the song myself, and whether I liked it – this is personalised radio after all – before making a judgement about whether the selection and placement of that event would be considered good programming (with accommodations) by broadcast standards. This allowed me to give marks out of 30 for each service and rank them all accordingly for flow, deducting two points from the final score for poor artist range. The tables and my detailed comments for each service are available on request.
In all cases I listened on a free tier where there is one, on a brand new account set to the UK. This was to simulate the experience, as far as possible, of someone trying out internet radio for the first time. I elected not to skip or rate the tracks, which might seem counterintuitive at first – why ignore precisely the features that make smart radio so smart? – but for this first exercise I wanted to recreate the lean-back experience of broadcast radio. Where an option existed to scale music selections between, say, ‘artist only’ and ‘adventurous’, I set this to the middle. In future analyses I’ll compare these features of each service. Note also that I concern myself here only with music flow; an in-depth look at the overall UX of each service will likewise have to wait for future reports.
The most immediately striking thing is that American services are much better at artist radio than their non-US competitors. Pandora, iHeartRadio and iTunes Radio all scored more than 20 out of 30, while the UK’s Blinkbox – recently acquired by Guvera – was the only non-US service to do so. Disappointingly, almost all played streams made up entirely of American music, the exceptions (ironically) being US services Pandora and iTunes Radio.
Fig. 1: Artist Range Comparison
In terms of artist scope, the range was huge; the most diverse service played an impressive 27 artists over thirty songs, the least diverse a paltry nine (see Fig. 1 above). Incredibly, only Napster appeared to include Foo Fighters’ influences in its artist radio feature, making them the only service to serve up gifts like The Replacements, Dinosaur Jr. and Husker Du. Interestingly, The Pixies didn’t get played once in 22 hours of listening, despite the fact they’re listed – and linked to – in the Rovi bio that almost all services use.
None of the services appeared to have any category that resembled recurrents – any categories at all in fact – which to my mind is the real opportunity for hoisting internet radio out of the margins and into the mainstream (see number 2 of Five More Things Internet Radio Should Steal From Broadcast). Some services – Pandora, iTunes Radio, Blinkbox – were better than others at using familiarity to attract and hold listeners, but recurrents (songs which generate passion and delight the listener) were nowhere to be seen.
Delving deeper into the familiarity vs. discovery question, there was a huge range across the services. Have a look at the chart below; the green column denotes the number of songs out of 30 that are popular on the service itself, while the orange line shows how many would be known to a general rock audience. (Blinkbox, iHeart and Pandora don’t give this information, but where it was available I went with Top 10 by artist, i.e. if ‘Monkey Wrench’ is in the Foo Fighters’ Top 10, it gets a Yes for ‘Popular on Service’.)
Fig 2: Familiarity vs. Discovery
At one end of the scale you have Tidal playing almost exclusively songs that are popular on Tidal but unknown to a general rock audience. iTunes Radio on the other hand is playing songs which are both popular on iTunes and, for the most part, known to a general audience. Neither one approach is ‘better’ than the other necessarily; Tidal will be a more satisfying listen to the aficionado looking to go deeper into album tracks and less well-known songs, whereas iTunes Radio will appeal more to the mainstream listener, which is exactly where it should be. Deezer and Spotify, however, are playing songs that are neither popular on their own service nor known to a general rock audience. Fail.
Property scheduling (see point 5 of Five More Things) was pretty non-existent across the board, giving rise to frequent tempo and texture clashes, mood clustering, poor artist separation and virtually no female voices at all. But it wasn’t all bad. Two services in particular led the charge for flow, familiarity, discovery and diversity. Read on to find out who they were.
(Note: Beats Music has no artist radio feature currently and is unlikely ever to have one. When iTunes re-launches later this year, the artist radio feature will remain with iTunes Radio, while Beats – or whatever it ends up being branded – will also be integrated into iTunes as its premium-only on-demand offering.)
Artist Radio Comparison
A disappointing start from Spotify. Intrigued to know whether services would swing to the left of Foo Fighters (QOTSA) or to the right (Nickelback), I hadn’t anticipated a ‘down’. With the odd exception such as Smashing Pumpkins, Spotify’s entire Foo Fighters radio stream consisted of nineties modern rock dirge such as Stone Temple Pilots, Bush, Creed and Candlebox, highlighting precisely internet radio’s biggest challenge – that it’s just as easy to offend as to delight.
Foo Fighters fans – and I think I can speak for every single one of them – are probably more likely to be appalled by mainstream acts from their ‘own’ genre than by harder-edged artists from others. Put another way, The Prodigy is less likely to offend on Foo Fighters radio than Evanescence.
Developers may well retort that collaborative filtering doesn’t work if users don’t actively take part in rating content, but it doesn’t change the fact that other services, as you’ll see below, do a better job of serving up a more diverse range of artists, bigger hits and more satisfying flow with no interaction at all. Spotify was also one of only three services – the others being Rdio and Napster – to play the same song twice in the 30-song session.
Spotify did do a pretty good job of spacing ads, leaving a minimum of three songs between breaks and never playing more than one spot at a time. Songs out of the ads, however, were nearly always weak, even by the standards of Spotify’s own popularity indicator – a missed opportunity.
Spotify – 12 out of 30
By a country mile Rdio was the least diverse in terms of artist range (see Fig 1 above), with just nine – yes, nine – artists played across thirty songs, all of them from the Creedbox Chili Pilots end of the spectrum. (UK service Blinkbox was the most diverse with 27 artists across 30 songs.) Artist separation was extremely poor as a result, with many acts additionally being locally irrelevant to the UK.
Rdio was also one of only three services – the others being Spotify and Napster – to play the same song twice during the 30-song session. It played the most ads of all the services – 13 in total; after opening with an ad-free sweep of five songs (all of them outside the Top 10 most popular by artist), the spots came in thick and fast, with some gaps being as small as one position. Playing fewer artists than ads does beg the question whether Rdio’s ‘artist radio’ feature might better be described as ‘ad radio’. Overall, a deeply unsatisfying listen.
Rdio – 13 out of 30
Bravo, Pandora! A smart radio provider unafraid to play hits – unsurprising perhaps, given that it’s a radio-first service, and Pandora’s much smaller library puts it closer to broadcast radio in terms of programming philosophy. Whatever you think of Pandora, it definitely understands radio audiences, a fact reflected by its massive user numbers.
After Blinkbox and Napster, Pandora tied with iTunes Radio for diversity, with a total of 23 artists played over 30 songs, and it was the first service of the eleven I reviewed to play non-US artists – Muse and Franz Ferdinand. What, you might ask, do these two artists – Franz especially – have in common with Foo Fighters? Importantly, they have hits, and more importantly still, Pandora played them. Play ‘Take Me Out’ by Franz Ferdinand – a band only tangentially related to Foos – over ‘Weathered’ by Creed any day. Other than these British bands though, Third Eye Temple Peppers abounded.
Pandora played only 9 ads over the 30-song session, my only complaint being a spot placed after the very first song. Songs out of ads were almost always hits (based on US chart performance as Pandora isn’t available in the UK). Barring a few quibbles over artist separation, artist radio on Pandora was an engaging listen, with a good ratio of discovery to familiarity. Impressive.
Pandora – 27 out of 30
Deezer was musically much the same as Spotify, i.e. exclusively US modern rock. After opening with ‘Congregation’ by Foo Fighters we had two funereally slow and sparse ballads from QOTSA and then Smashing Pumpkins, which property scheduling on broadcast radio would never allow, especially so early in a stream.
Credit to Deezer for being the first to play a female-fronted band after nearly eight hours of cock rock, highlighting just how bad all services are at balancing gender – even allowing for the fact that this is a very macho genre. Deezer’s recommender seemed to have a particular penchant for Stone Temple Pilots, at one point playing three songs in a four-song sweep that were either STP or Scott Weiland.
Other than that, artist separation was pretty good on the whole. But as with Spotify, in all but one case Deezer followed ad breaks with songs that were unpopular even by the standards of their own users, offering listeners a great reason to tune out.
Deezer – 13 out of 30
What a breath of fresh air iTunes Radio was after eight hours of Everbush Audio Chains. iTunes Radio was the first service to venture into territory occupied by The Killers, The White Stripes, The Black Keys and other US alternative rock – precisely where Foo Fighters radio ought to be in my view – as well as the first to really weigh in on British acts like Oasis, Snow Patrol and Led Zeppelin. We even got a little Guns ‘N’ Roses and Van Halen, the novelty of which was almost too much to bear. With 22 separate artists in the 30-song stream, iTunes tied with Pandora for second-most diverse service after Blinkbox and Napster (27 artists apiece).
At the ‘discovery’ end of the spectrum things got a little weirder. Veridia were unknown to me and not at all up my street, but props to iTunes for spinning at least two female-fronted songs, even if they were both by the same artist. Only two tracks in the 30-song stream weren’t in that artist’s Top 10 most popular tracks, which meant that both ads and unfamiliar music were always cushioned by familiarity.
Just a couple of quibbles: the volume level on ‘The Pretender’ by Foo Fighters was almost inaudible for some reason, and song 28 inexplicably took us on a brief excursion into House music with a track by Matisse & Sadko. I’ll overlook both slip-ups on account of iTunes Radio being the only service to spin Sweet Child O’ Mine – possibly the highlight of over 20 hours’ listening.
iTunes Radio – 26 out of 30
As a former employee I had always bought into the received wisdom that, for all its failings, Last.fm’s recommendations are the best in the business. I’m not so sure anymore. It’s not that Last.fm’s are awful, just that other services (in radio anyway) are better. Having killed off subscription radio and effectively outsourced playback to YouTube and Spotify, Last.fm’s recommendations have to be good, because scrobbles and big data are now its USP.
The artist mix came half from Pearl Green Sour Peppers and half from more progressive stuff like Wolfmother, Probot and Muse. I had thought, given Last.fm’s choice of YouTube for radio playout, that selections might be a little less ‘out there’ than previously, but exactly half of them were unpopular (at least not Top 10) even with Last.fm’s own scrobblers.
Last.fm has never claimed to be mainstream – quite the opposite in fact – so making a direct comparison with Pandora, iTunes Radio or Blinkbox is probably unfair. But listening to it up against these other services does bring its woes sharply into focus; like many services, Last.fm’s business model depends on scale, but its core value proposition – discovery – is niche by definition (see the first of Five More Things Internet Radio Should Steal From Broadcast. As a ‘universal music wiki’, however, it is still second to none, and one wonders whether this might be its future.
Last.fm – 17 out of 30
Another impressive listen. Blinkbox Music’s ambition to go after the ‘passive massive’ is strongly in evidence here, with an engaging, mostly hit-driven playlist aimed at the mainstream. After two big openers from Foo Fighters – nice touch – we heard acts as broad-ranging as Kings of Leon, Skunk Anansie, Weezer, Aerosmith and 30 Seconds To Mars. Blinkbox tied with Napster for diversity, with 27 artists across 30 songs, tempered only by a special fondness for a dreary wankrock act named Adelitas Way. It was also probably the best at mixing the old with the new.
Blinkbox doesn’t give track popularity info, but to judge from UK chart performance approximately 17 out of the 30 tracks were hits. This seemed slightly below par for a service so avowedly committed to reaching mainstream listeners. One WTF?? moment came in the form of ‘Psychotech’ by eighties one-hit wonders Westworld (remember ‘Sonic Boom Boy’?), which stuck out like a sore thumb at a hand model convention.
With such a broad range of artists, Blinkbox suffered from none of the separation issues that plagued most of the other services. Other than a single spot for upgrading to Blinkbox Music More, there were no ads at all in the stream, even though I was listening on the free tier. (I have previously used the iPhone app, which featured quite a few ads). As the best music flow from a non-US service, Blinkbox Music would appear to be a canny acquisition by Guvera.
Blinkbox Music – 22 out of 30
I’ll make no comment here about the sound quality of the world’s ‘first high-fidelity streaming service’ – there’s plenty of debate about that elsewhere. I’m focussing here on music flow. While Tidal should be applauded for playing almost exclusively popular songs (as judged by its own users), note that the only two that weren’t popular were the opening tracks – ‘Where The Story Ends’ by The Fray and ‘Everything To Everyone’ by Everclear, a double fail given that the next two songs were by The Fray and Everclear.
Tidal was the second-least diverse in terms of artist range with a paltry eleven acts across thirty songs, beaten only by Rdio’s nine. Artist separation was accordingly very poor, with frequent instances of back-to-backs by the same artist (Pearl Jam, Everclear, Eve 6, Screaming Trees). In broadcast radio this is known as a ‘twofer’, and as a feature with presenter set-up it adds flavour, but I’m not convinced it works on internet radio.
A dreary and uninspiring listen for the most part, one that had me wondering more than once whether I had selected The Fray as my starter artist rather than Foo Fighters.
Tidal – 15 out of 30
iHeartRadio served up another 30 songs by Candlebush Blind Dog, disappointingly after iTunes Radio and Blinkbox, but in all other respects a well ‘programmed’ two hours. As with Pandora, I was listening with the help of a VPN, and since no popularity indicator is offered here either, my programming and familiarity judgements were based on US chart performance.
To my surprise I heard no ads at all, making it impossible to judge programming in and out of them, so iHeart’s score of 26 out of 30 might be slightly inflated. The only thumbs down arose from occasionally poor artist separation. Solid but pedestrian.
iHeartRadio – 24 out of 30
Oddly, Google Play was the first service to spin Rage Against The Machine, a novelty in itself after 18 hours of Red Audio Templebox. With no free tier on offer, there were no ads, so as with iHeart Radio all programming judgements were based purely on properties, flow and popularity. So many of the tracks in the stream were unpopular and/or unknown on the service that familiarity was a serious issue. An uninspiring listen over all, and not just because of tired ears toward the end of a marathon listening session.
Google Play / YouTube Music Key – 13 out of 30
A curious but refreshing mix from Napster, ranging from Royal Blood and Brendan Benson at one end to Nickelback and Evanescence at the other, again highlighting precisely internet radio’s problem – that it’s just as easy to offend as to delight.
Props to Napster though for being the only service to include influences in artist radio streams, resulting in a welcome detour into territory occupied by The Replacements, Husker Du and Dinosaur Jr. (I’m reliably informed by AllMusic that their artist radio feature also features influences, but since I can’t get it to actually work I’m unable to corroborate.) That only Napster played these artists probably says more about all the other services than it does about Napster, namely that they are missing a golden opportunity to improve both recommendations and familiarity in one go.
Credit to Napster too for pushing the limit as far as Lana Del Rey and First Aid Kit in the search for female voices that, while a stretch in terms of similarity, balanced the gender scales somewhat. All in all, Napster was a diverse and surprising listen with just enough familiarity to keep the listener hooked in.
Napster – 19 out of 30
So, for music flow, artist range and programming quality, Pandora wins by a nose with an impressive 27 points out of 30. iTunes Radio comes second on 26, although my sense is that if I had factored in UX as well as music selections, iTunes would have nudged ahead. (Arguably iTunes also has to work a lot harder to filter radio selections from its huge catalogue, several times larger than Pandora’s.)
Internet radio still has a lot of growing up to do from a programming perspective, but Pandora and iTunes are leading the way, followed by iHeartRadio which, possibly on account of being a broadcast/webcast hybrid to begin with, gives the impression of having been built by radio programmers who understand mainstream audiences.
I mentioned that this listening exercise has caused me to reconsider my allegiance to Last.fm for radio. As it’s unlikely Pandora will have another crack at the UK any time soon, I’ll be switching to Blinkbox Music in the short term, and when iTunes relaunches in the UK I’ll be glad to make iTunes Radio my mainstay, assuming UX and flow are on a par with the US service. If Beats can compete with Spotify for on-demand, I might even consider switching wholesale to iTunes.
So watch out Pandora, you have a serious competitor in radio for the first time – one that, integrated free into iOS, will achieve massive scale overnight, and now has the world’s greatest music recommender – Zane Lowe – on board. Spotify – you’re a long way behind the pack.