Sony’s Walkman forever changed the way that people consume music by allowing them to listen to their favourite music, privately, wherever they chose, even in crowds. The portable music player remained fairly static for ten years or so until CD came along, bringing higher fidelity and more convenience. Aside from a small flurry of activity around the time MiniDisc appeared, it took the introduction of personal digital music players to reignite consumer interest in mobile music and show them what really is possible.

With a fall in memory prices, portable MP3 players started appearing and users could wander around listening to around 16mb of tunes compressed so heavily they sounded like they were recorded in a diving bell. Then suddenly, hard drives were small and cheap enough to store 5GB of music on, and the world hasn’t looked back since. Sony have lost some of their dominance over the portable audio market as companies like Apple and Creative enjoy huge market share with players like the iPod and

Formats and colour displays aside, there isn’t much to separate digital music players apart from the amount of tunes they can store. How can Sony, the company who invented personal portable music and traditional dominator of the field revolutionise it once more by introducing something that really is different?

The answer might well be StreamMan – and the surprising thing is that it’s not really about a gadget at all.

As Simon Perry is always fond of reminding me, when consumers have access to thousands of pieces of media, how do they decide what you want to listen to or watch?

StreamMan’s current incarnation is as a stream music service to Symbian mobile phones – though its potential goes far beyond that. Independent of what ever hardware Sony may choose to deploy it on, StreamMan is really about finding music and creating intelligent channels, but more about that later. Its applications go beyond just music and phones, but to films and other digital entertainment and other platforms – such as suggesting what you want to watch tonight on television.

In short, StreamMan is all about metadata – information about the media contained in the system. Tracks are categorised and described with fifty fields of information. If a user says she likes a particular track, then StreamMan can create a whole channel based on similar tracks – and the more data it captures from the user, the more accurate the results are.

I spoke to Robert Ashcroft, Senior Vice President, Sony Network Services about the StreamMan concept, and what it means for the future of music and media discovery.

Tell me how the StreamMan concept came about?

We’ve seen portable audio devices coming up with more and more capacity, where you can just put enormous collections of music on them. This begs the question of whether people actually want to pay for all of that content because you might be walking around with, in the case of our own NW-HD1, [Sony Style] 13,000 songs or somewhere in the region of €13,000 worth of music in your pocket.

What’s been happening is that people are getting their music from a variety of sources. Some of it from paid downloads, some is captured in the wild, some is ripped from CDs – but you start getting to a point where people have access to an enormous music collection. The question arises, if you really push this to the limit is – ‘If you have every piece of music that had ever been written on your hard drive, which is not inconceivable, how would you decide what to listen to?’

This is really the original motivation for StreamMan – if you have an intelligent personalisation engine which becomes your personal DJ, it can play you music and you can react to that music, saying that you like or dislike it. You can train it to send you music you like and you can save lots of different channels that correspond to different moods, different contexts and different types of music – then you can pick amongst your personalised channels and discover music, and make up playlists if that’s what you want to do. Or you can just let the service suggest stuff to you!

You can do that with any large collection of music, whether you owned it all on your hard drive or you were streaming it from a central server.

So StreamMan is separate from its hardware presentation, it’s not about a device – it’s about intelligently finding music that you like?

Yes. Ultimately if you crystallise it down to the absolute essence of it, that’s what it’s about. It’s liberating in that sense. With that thought, you can then go into lots of devices and applications – obviously the one that springs to mind and is the first implementation of it is streaming music to mobile phones.

Will it be implemented in mobile jukeboxes later on to help users find the right track amongst their 13,000 tunes?

It’s a possibility – but you have to understand how it works first, to see that we’re a little way off from being able to do that. You have to start off with a fully normalised [Hyperdictionary] database of music. The whole music industry is album-centric in its organisation. Imagine how many albums, including compilation albums, have the same recording of Candle in the Wind, from Elton John? If you normalise it, you may, say end up with five or six major versions of it.

You will still have a substantial music catalogue, but it’s somewhat shrunken from the numbers that are bandied around by some online services with regards to the number of songs that they offer.

You move into a song structure rather than an album structure, and you know how many albums a song is represented on. If you then then go further, which is what we’ve done with the music on the StreamMan service, you then characterise each song with, on average, around fifty objective and subjective characteristics that then describe it. That then forms the basis of your intelligence engine. When you say that you like or dislike a song as you listen to it, the server looks up the fifty or so characteristics of the song to understand why. It might be something to do with beats, or cadence, or instrumentation or pace.

In order to bring that intelligence, you would have to have a database on your hard drive that held all that information about the songs or at least be able to look it up with such accuracy that you knew exactly which song you were looking at.

That’s an incredible amount of metadata to compile.

It is – and this is why the StreamMan personalisation engine is so powerful. We’re probably some way from this sort of high-value music database being available in any format other than on our servers – and if it’s the source of our intelligence, then we’ll probably keep it on our servers for quite a while!

It’s an incredible piece of intellectual property that you could probably license and turn into a revenue stream on its own.

There are many things we could do with it – right now it’s powering the StreamMan service. On top of this very rich database we have our own personalisation intelligence which powers the StreamMan Player. It’s not just the database, it’s what use you then make of that database.

So we started from the vision of ‘If you have access to everything, how do you decide what to listen to?’. And then we had a lot of hard work to do to turn it into a practical, easy to use product.

It’s now available on Symbian smartphones working under GPRS, and serves a 16 kilobit AAC mono bitrate because all of these practical elements bring it to market in today’s network environment.

This doesn’t mean that we can’t see it evolve in the future because the essential vision is for an intelligent interface for music.

My job is to run Network Services – in our own minds, the question was ‘Is this network intelligence? Is this a virtual product? Is it a service?’ Whatever it is, we think that’s it’s very powerful.

We’ve just launched the second version of StreamMan, in Finland. It had a soft launch in June, which was literally just for mobile streaming. Teliasonera have seen it as a convergent product where you can listen on your mobile phone, you can interact and you can train your stations until they really give you the music you want and you can listen to them on your PC. It’s a passive player, it just plays the personal stations that you’ve created – but there we’re able to do 96kbs stereo, which is really high quality sound.

It goes well beyond the current personalised web radio stations because they still come from a search- and genre-based mentality. If you imagine this was voice recognition, you could have a computer on the wall and say to it ‘Play me some music for a party.’ Pick your genre – then do you want happy, powerful, relaxed, romantic? Is it action, chill-out, driving, party? It’s a completely different way of getting to different types of music. What’s your context? What’s your emotional landscape? What type of music do you like generally? Then you can choose roughly what decade – then it starts firing music at you.

If you ask StreamMan to come up with a suggested list, in each case when you’ve defined the parameters come up with fifteen songs. If you enter the same parameters over and over again, it will generate different lists each time, by saying ‘Well, we’ve looked in our database and we’d like to suggest these.’ If you see a song you like or recognise, you can start a channel based on that song – then you can train it until it’s the sort of music that you like.

StreamMan has 40,000 normalised tracks on it and it’s heavily influenced by Finnish content because that’s the market. It covers more than 90% of the available Finnish catalogue. It would take a huge amount of effort for an individual to acquire that content on their own, so it’s a very convenient service with a very powerful suggestion engine.

What we’ve found so far is that it very much appeals to 30 to 50 year olds, because we all know the music we like, but don’t all have the time we used to have to devote to getting it. But that’s just today’s picture – who knows where it’ll get to when we’re able to bring it to mass-market phones.

What about other markets? Will you be rolling it out to other European or American markets soon? What’s next?

Yes we are, but I haven’t got anything to announce – but we expect to have some announcements soon. It’s a business to business, server-based system so we can roll it out very quickly – but we have to interface with the phone operators’ billing systems and customer registration systems, and then it will appear throughout an operator’s network.

It seems absolutely ideal tool use with for Sony’s Connect music store.

It’s funny you should mention that. The one thing the music store doesn’t have is a web radio service – it doesn’t take much imagination to see that we’ll have one in a forthcoming release. We’re going to watch them both evolve and we’re going to combine StreamMan’s intelligence as we see an opportunity to do so.

Personal devices are converging, and handset manufactures are pushing phones as games consoles, music players and cameras. If there’s a decent phone out there that plays music at an acceptable quality level and has StreamMan integrated, might it not cannibalise Sony’s own Walkman business?

It’s always been true that you get multifunctional devices and you get dedicated devices – and StreamMan can appear on a moderately priced smartphone or on a dedicated device. It’s all engineering in the end.

What about applications other than music?

The work on the categorisation of the entire popular music catalogue, which has been proceeding apace, has been under way for the last nine years. We’re a couple of years away from having a complete set, before we move onto Jazz and Classical. It’s a very laborious task, as you can imagine, having experts do this value-add on the entire music catalogue. The music catalogue is a much more laborious task than the video catalogue, because there is just more music out there, it’s been going on for longer.

We think this is a very fruitful direction in terms of giving people intelligent access to entertainment content. I don’t want to get ahead of myself, because today we’re launching it with StreamMan as it is – I’m just sharing a vision with you. In the end, it’s not about the technology, it’s not about someone coming up and saying ‘We’ve got a 40gig device with a sim card.’ That’s not the point. The point is, the intelligent content management interface.

A lot of the devices that are being launched now will be entirely out of date in a year – but this project won’t date. After all, you started work on this nine years ago when you started applying metadata to you music catalogue, and it’ll still be valid ten years from now.

We’ll continue to calibrate it and improve the user experience, but we think it’s a very powerful idea. The evidence we have, because we’ve been live since June, is that the average length of time a song played on the service starts off at between 40 to 50 seconds, and that’s a combination of songs that are listened to throughout and songs that people skip within five seconds. Over a period of about three or four weeks, we see that every user follows a pattern, that they start out listening to only a few seconds and it rapidly increases to where they’re listening to 70 to 80 seconds. What that means is that they are training their channels and we’re delivering increasingly the music that they want to listen to from beginning to end. Ultimately, that will reach an asymptote of around 2.5 minutes, as the average song is three minutes – so essentially they are listening to everything and just occasionally skipping or whatever.

You’ll be capturing that user data and using it to improve the service in the future?

Absolutely – and the other thing we’re able to do is to share that information, on an aggregate level, with the content owners – so it becomes a very powerful feedback mechanism. There is enormous interest from music labels in getting direct, accurate feedback on new content, it gives easy access to back catalogue, for the mobile operators it’s a compelling data service, and for us, it’s a very interesting entertainment product – we think of it as ‘music beyond the iPod.’

The people who are doing portable audio really have to think what’s new and what’s next – what’s the leapfrog concept? And we think that StreamMan is a new concept.

Sony Network Services