Friday, July 30, 2010

90% of all e-mail is spam

That's an incredible statistic that took me aback when I first saw it (in March from Symantec), which is why security vendors have been so successful in the development and takeup of anti-spam products – such as the ones that Google acquired and implemented into their Google Apps platform, where our customer mailboxes are now all hosted. Spam e-mail levels were also up 16% in the second quarter of this year compared to the first (according to Google).

Previously mailboxes for Virgin Media customers were hosted on three different platforms, all of which took a different approach to dealing with spam (and in fact viruses) coming as they did from the three ISPs that we ran before Virgin Media came together.

In the last few weeks, as the @ntlworld.com migration from the old platform to the Google one has completed, we've seen an increase in customers reporting an increase in spam – and I'd like to explain why this is, and what customers can do about it.

Our previous platform was configured to reject the most obvious of spam messages, so they would be either bounced or actually deleted without ever hitting a customer's mailbox or being seen by them – so, while customers may have noticed an increase in spam messages there's actually no more being attempted to be sent to them than there was on the previous platform, now they just appear in a user's spam folder on webmail.

The previous platform was classifying around half of e-mails inbound for @ntlworld.com mailboxes as spam – which is a heck of a lot of e-mail:
[Note: The volumes have dropped as mailboxes were migrated off this platform and to Google]

As you would probably expect, we always took a conservative approach to what was and wasn't classified as spam given that we didn't have the tools to allow a customer to 'fine tune' their own spam filter (unlike how we do now on Google) – but even then we were still deleting around 20 million messages per day of the most obvious spam without them ever getting to a customer's mailbox.

So, what should a user now on the Google platform do if they are seeing a noted increase in spam messages?

1. Tune their spam filter

In Virgin Media mail (i.e. webmail) you can tune the spam filter to tell it what a user considers to be both spam and not spam (as one person's spam is another's valuable e-mail).

This is easy enough to do, by opening the message and clicking on 'Report Spam' if it's a spam message delivered to a user's Inbox – like this one I received overnight:
[Note: Screenshot, like others, is of Gmail – where I import my Virgin Media Mail. Slightly different colours etc accordingly.]

Similarly if a user finds what is a genuine message appearing in their spam folder they should open the message in webmail and click 'Not Spam'.

Doing both these helps the platform learn what the end user considers to be spam and not spam and is used in the decision making process around e-mail delivery to them in future.

2. Add contacts

If someone regularly e-mails a user and they want the e-mails delivered rather than classified as spam, they should put them in their 'Contacts' (link on left hand side navigation of webmail) address book.

This also makes it a bit easier when typing e-mail addresses into webmail in future as it auto remembers them, as well as other features like being able to create a mailing list of people someone regularly e-mails.

3. Use filters

Once the spam filter has been tuned over time to a point where someone is confident that no genuine e-mail is being classified as spam, they might want to consider setting up a filter in Virgin Media Mail whereby any spam messages will be deleted rather than put into the spam folder.

There's more info on how to do this in the help pages on webmail (and we do advise to only do it when a user is confident no genuine mails are appearing in their spam folder), and I think this is something I must do myself this weekend given the amount of mail in my spam folder (which is down to my address having been used in public forums and on web pages for the last decade):A filter whereby mail from a particular user will never be classified as spam (in addition to adding them to the contacts list) can also be setup. This is done by following these steps:
i) Open a message from the e-mailer in question
ii) Click on the option to 'Filter messages like this' from the menu (use the down arrow) beside the 'Reply' link
iii) Enter the sender's e-mail address
iv) Click 'Next Step'
v) Select the checkbox beside 'Never send it to Spam'
Once done messages from that particular e-mail address will always be delivered straight to the Inbox and never appear in the Spam folder.

4. Prevent it in the first place!

Of course the best way to not get so much spam is to make sure it doesn't get sent in the first place - which is done by being careful where an e-mail address is used and who it is given out to, never replying to a spam e-mail and never buying a spammer's products.

Another good tip is to try and make life a bit more difficult for the automated software spammers use when you do post on the web such as on social networks. When posting on the web, use spacing around the '@' sign to do this – e.g. support @ virginmedia.com rather than support@virginmedia.com.

These, and other good tips, are covered in this BBC article.

However, it is important to note that a user will not be able to prevent all spam – spammers have software that guesses e-mail addresses in huge volumes and sends them spam e-mail. This is known as a 'dictionary' attack and, while we filter huge numbers of these attacks into user's spam folders, some will always get through to Inboxes.

We're going to add some further advice to our website to beef up the advice on how to manage spam in the next few days to provide more assistance.

Saturday, July 17, 2010

8 out of 10 prefer Google

For some time (as covered in previous blogs) we have been running a project to move our consumer e-mail services over to the Google Apps platform.

New customers have been provisioned onto the Google platform from mid last year, and we are now in the final stages of migrating the last of 6.8m existing customer mailboxes over to Google with the imminent completion of the @ntlworld.com Google migration - after which we open up @virginmedia.com e-mail addresses to existing customers who wish to adopt them and close the project down.

In April we undertook some questionnaire based research of customers who had been affected by the project in order to see how we've done and to make any changes needed for the remainder of the migrations - although clearly we have been tracking calls and feedback to our customer care teams on a daily basis throughout the programme.

We surveyed a statistically significant 975 customers, who were split as such:
  • 14% 'new' customers (who joined after we launched @virginmedia.com addresses for new customers and have only ever been on the Google platform for their e-mail)
  • 31% migrated National customers with @virgin.net e-mail addresses
  • 55% migrated cable customers with @blueyonder.co.uk e-mail addresses
The first question we asked was a general status check as to what people thought of the new webmail - and over 80% of them were positive about it:
Fig 1: Overall webmail rating of new and migrated customers. Click on image for a larger version.

One thing we particularly wanted to test was how the migrated webmail users reacted to the new Google platform, given that it is significantly different from the older interfaces that customers were on before they moved to Google.

Encouragingly three quarters (well, 74%) of them found the Google platform to be an improvement:
Fig 2: Comparison of previous webmail service with Google platform for migrated customers

A big consideration we had in moving customers to Google was the customers who still choose to use client software (such as Outlook Express or Windows Mail) rather than webmail - and that we wanted to minimise the impact to them, bearing in mind that some would still need to make changes in the interim and that all would in the longer term as we adopt the Google client settings.

We put a number of tools in place to minimise this impact, and are encouraged by how few client users noticed the migration - with many of those telling us that it was only because we both e-mailed them beforehand to let them know it was coming and afterwards to confirm their mailbox had been moved:
Fig 3 : Impact of migration on client software users

This is backed up by 9 out of 10 customers (89%) telling us that the detail of the e-mail communication was "just enough" for them and that 70% of users did not need to use the help pages:
Fig 4 : Customer usage of Virgin Media Mail help

We've been doing some usability work to address customers who found it difficult to find the help content, but encouragingly they managed to get by with very few customers needing to contact us compared with our forecasts:
Fig 5 : Actual customer contacts compared with forecasts. Scale removed as commercially sensitive.

We also asked customers if they wanted to share any free-form comments on how their experience has been, and I thought some might be interested in a selection of them:
Fig 6 : Free-form customer feedback about Virgin Media Mail

With the project coming to a close, our team are now working on future developments including improved mobile functionality - which is something our customers have been telling us they would like to see.

I had the privilege of being the commercial owner of the project throughout many of the key timescales, and would like to congratulate and thank the project team on doing such a good job throughout (you know who you all are), and thank my successor Giorgi and Francesca from our research team for the data and the nicely reusable PowerPoint slides :)

Sunday, July 11, 2010

Activate Me

One of the many projects I am working on at the moment is one with the broadband team to overhaul our cable broadband activation process and to rollout a new 'hub' device which merges the cable modem and wireless router into one.

In fact I have wangled myself onto the 'hub' trial, and my device that arrived in the post last week looks like this:This is for customers on speeds 20Mb and under, with a further device doing the same function to come for those on 50Mb and above.

From reviewing feedback from customers and staff members it came through loud and clear that setting up of a wireless network is difficult and can cause all kinds of issues, so the merging of the two devices into one and having engineers setup the network for customers as well as a much improved activation process will make life easier for one and all.The new activation process also streamlines the setup of Virgin Media e-mail addresses and the installation of other tools such as security and Digital Home Support for those who wish to use them, and we’ve done plenty of testing of the flows with customers also in recent usability testing – getting some brilliant feedback and making a number of tweaks thanks to the usability work.

It’s been an enormous amount of work from the project team, but as we’re at the stage of signing off the final activation flows and testing the physical 'hub' devices we’re not far off and customers will start seeing the new process soon … once we’ve got everything ready and everyone who needs to be is trained up.

I find it all very exciting really ... And on that note I’m off to test my new hub!