In October I blogged about our first exploration of Snapchat as a tool to communicate and engage with people on the work of the Foreign Office. I signed off with a challenge to myself to understand how best to measure and evaluate our success on an app with very limited analytical capability and no website for API data. I’ll lay those challenges out here.
- We can measure our follower growth but the Snapchat app doesn’t provide any data on demographics. We know it’s a primarily a youth dominated channel but how do we know the audience we’re building is the one we’re aiming for?
- It also provides very limited analytics on engagement with those you do reach. You can see how many people viewed, took screen shots or opened a snap but the only way to know their reaction to your content is for a follower to directly message you back. You have to work a lot harder for that kind of engagement from your connections than a retweet, favourite or comment with which you’d normally deduce some quantitative and qualitative goodness.
We’ve continued to build our presence on the app with event driven content such as some recent behind the scenes footage at Wembley stadium for the Indian Prime Ministers visit but its been a smaller, more targeted project, with which we had some genuine success with measuring impact.
I’ve recently been providing some digital support to our outreach recruitment team who are tasked with boosting the FCO’s fast stream intake for a more diverse workforce, particularly those of BME and low socioeconomic backgrounds.
I spotted an opportunity to create a suitable product for the outreach team, build the very audience we’re looking for on our new channel and extend our understanding of how to evaluate given the very specific target audience.
The idea was simple. We had 5 overseas posts record several snaps that showcased the variation, scale and diversity of the FCO. Scheduling these one after another over the course of a day curated a vibrant ‘#BeForeignOffice’ story that lasted for 24 hours. Key to this was the story being created the same day the outreach team then delivered a careers event at Leeds University where they directly promoted the channel and its latest tailored story. Gaining support from colleagues such as those that run our LGBT group @FCOFlagg also helped produce a really focused product. Colleagues in Budapest, Brussels, London and Washington DC also understood what the content needed to look and feel like.
The end results were 5 posts in 4 countries generating 21 snaps to create our story. The first snap was viewed 328 times and the last snap in the story 280 times.
- Dividing the last snaps views by the first provides us a completion rate of 85%.
- Snapchat has confirmed views are unique so we can work out some form of engagement rate with each snap from the views in comparison to our follower count.
- We can calculate our average view count of 279 by looking at all the snap views together so we can now also gauge what content works and what didn’t.
- We also know how many followers we added so we can see our growth rate of 40%.
So we’ve covered reach, growth and what success looks like. But what about qualitative stuff like reaching our target audience and sentiment analysis?
- Given we promoted it directly at a University we could be more certain than ever before that we’re reaching the target audience. Although still not infallible, because the outreach team will be focusing on a broad range of universities up and down the UK we can also be more confident we’re reaching the intended demographics via a targeted delivery.
We received live snaps back direct from the outreach event at Leeds University but there was scope for more feedback so when the outreach team returned to the office we recorded another video in London that asked our followers if they were at the Leeds event and would they consider joining the FCO.
- We received 14 snaps in return that answered yes to both questions. If all new followers were from the Leeds event that’s a 9% positive success rate but given creating Snapchat content can be labour intensive we also promoted it on other social media accounts to really eek out its return. However, growth from cross-pollination of followers between our different channels won’t always be applicable given the difference in audience and expectations so this remains a challenge.
- We didn’t on this occasion but we could have gone one further and sent this video to only our new followers and then measured how many actually opened the snap for an open rate engagement analytic.
The findings are still not at the granular detail we’ve come to expect from familiar channels such as Facebook and Twitter. We’re also not able to integrate Snapchat or other “Dark Social” messaging apps into our data analysis tool Ripjar but it’s a starting point to creating benchmarks that evaluate our channel and the outreach campaigns success. Despite the manual process required to log these details they will become vital for us to demonstrate and improve both channel ROI and hopefully the broader policy aim of a more diverse Foreign Office workforce.