For months now I’ve been wanting to do an experiment on Facebook but haven’t had the time. The Daily Beast finally beat me to it with their Cracking Facebook’s Code article yesterday, which shows essentially:

Measuring social media is hard. It’s easy to be overwhelmed by minutia, there’s yet to be a measurement site that even comes close to adequately tracking people & content across social media platforms, and there are no standardized benchmarks to know [...]]]>

Measuring social media is hard. It’s easy to be overwhelmed by minutia, there’s yet to be a measurement site that even comes close to adequately tracking people & content across social media platforms, and there are no standardized benchmarks to know if 43 (fans, comments, @mentions, whatever) is good, bad, or average. And the truth is, social media is more art than science. Not to mention we’ve become obsessive about measuring this just because we can (or at least, we think we can). Besides, when’s the last time you checked the ROI of that poster you put up, the press release you sent out, or the postcard you mailed?

Let’s talk about a new way.

Putting people front and center in your measurement plan.

What if every day, or week, or whenever, there was a list of people delivered to your inbox. These are the people who commented on your blog, or your facebook comment, or began following you on twitter, or favorited your slideshare presentation. The list would include the actual text of their comment. In v2.0, you could give values to each of these activities. As in, 5 points for an RSS subscriber, 1 point for a facebook like, 2 points for a delicious tag, etc. And the measurement tool would know the network size of each of these people. Total score is a function of the weight of the user’s action, and the size of the user’s influence base. The list delivered to your email inbox would be ordered by descending total score. In v3.0 this is not only an email, but an entire database. Search by comment keyword, by “most influential/engaged facebook fan” or how a user’s score has changed over time. This is as much a customer relationship management tool as a social media measurement tool.

But wait, that’s not all.

How is your content doing online?

You want to care about this for a few reasons:

1. If a piece of content is suddenly doing well, maybe there’s something you could do (like responding to a tweet, taking a photo, adding tags to a YouTube video) to increase the momentum.
2. If you can discern a pattern to the characteristics of the content that is doing well (or for that matter, is doing poorly), maybe you could build on the thing that’s working, or fix the thing that seems to be broken.

This list of content items would be emailed to you. You’re probably going to want to segregate new content items (recently posted) from recurring content items that continue to get pageviews or comments long after you’ve first posted them. In v2.0, you’d be able to tie the size of your network to each content item, so a blog post that you also tweet about to your 10,000 followers would have a higher score than the comment on your flickr photo that gets 4 views a day. In v3.0, if a recurring content item jumps some x% over last week, month, etc, then an emergency email would be sent to you, with the assumption that there’s probably something urgent that needs your attention. Clearly, these emailed lists would also benefit from a database that was searchable, and showed trends over time.

But wait, there’s more.

How did people arrive at your content?

The two big categories here are links and search terms. Within links, you’ve got to consider referrals, direct links, and embedded content you’ve got elsewhere. Within search terms, you’re going to want to know the search terms people used to find your content, the search terms they’re asking for on your site, and the hashtag & list terms they’re associating with you on twitter.

So this is your 3rd email of the day, week, month, etc. At this point, you’re probably asking yourself not how people arrived at your content, but why you should care. For links: having the opportunity to thank, and more generally connect with, the folks that are sending traffic your way. For search terms: to both guide future content creation decisions, and refine any SEO/SEM that you might be doing. And again, with the descending scores, and the searchable database. In v2.0, it would be nice to group similar links, and similar search terms together. In other words, I know that search phrases including the word foursquare drive quite a bit of traffic to my site, as do many different links from Beth’s blog and Ian’s blog. There are some nifty tricks in Google Analytics to combine each of these sets of links & terms, but I want a simpler solution.

Finally we arrive at the place where most folks seem to start.

What’s the size of your  network?

You’ve got your people, your engagement, and your traffic. These are big aggregate numbers that are best tracked over time, and hopefully compared to your competitors.

It’s probably not the case that a subscriber to your slideshare account is ‘worth’ the same as a fan of your delicious tags. So again, it would be nice to be able to give a weight to each of these types of people, engagements, and traffic. And you’re probably going to want to see that “people” score as a total of all platforms, or by several platforms at a time. Like in Google Analytics, it would be nice to attach annotations to these aggregate trends.

It’s impossible to do all of this measurement though right? A total pipe dream. An analytics package that would be impossible to implement? A fools errand to try to connect so many platforms, so many third party tools, with so many divergent goals?

Probably.

But here’s the tools you would need to do it.

True, some of these cost money, and less than half the platforms have open APIs, but let’s at least take a quick gander at each of them. These are, of course, in addition to the many one-off possibilities I’ve mentioned previously.

WordPress Stats Plugin

*You’re going to need to install the plugin to get these stats.

Google Analytics

*GA has a TON of data to dig into. It’s easy to fall down the rabbit hole. I’ve been promising a more in depth post on this for months. It’s still in the works.

Feedburner

*of particular note: check out the publicize tab to know who’s subscribed to your blog via email.

Facebook Page Insights

*Only page admins will get access to this data.

BackType

*of particular note: this is the only way I’ve been able to figure out the amount of Facebook buzz my content gets from folks who aren’t my friends/fans.

Twitter RSS Search Query

This one’s a bit of a hack. You can “subscribe” to an RSS feed for any search term on Twitter (like @yourname). This is a great record of all of the people who mention/retweet you. Google reader will then graph for you the number of “posts” by day of week, time of day, etc.

Bit.ly

*If you haven’t already, please give TweetDeck (or HootSuite, or CoTweet) your bit.ly API, and it will shorten and track all of your links automatically. For added bonus, connect bit.ly > TweetDeck > Packratius > Delicious. More on this soon. It’s magical. You can also stalk other people’s bit.ly link stats by adding a “+” sign to the end of the link.

TweetMeme

*Especially useful to track twitter links that might not mention your name, or anything specific to your brand.

YouTube Insights

*These stats get pretty confusing. Referrals go missing, the numbers don’t add up, totally unpredictable when or how videos will go viral. But work with what you’ve got.

Flickr Stats

*Sadly, only available to (paid) pro account users.

LinkedIn

*Just learned this one last week. If you upgrade to a (paid) pro account, you can see the actual names of the folks viewing your profile.

SlideShare

*Pro (paid) accounts get additional analytics. But I couldn’t find any examples online, so it’s unclear how much more detailed the stats get.

Foursquare Venue Analytics

*You’ll have to claim your venue to get access to this data. Do it now.

Alright, quick review

Measurement is hard. Don’t get distracted by the numbers, and lose sight of the people, content, links, and search terms; in other words, text > numbers.

1. WHO: Focus on active relationships: people engaging direclty with your brand by commenting, liking, joining, etc.
2. WHAT: Keep track of content: especially new items, and recurring items that spike in traffic; search for patterns
3. WHY: Pay attention to passive relationships: people engaging more indirectly with your brand via linking to your content, or searching for your keyword terms.
4. HOW MUCH: Spend the least amount of time on the size of your network: but don’t ignore it completely. Compare your stats over time, and to your competitors.

There’s a plethora of tools. One day someone will integrate them all. Until then..if you’re interested in getting a copy of this prototype document, just send me an email (Google Docs didn’t handle the images & formatting very well, or I would just post online).

Possibly related posts:

1. Theatre & Social Media: TCG 2010
2. Theatre & Social Media in 2009

The Theatre Communications Group 2010 National Conference wrapped up last night. I heard word it was the largest in the conference’s history, clocking in at just under 1,000 attendees. A sizable chunk of those folks live tweeted several breakout sessions, mostly [...]]]>

The Theatre Communications Group 2010 National Conference wrapped up last night. I heard word it was the largest in the conference’s history, clocking in at just under 1,000 attendees. A sizable chunk of those folks live tweeted several breakout sessions, mostly under the #TCG2010 moniker.  My presentation, Social Media Strategy: Why ROI Isn’t Enough went quite well. I’ve received so much great input from folks over the past four months about what I should talk about, and was overwhelmed with gratitude from the many good luck emails in the days leading up to my presentation. So without further ado, for those of you who couldn’t be with us in person, and may not have caught the original tweeted link, here goes nothing:

You can also check out the live tweeting commentary. A few folks requested my notes, since I rattled off a bunch more stats.

Shortly after my presentation, my interview podcast with Technology in the Arts also went live! It in, I discuss the most significant findings of my social media research for theatres.

Speaking of, now I’m off to gather my thoughts on speaking at Americans for the Arts next weekend in Baltimore about Branding Yourself Through Social Media with the estimable Ian David Moss.

Possibly related posts:

1. Theatre & Social Media in 2009
2. A Social Media Measurement Plan
3. Social Media Strategy Recommendations for 2010
4. What do YOU want to know about Theatre and Social Media?

Ok, not Facebook per se, but a rant about theatres using social media inauthentically. I get it, you’re busy, there are a lot of social media platforms, and you’re not exactly sure which one(s) are worth your time. So, [...]]]>

Ok, not Facebook per se, but a rant about theatres using social media inauthentically. I get it, you’re busy, there are a lot of social media platforms, and you’re not exactly sure which one(s) are worth your time. So, you decide to link your Facebook account to your Twitter feed and not bother checking in on your Twitter community. A full 25% of TCG theatres on Twitter have committed this tragedy (see autosync below).

I’ll be going more in depth about each of these segments at the TCG conference next week, but for now, let’s focus in on this Facebook issue (MailChimp, TwitterTools, and TwitterFeed are also culprits, but far less popular than the Facebook method). If you’re a theatre on Twitter, you probably care a lot about the volume of buzz: the number of tweets mentioning your brand. I call this @mentions. The graph below shows the difference in this metric between “Real” tweeters (any theatres that posted specifically to Twitter, via any method they want, anytime in the last month) and “AutoSync” tweeters (those theatres who blindly and shamelessly fill their tweet stream with solely Facebook posts.

On average, if you’re tweeting daily for real, you’re seeing 5x the volume of buzz each week about your brand. Now of course, there are also those special theatres that get nearly 100 @mentions each week. I’ll be talking about them too next week. Next, check out the weekly blue line compared to the daily red line (still on the above graph). Even if you have time to update Twitter just once a week, you’ll get higher buzz on average than simply autosyncing your Facebook profile.

And what about the fan base?

Say you can only update Twitter once a month (!); you’re still likely to have a higher fan base than those theatres who “tweet” daily Facebook updates. And just who are these culprits? n00b’s who don’t know any better?

It turns out, sort of. It looks like once theatres hit their one year anniversary on Twitter, they tend to start really paying attention to How to Tweet. And the longer they’ve been around, they more frequently they’re tweeting. That’s real good news.

So maybe I’m being a little too harsh? Let’s see what the fans have to say. But first, let me say, the theatres I’m calling out here are by no means the only bad guys–I just happened to notice these tweets while finishing up my research. I also have no way of knowing if the theatres may have reached out to these tweeps in some non-public way (via DMs or on another platform). But let’s get real here folks.

So you offer a give away, an eager fan reaches out, but nobody ever responds. A disenchanted fan? Oops.

You’re promoting a show, an eager fan wants to ask a question about said show, but nobody ever responds. A potential lost sale? Oops.

Don’t forget it’s not only your audience members who are tweeting about you. Also your sponsors. Those folks that give you free stuff. Bet they would have appreciated a thank you & a you’re welcome. Oops.

Here’s another fan, a subscriber it seems. For now at least. No response to her. I wonder if she would have invited friends along the next time she attended a show. Oops.

You’re always wondering if social media sells tickets. Here’s a potential sale. But no response. Oops.

What a great compliment from an awesome colleague. Too bad there was no response. Oops.

A student, excited about a play, willing & able to recommend it to all her friends. What a missed opportunity for audience engagement. Oops.

Other times, it’s potential artistic collaborators you’re missing out on. What a bummer. She could have been a great playwright to bring on board. Or at least, chat with for a moment online. Oops.

The sad thing is, most of these theatres will never know I was calling them out. I’m not their Facebook fan, I haven’t liked their page, I won’t be commenting on their blog, and I won’t be bothered to call their box office. But I will be talking about them on Twitter. Do you want to guide the conversation about your brand online, or do you want to leave it in my very capable hands?

Possibly related posts:

1. The Epic Facebook Experiment

Nonprofiteers have been collectively organizing online for years–from email listserves to teleconferences to online message boards. But Twitter, and specifically hashtags, has created the ideal space to carry on industry-wide conversations across time, place, and topic. This is interesting [...]]]>

Nonprofiteers have been collectively organizing online for years–from email listserves to teleconferences to online message boards. But Twitter, and specifically hashtags, has created the ideal space to carry on industry-wide conversations across time, place, and topic. This is interesting to me. So as the final paper for my Strategic Management of Nonprofits class, I decided to tackle an analysis of “the breadth and depth of conversations in two nonprofit sectors currently using Twitter hashtags to generate and moderate new idea creation and dispersion.” Quite a mouthful, huh? Sometimes I’m a little wordy.

I want to be upfront here: this post isn’t going to help you sell more tickets, or raise more money, or finally figure out Foursquare. This post is about our community, how we can make our community of nonprofit professionals even better at doing what we do best–inspire each other to change the world. The original paper was a little formal, so I’m going to try to cut and paste and reshape a bit in this post.  Bear with me here for an extra long post…

What are hashtags?

In the past three years, the use of social media by nonprofits (and the general population) has grown exponentially. There are now 500 million people using Facebook, 200 million searching Twitter, and 150 million watching YouTube . Six years ago, none of these sites existed. As an asymmetric public network (user A can “follow,” or read, user B’s “tweets,” or posts, without user B’s permission and without user B having to read user A’s posts), Twitter best provides the opportunity for large, decentralized, archivable, searchable conversations.

In Twitter’s early days, features were limited so users created their own taxonomy. In August of 2007 Chris Messina first proposed using the pound symbol “#” to help group similar conversations happening on Twitter. This combined the searchable tags of metadata on sites like Flickr with the topically oriented channels of IRC. In their 2010 usage, hashtags have come to serve at least five different functions:

1. To keep track of an ongoing conversation (#nonprofit)
2. To broadcast the happenings of a one time event, conference, or emergency (#haiti)
3. To get into Twitter’s list of trending topics (#justinbieber)
4. To comment on the intent of the post (#ironic)
5. To provide additional metadata about the tweet, such as location or speaker (#NYC)

Searching for any of the “#words” on Twitter’s native search client returns all posts marked with such a tag over the past week. Searching for the same on Google’s “Search Updates” returns selected posts from approximately the past month (and will eventually include all posts since Twitter’s inception). Specialized archiving programs like TwapperKeeper, Searchtastic, and The Archivist store all posts marked with a #word, and allow for an exportable excel document. Hashtags themselves can be searched at hashtags.org, the semi-official repository of hashtags in use. However, as a user created phenomenon, new uses and groupings of hashtags are constantly being created.

How are nonprofits using hashtags?

Nonprofits have run with the hashtag concept and there are now hundreds of public conversations on Twitter covering topics as diverse as #philanthropy, #climate, and #pubmedia.  I am particularly interested in the potential for information and ideas to cross industry, functional, and demographic boundaries in these hashtag conversations.  Two such tags (#edchat and #2amt) appear to be quite different in the:

• History of the hashtag’s origination
• Degree of current moderation implemented
• Frequency, volume, and diversity of posts
• Number and diversity of users

So let’s dive a little deeper into what makes these conversations (and these people) tick.

#edchat

In July of 2009, the #edchat conversation was created by Shelly Terrell and Tom Whitby, “for all educators to post their thought-provoking conversations throughout the day.”  By October, 2009, #edchat had morphed into a guided discussion group occurring on Tuesday at 12pm EST and 7pm EST, each addressing a particular topic of interest to teachers and education administrators. Said one user,

“I find the fast pace discussion exciting. The varied opinions and points of view are enlightening, and the discussion is always deep and meaningful. Some might think of it as organized chaos, but it is just like sitting in a room of brilliant educators and never having to move around the room to hear everyone’s conversations. What’s even better, you can jump in at any time when you see a topic, statement or idea that grabs you or that you feel strongly about without feeling like you’re interrupting a conversation.

The #edchat discussions are also a great place to find like-minded educators to add to your PLN (Personal or Professional Learning Network). Often, after a session, you will find your email inbox full of people from the chat who found you on Twitter, and you can search them out as well to follow.”

Moderation

These weekly online conversations are archived at a public wiki,  which is visited by users from dozens of different countries.

Topics are crowdsourced:

“Anyone is able to suggest topics for our Twtpoll. We [@TomWhitby @Web20classroom and @ShellTerrell] agree on the 5 questions that will go in the poll. Usually, we use those questions that have been suggested on the #Edchat group on the Edu PLN Ning, which is free for anyone to join. The over 600 participants that are familiar with the process retweet the poll which @Web20Classroom creates every Sunday. Anyone can vote on the topic, even those who have never participated on an #Edchat.”

Posts

In the first week of May 2010, well over 4,000 #edchat conversations were posted by more than 1,000 users (raw data file).

Not surprisingly, the volume of conversations peaks on Tuesdays when the live chat is occurring, and tends to linger into the following day.  Posts per user average 1.5 on days other than Tuesday, which jumps to 5, and on Wednesday which increases to 2.1. With the average Twitter user tweeting less than once per day , this data shows that #edchat contributors are highly engaged in general, and in this conversation in particular.

Now comes the hard part. How to analyze 4,000 tweets? I created a relatively crude algorithm that defined any post containing

• “@” as a reply (i.e. by convention, posts beginning with @username are in reply to said user; these posts were coded manually, before applying the algorithm)
• “RT” as a Retweet (i.e. reposting another user’s content so that one’s own followers can see the content)
• “http” as a Shared Link (i.e. providing information)
• “?” as a question (i.e. seeking information)
• all other posts are coded as a comment

I found that more than 1/3 of tweets are duplicated content. This spreads original ideas beyond the #edchat followers to the hundreds of thousands of people following the contributors, but not the hashtag. The question:answer ratio is nearly 1:4, with 6% of posts containing a question, and 23% containing an answer. This means that educators are able to get their questions answered by a larger and likely more diverse pool of respondents than if they were to rely on those within their school or personal networks. Approximately 1/5 of tweets are posted expressly to share a link to a website or article. While the majority of links were to education specific content, I was also able to find links to a diverse array of mainstream media.

This breakdown of post content is fairly consistent on all days other than Tuesdays, where there are many more comments and replies, as one might expect in a live chat.

On any given day, about 15%-40% of the community is engaging with the #edchat hashtag. Of course, this only covers one week’s worth of data, so it’s likely this community is actually much larger.

Using the ever-entertaining/beautiful Wordle (where larger words in the picture occur more frequently in the data set), a review of the frequency of word usage reveals that the majority of the conversation revolves around education topics. But the value in the content analysis of tweets is likely in the long tail of words not associated with education.

Users

I selected the 67 most engaged users  by limiting analysis to those who had posted 14 or more times in the past 7 days. Why that often? I dunno, it just seemed like a good cut off point. Their frequent usage captured 48% of all tweets in the past week, even though they count for just 6% of users.

In their (160 character max) biographies, users described themselves most often  as teacher, education, technology, school, and learning.

These 67 users are followed by a collective total of more than 103,000 other Twitter users (assuming no network overlap); while they range in popularity from 33-10,316, on average they have more than 1,500 followers. They have tweeted between 165 and 24,179 times, for an average of close to 5,000 tweets. Considering the average Twitter user has just 27 followers and has tweeted less than 300 times , these #edchat contributors are in fact immensely popular and engaged users.

There are only slightly more males than females contributing, and ¼ are from outside the US, while 19 different US states are represented.

Even within the US, users are scattered far and wide. Similar to the content analysis of posts, the “value” is likely in the number of users contributing one or two ideas sporadically, in the long tail of user contributions.

The future of #edchat?

The #edchat hashtag has become a robust online conversation hub with a clearly active and diverse user base, with helpful moderators, discussing topics of interest to educators, education administrators, and education policy makers. However, it is a delicate balance to strike between keeping community members engaged in a topic they’re passionate about, limiting the chaos of a decentralized group, ensuring a diversity of inputs, and promoting innovation. The current wiki system and Personal Learning Group on Ning seems to keep track of archived content well, but doesn’t yet seem (from an outsider’s perspective) to provide a cohesive community of users. As the group quickly approaches their first birthday, I wonder if the most active users would consider creating a blogging collective, to further flesh out their ideas. Perhaps similar to 2amt…

#2amt

Late one night in January 2010, a conversation began to brew on Twitter among theatre artists and administrators about what the future of theatre might be. Nick Keenan Kris Vire quickly gave it a tag (#2amt, standing for “2 AM theatre”), and by the end of the month David Loehr conceived the 2amt blog to keep track of #2amt generated content, active users, and to expand on some of the more worthwhile ideas discussed. There is also an “official” Twitter handle of @2amt (also moderated by dloehr) that helps to organize content on Twitter. In less than four months, the blog has generated 50 posts, and the #2amt tag 5,549 tweets. Said one of its founders,

“The #2amt conversation became a successful methodology almost accidentally for innovative brainstorming because it quickly synthesized a broad range of perspectives on a broad range of topics. It combined the brainpower of theatre producers, funders & patrons, promoters, and critics to solve common problems from all angles at once. It was, and continues to be, an agile way to have a conversation… but the fact is that #2amt is not an accessible conversation for most theater makers to participate in, and conversations that come out of community groups like the Summit and the League of Chicago Theatres are still more potentially actionable than the high-level strategizing and future design brainstorms that #2amt is so good at.”

So here’s where I become a subjective participant of the conversation rather than an objective observer. In full disclosure, I’ve been contributing to the #2amt conversation for the past few months, and have on more than one occasion been promoted by the community. I can’t claim impartiality, but I will still try to be critical…

Moderation

Unlike #edchat, topics discussed on #2amt aren’t moderated and don’t intentionally take place at any particular time or day, despite the “2am” moniker.  Whether this is because the group of contributors is still relatively small, the hashtag’s usage is recent, or contributors lack the coordination or desire is unclear. However, the 2amt blog is strictly moderated. Blog authors are currently limited to #2amt founders, although guest posts have been requested from active #2amt contributors.  The contents of the blogroll are outsourced to the community, and is open for anyone to join. No data was available to compare readership of the 2amt blog versus the edchat wiki.

Posts

The #2amt tweets show a remarkable difference in frequency and content compared to the #edchat tweets (raw data file). In order to analyze a statistically significant number of tweets by #2amt, I had to expand my length of time from one week, to #2amt’s history of about 4 months.The data set also stops May 11 (because, uh, the paper was due May 13 ).

Average tweets per user per month increased steadily for the community’s first three months, and then dropped considerably in April (and the first 1/3 of May). It’s possible that this level of engagement wasn’t sustainable for the community.

March also saw a marked difference in the distribution of content types, where replies to another user accounted for an astonishing 70% of all conversations. The #2amt community seems to be more engaged in back-and-forth conversations than the #edchat community. They have approximately 3 times the number of replies, and only ½ the times of Retweets or shared links. This is likely a function of a smaller, tighter-knit community. While users may be engaging more with each other, it seems likely they are sacrificing external viewpoints from related websites/articles.

While #edchat contributors average 4 tweets per week, #2amt contributors are average about 9 tweets per month. It’s unclear without more analysis if this is a function of how new, small, or decentralized #2amt is.

However, the distribution of new and returning visitors was roughly equal in March and April, limiting the likelihood that users were “turned off” by March’s conversation volume. There’s also an interesting “snowball” effect where the total number of users participating in the #2amt conversation is growing each month, driven by an ever-widening circle of “new users” (defined as someone who didn’t contribute in the previous month(s).

Again with the Wordle. This time, user names dominate the picture (and who else but David Loehr at the center). Also easy to spot Chicago and new play related themes.

Of the top 10 most used (non-common) words in tweets, more than 70% referenced theatre-specific terminology. These informal conversations tied to a narrow topic may serve to preserve strong group affiliation at an early stage in the group’s development.

Users

Contributors to #2amt demonstrate far less diversity than #edchat contributors. I again chose 14 tweets as the cutoff point for “engaged users,” (although the #2amt time span is extended to just over 4 months, compared to the #edchat data at 7 days).

Males account for the majority of engaged users, a plurality are currently residing in Chicago, IL (the hometown of founder Nick Keenan), and only 2 users come from beyond North America. However, as the online network grows, it is already expressing geographic diffusion beyond the Midwest to both US coasts and several major metropolitan cities.

Another wordle showing how #2amt contributors describe themselves, revealing an abundance of directors, actors, and playwrights. Anecdotally, theatre practitioners are relatively evenly split between male and female, while disciplines and functions within theatre tend to follow gender lines (i.e. costume designers and marketing directors tend to be female, critics and production managers tend to be male, etc). This could be the reason for the uneven gender split among #2amt contributors currently.

The top 46 users  account for 87% of tweets, but only 13% of users—a fitting approximation of Pareto’s 80/20 rule . Total followers among these most active contributors is 33,725, average follower count is 733, while they range in popularity from 100-8,000 followers, with a standard deviation of 1,178. They have tweeted an average of 3,573 times.  Compared to #edchat, contributors to #2amt are about half as influential, and a quarter less prolific over their lifetime on Twitter, but much more similar to each other than to the general Twitter population.

The future of #2amt?

The #2amt hashtag has made great strides in creating a community of passionate, engaged, and thoughtful artists and administrators. While the pool of contributors remains small, and is short on the diversity critical to new idea generation, it remains a giant leap forward for the theatre industry. Never before have theatre practitioners had the access to like-minded people from across the country working in so many different theatre disciplines. To strengthen the quality of conversation and promote diffusion throughout the entire industry, and not just small independent theatres, #2amt should consider actively searching for potential new contributors that are distinctly different from the current group, consider writing more blog coverage about the “mainstream” theatre industry, and propagate more ideas from outside the arts community.

The future of Twitter Hashtags?

There are clear limitations to reviewing only 2 hashtags to make conclusions about the entire nonprofit industry’s usage of the Twitter hashtag. The data collection and analysis process was severely limited by data that is publically available (and free) and within my own time constraints (shhh, don’t tell Dean Oster I wrote this entire paper in 48 hours). A more robust conclusion could have been reached with:

• A more pointed discussion of the precursors to industry-wide conversations (like email listserves, conferences, and online discussion forums)
• A larger review sample of hashtags
• More structured data collection and analysis methodology
• A more sophisticated formula for content analysis which breaks out in versus out of industry words
• A follow up with hashtag contributors to see if any of the ideas generated are actually being used in the field, and eventually their success rates
• A survey of hashtag users motivations and demographics
• A regression of predictive factors that lead to a statistically significant combination of optimal post frequency, volume, and content together with contributor volume and diversity, and some way to measure idea implementation. This is probably the biggest missing piece of this analysis currently. In other words, I can tell you what the numbers are, but not yet what they should be.

Methodology

Interested in doing this for a hashtag of your own? Wanna pick apart what I did?

1. I began with a review of the history and current usgae of Twitter hashtags within the nonprofit sector.
2. I research possible hashtags associated with various sectors/topics of interest to nonprofit managers (#nonprofit, #socent, #philanthropy, #nptech, #2amt, #newplay, #ced2, #econdev, #bop, #microfinance, #education, #edchat, #climate, #green, #envirnoment, #water, #health, #pubmedia) and selected two based on their statistically significant volume of archived content, my own familiarity with their topics, and the potentially meaningful differences between them.
3. I reviewed blogs and news articles about the #edchat and #2amt Twitter communities, and the other online content they had given birth to.
4. I used TwapperKeeper.com to export an excel workbook of 4,333 rows & 10 columns of data about #edchat, and 5,550 rows & 10 columns of data about #2amt.
5. I created two new content analysis formulas
   1. For determining new versus returning users within a certain time period: =IF(COUNTIF(A$2:A5548,A5548)>1,1,0)
   2. For categorizing post contents: =IF(ISNUMBER(FIND(“RT”,D19)),”RT”,IF(ISNUMBER(FIND(“http”,D19)),”share”,IF(ISNUMBER(FIND(“?”,D19)),”question”,”comment”)))
6. I collected data and analyzed the Twitter profiles of 114 highly engaged users between the two communities.
7. I used an online tool called Wordle to create word clouds of highly used terms in hashtag posts and in Twitter bios to deconstruct what communities talk about, and how they describe themselves.

Conclusion

All in all, I’ve found that nonprofits are using Twitter hashtags to diversify thought leadership within their sector and to crowdsource new idea generation. They have a long way to go in figuring out how to implement these ideas, ensure the diversity of contributors, and maintain group cohesion. There are several possible models for archiving content, moderating conversations, engaging contributors, and implementing ideas in the field.  It’s unclear without further research if an optimal level of diversity, frequency, or volume exists to foster sector-wide innovation. Is that a PhD paper I hear calling?

I never would have thought of writing this without Dean Sharon Oster’s inspirational speech on cross-sector innovation. I wouldn’t have understood the importance of diversity without Professor Rodrigo Canales’s introduction to Barabara Levitt and James March’s paper, Organizational Learning (pdf), from the 1988 Annual Review of Sociology. And while I didn’t user nearly enough of it, I definitely wouldn’t have the capacity for statistics without Professor Ed Kaplan’s patient demonstrations of policy modelling.  And of course, a million thanks to everybody at #edchat and #2amt for letting me secretly cyber stalk them. Last but not least, want to save a copy of this paper as something you will treasure forever and ever? Click here.

So what do you think? Have another great nonprofit Twitter hashtag I should check out? Know of a better content analysis algorithm I could use? I was only sort of kidding about that PhD paper…eventually I’ll need a more diverse data set and more sophisticated analytical tools. Especially if I can’t find anyone to hire me soon.

A look at where non profit regional theatres stand in the world of social media in 2009. The final research report to accompany my Social Media Strategy Recommendations for Yale Repertory Theatre. The good (hello, Facebook!), the [...]]]>

A look at where non profit regional theatres stand in the world of social media in 2009. The final research report to accompany my Social Media Strategy Recommendations for Yale Repertory Theatre. The good (hello, Facebook!), the bad (why are we still writing terrible blogs?), and a wakeup call to all those who’ve yet to even dip their toes in the social media waters. And next week I start my Social Media Marketing class over at the School of Management! Stay tuned for a glimpse of the latest social media research *outside* the world of theatre.

Possibly related posts:

1. Theatre & Social Media: TCG 2010
2. A Social Media Measurement Plan
3. What do YOU want to know about Theatre and Social Media?

The gauntlet’s been thrown. I thought @americanrep was doing a great job engaging folks online so I wanted to track a little closer what they’re doing, who’s interested, and what the impact might be. Now if only this analysis could be standardized and automated…Link to more

Possibly related posts: @mentions as measure of user engagement

Taking a deeper dive on one test twit to see what kind of online tools exist out there, and what we might learn from the mighty @pcsghost. Link to more

Based on a few requests, I’m starting a non-LORT indexed list as well. It’s possible I’ll collapse the two lists at some point. But for now, if you’ve got theatres I should add, feel free to let me know via @devonvsmith. [...]]]>

Based on a few requests, I’m starting a non-LORT indexed list as well. It’s possible I’ll collapse the two lists at some point. But for now, if you’ve got theatres I should add, feel free to let me know via @devonvsmith. WARNING: I’m a full time student, but I’m doing my best to respond to tweeps in real time regarding the study. So if I’m slow to update, rest assured I’m busy in class learnin’ stuff. Link to more

Possibly related posts:

1. LORT Twitter Index
2. PS: non-LORT folks

So here’s a theory I’m testing out: Having more followers, tweeting more often, and/or being followed by more lists doesn’t necessarily lead to more @mentions (the holy grail, at least for now). I’ve yet to determine what does generate @mentions, but I’m working on it…more soon. Link to more

Possibly related posts: Tracking @mentions