CrowdSourcing: Failures and Successes

Geoff Mattie of Project Management Institute published a post asking, “Does Crowdsourcing Work in a Project Environment?” that raises some very valid concerns that indicate to me that he is far sharper than the average bear.

This infographic from 99designs is a good introduction to what crowd-sourcing is (click the image to see the entire infographic):

The Power of Crowdsourcing by 99designs

Click the image to see full infographic

There are different ways to use crowdsourcing. The example used in the infographic of having random people brainstorm solutions could work well, but it has also created crowdsourcing disasters. Other uses though are more problematic especially regarding quality – or lack of it.

The best known and most widely used application of crowdsourcing I know of is Akismet, a spam blocking plugin that is a default in WordPress blogs. What Akismet does – blacklisting the most active and intelligent bloggers’ comments – is an excellent example of crowdsourcing gone wrong.

There are huge issues involved in using crowds as your source for answers. The spam or not spam poll I did illustrates the largest problem with crowd-sourcing. In that poll and the comments it surfaced, it becomes obvious that many people believe they know what comment spam in blogs is but their definitions are all over the map and very surprising.

Some went so far as to say they flag every comment they don’t like or every comment from someone they don’t already know as spam. The beliefs I mention above and that are shown in my poll are what causes Akismet to either send to a spam area – or depending on how the blogger has Akismet configuration set – instantly delete comments from the best regular readers and commentators who frequent blogs I do.

Crowd Sourcing Drawbacks

StyleHop on CrowdSourcing

Most bloggers do not realize that when they flag a comment as spam they are effectively blacklisting and banning that person and preventing their comments from appearing in ALL other WordPress blogs.

If it only kept your comments from appearing in their own blog that would be fine – but that isn’t how Akismet works.

The Akismet problem is so serious – taking away the voices of the most
intelligent and influential – that we developed an Akismet Alternative.

Most people see only what THEY know, believe there is only ONE correct way to do anything, and are sure they are right and everyone else is wrong. They don’t realize there are usually many ways and many opinions based on different backgrounds.

Only those who source their knowledge from large groups of disparate opinions can obtain the big picture. If I were to use crowd-sourcing I would carefully choose my “crowd” depending on what I wanted to know.

If I wanted to develop best practices I would select a “crowd” of the most intelligent, brilliant people I know in that niche area of expertise.

Crowdsourcing

Click Image for Crowdsourcing Successes and Disasters

If I wanted to know what some specific target audience wants to buy I would carefully select only those in that target demographic because anyone else’s opinions would not be valid for that study.

People like me know we are not “normal” and one of the largest hurdles for many business owners to get over is that their target audience is NOT like them. They do NOT know what you know. They are NOT familiar with your industry jargon. They are NOT as passionate about whatever it is you love and do NOTwant to spend their spare time learning all about it. They just want to buy what they want to buy and usually that is whatever the people they follow use or recommend.

The target demographic of most businesses is made up of followers – not leaders. Business owners and managers are usually leaders. These are very different kinds of people just as technical people are usually very poor sales people and those who excel at sales usually do not have strong technical skills. (Yes there are SOME exceptions – brilliant people are often good at a vast range of things – but I have yet to meet anyone who excells at everything.)

Akismet proves to me that crowd-sourcing for blocking spam or blacklisting spammers does NOT work and would only work if they hand-selected the bloggers with wisdom enough to know that only those who are obvious spammers should be banned from commenting – not just those whose opinions you do not like.

Bill Casselman raises many more concerns about CrowdSourcing. His definition?:

Crowdsourcing is getting low-pay or no-pay outside amateurs to do company work.

He continues:

A Business Concept as Old as Suckers

The word crowdsourcing may be new but the concept is as old as P. T. Barnum. You remember P.T? He’s the American sharper who invented the freak show, the department store and the three-ring circus. Barnum’s most famous utterance? “There’s a sucker born every minute, and two to take him.” To which that spry iconoclast H.L. Mencken (1880-1956) added: “No one in this world has ever lost money by underestimating the intelligence of the great masses of the plain people. Nor has anyone ever lost public office thereby.”

He has other objections to uses of crowdsourcing I had not thought of so you may want to head over and read the rest of his diatribe against other types of crowdsourcing.

There is another crowdsourcing disaster on the horizon that could seriously affect Internet users who use Google search. Google recently announced a Chrome Personal Blocklist browser extension add-on that COULD be used to blacklist sites exactly the same way Akismet blacklists bloggers:

“While Personal Blocklist is designed to allow individuals to build up their own unique blocklists, Google says it may use the data to influence the search results for others. From its announcement: “The extension also sends blocked site information to Google, and we will study the resulting feedback and explore using it as a potential ranking signal for our search results.””

While that quote says “potential ranking signal” there is nothing to keep Google from using it to censor Google results. If you are one of the naive who trusts Google implicitly I implore you to read these posts:

The following two tabs change content below.

Gail Gardner

Small Business Marketing Strategist at GrowMap
Creator and owner of GrowMap.com, Gail is primarily known for mentoring small businesses and encouraging bloggers to join collaborations to share skills and support small business.

Comments

  1. Matt Clarke says:

    As an owner of 4 sites I think that the Google blocklist mention above is very worrying. My sites feature only the best quality content but unfortunately not everybody understands what the true definition of good content is. For example, I know a number of people who think that standard display adverts like Adsense on websites are intrusive and spammy. All of my sites are adsense funded and I would hate to think that someone could add my site to their blocklist because it has adverts on it….and that in turn could damage my rating with google. There needs to be some way of defining the reason for blocking if google are going to start incorporating this info into the rankings.
    Matt Clarke would love you to read ..Funny Football QuotesMy Profile

    • Hi Matt,

      The problem we have is that G just doesn’t care about the individual site owner and you’re correct that there can be no consensus on something as subjective as what is spammy and what isn’t.

      They are selling that block feature as a way to refine YOUR OWN content and then using it for something else entirely. There are many reasons a person might not be interested in a site – but that doesn’t mean that site won’t interest OTHER people.

      That you aren’t interested TODAY doesn’t even mean you might not be interested tomorrow. The average person isn’t going to be thinking about their own tomorrows much less what anyone else in interested in.

      I check sites before I approve comments and at least the two pages linked in this comment display ads in a reasonable quantity (3) in optimum positions.

      As long as a blogger doesn’t make the mistakes I mention in How NOT to Monetize Your Blog they should not be penalized – but I’m betting many will.

      We MUST take back the power we have handed that one company to decide who succeeds and who fails. The Google Monopoly is dangerous to our freedom of choice and Google’s Evil makes it obvious they can’t be trusted.

      It is up to all of us to find alternatives and explain why we MUST use them to everyone we know and in our blogs. The are taking away our freedom of choice as I show in the screen captures on this page about the Google’s Panda update.

      They are marching us toward one so-called “choice” and I for one am totally opposed to giving up ANY of my freedoms. I do not answer to THEM – I answer to a higher power.
      growmap would love you to read ..Traditional Versus Collaborative World ViewsMy Profile

  2. Architectural Visualization Course says:

    Like every coin has two sides. Same thing goes with this crowd sourcing also. However, once can avoid it’s negative sides by following this article as it let you know it’s pros & cons.

  3. Here’s my experience with crowdsourcing as recently as Jan 2012:

    Things I’ve learned about Crowd-sourcing Media content:

    • More than 75% of respondents were from Eurasia.
    • Respondents did not understand north american idiosyncrasies
    • Language barrier was evident
    • Visual change instructions are incredibly difficult to verbalize in an email.
    • Recommendation: Its worth paying a single person who’s work speaks for themselves.
      Bottom Line: Its just like code outsourcing, you get less than what you pay for.

      The time I had to wade through the terrible (I mean MSPaint bad) designs that were submitted, I would GLADLY pay someone who has a reputation and who, immediately understands western culture.

      Crowd-sourcing tournaments are only good if you have no clue what you want, and are in dire need of direction. Then, hire a professional.

  4. We see the trend where ‘crowd’ generated solution that is not only pertinent but lot cheaper than the expert generated solution.
    Lisa would love you to read ..ParfumMy Profile

  5. It’s relative, you know. Of course, it fails many times, but it also has pros. A start-up with a small budget can get to choose from a bundle of logos/designs made solely for them!
    And if they dont like any – there are no bounds. One can end up finding some really good designs there, without spending a fortune.

  6. iPhone App Development says:

    I think before broadcasting your problem online better is to try to solve it by yourself or take help from your team expert. If not then use the technology and make it viewable to others. I know if doing so, there are few problems of spamming but if you have able management like suggested one here then i think no to worry about. Sometime broadcasting your problem online can get a better solution than not found in yours.

  7. I didn’t even know that GASP was made to solve this problem. I’ve been using akismet and GASP together. But I better disable Akismet and see what happens.
    I think it was a great idea to make the program/plugin work like that. It just went wrong. How else are you going to make a good blacklist?
    There are too many human spammers that need to get blacklisted. You can’t let a team of people deal with that.
    Danny would love you to read ..Black & Decker NPP2018 18-Volt Cordless Electric Pole Chain SawMy Profile

  8. Thanks to genius brains like Jeffe Howe who introduced the concept of crowdsourcing in an interesting way. Dealing with Akismet can also become a headache sometimes but it is applications like Akismet which makes wordpress blog more challenging to use. However, blocking even the good guys can become its own downfall. Blocking spammers is one thing but blocking even the best commenters doesn’t sound too friendly.

  9. thanks for this post..i now know that flagging can have consequences…i am now being careful when i do this from now on

  10. im so disappointed with Akismet..they flag you even if your are not a spammer..it has become a nuisance for us
    Mexican Drinks would love you to read ..A Simple Quesadilla RecipeMy Profile

  11. thank you for telling me about crowdsourcing..i never knew this term existed.thanks for educating me
    Buffalo Tees would love you to read ..Buffalo Cool Place T-ShirtMy Profile

  12. People should start using GASP now.. There are certainly times when program Fails.. I don’t really recommend anyone to use Akismet since I’ve already heard a lot of complaints about their service and blocking comments problems.

  13. That is the beauty of WP. Anybody can take one of their templates and modify it to meet their personal or business needshttp://colorfashion.co.cc/. The power of open source collaboration.
    Fashion would love you to read ..Collette Dinnigan Fall 2011 RTW ShowsMy Profile

  14. Hello Gail,

    Here’s another thought for you and a different slant on “crowd sourcing”. To be technical, crowd sourcing is also all those nice neat freelancer websites all over the internet. For those that wish to hire intelligence, or at least that being the goal, they can goto any of these websites, pay a fee, and they get the expertise they need for their efforts.

    In a Utopian world, this would be great, but over time, even this is polluted to a degree. Please understand that I have nothing against foreign individuals, there are a great number of them with an exceptional command of the English language, and are very much deserving of the fruit of their efforts, which I’d expect is well underpaid given their skill. However, there is a far greater number that lack that command. That being said, please allow me to continue.

    These markets are increasingly crowded with individuals more than willing to work for pennies an hour, and although there are some, given their lack of command with the language, there are others working for sweat shop wages, with better skills, that are more deserving just like all high skilled people. The problem with the market is that these “crowd source” areas are so flooded that the services are now commodities. For anyone, say in the US, to attempt bidding the projects, they would assuredly make less than minimum wage at a local burger joint.

    This is another area where I see “crowd sourcing” failing. In people’s zeal to make internet riches, there are far too many foreign individuals, and others, willing to work for less than minimum wage. They do so because, unlike me and many others, they can afford to do so and still survive, while the rest of us would surely starve.
    Jim Sanders would love you to read ..trogar1971 wrote a new blog post- The Base Internet Marketing Strategies Cornerstone – Knowing Our Target MarketMy Profile

  15. I have heard of this crowdsourcing but didn’t know that much about it. Thank you for sharing this valuable information.

  16. That infograph illustrated the power of crowd sourcing very well. Cheers

  17. I find this new knowledge about Akismet worrying. Certainly, what looks like spam to one is not necessarily spam to another. I have worked with someone who considers every comment with a link attached to be spam unless that comment was very, very intelligently written … but when he is unable to check his spam box for a few days and the “spam” messages accumulate into the hundreds, he simply marks them all as spam, and heaven save those that had real opinions in them but got buried in all the trash :-(

  18. All thumbs up on this analysis… while the concept of crowd sourcing is very much appealing for most people, it’s very important that they do it with best practice methods.
    Andrew would love you to read ..My Shameless Happy Birthday VideoMy Profile

  19. This is a really worrying phenomenon. We have noticed that legitimate comments do end up in Akismet spam catcher and always have to ‘rescue’ them. Didn’t realise that they were pulled from all users of Akismet!

  20. I completely agree with your analysis. Crowd sourcing is really an attractive old thing and the expressions are quite new. I believe crowd sourcing is successful although I think crowd sourcing websites are not similar from content websites.

  21. “People like me know we are not “normal” and one of the largest hurdles for many business owners to get over is that their target audience is NOT like them.”
    -I think your sentence above pretty much sums it up. The issue with crowdsourcing brings me back to studying statistics in college. To get a random sampling of answers from the general population, crowdsourcing might be effective. However, for consulting purposes on a technical matter, I would definitely shy away from depending on this.

  22. I have also had problems with Akismet. It blocks comments that are not spam.

    It makes me wonder what words or how it’s algorithm works to mark completely non-spam comments as spam?
    Gustavo would love you to read ..5 Mistakes Commonly Made By Linux NewbiesMy Profile

  23. I had the same problem with Akismet. A friend and I often exchange comments. Recently we discovered that our comments were being blocked. I rarely comment on other blogs so we don’t even know why Akismet chose to spam us. I started looking for an alternative and found the gasp plugin,Thanks

    • Welcome to GrowMap, Nancy. I’m glad you found our solution. Not only does it only take a few “spam” reports by other bloggers to get you banned – someone hateful can actually write comments using your information which is usually easily available and ban you on purpose.
      growmap would love you to read ..GrowMap Anti-SpamBot Plugin TestimonialsMy Profile

  24. I had the same problem with Akismet. A friend and I often exchange comments. Recently we discovered that our comments were being blocked. I rarely comment on other blogs so we don’t even know why Akismet chose to spam us. I started looking for an alternative and found the gasp plugin. I recommended it to my friend but she seems to think Akismet will be okay even though it dumped her comments on my blog into spam too. Ah well. I guess you can’t save everybody. :o
    terescia would love you to read ..NOOK Color eBook ReaderMy Profile

    • Hi Terescia,

      I applaud you for your wisdom both in finding and implementing a solution and in not beating yourself up for not being able to get your friend to understand the importance. As you say, we can NOT save everyone.

      Some bloggers are called to share their wisdom and being able to comment on other blogs and provide high quality advice are essential to growing their influence which can be used to create a better world. I encourage any blogger interested in doing that to contact me.
      growmap would love you to read ..How to Win Friends and Influence Bloggers- OthersMy Profile

  25. Polo Shoe Fanatic says:

    I fully agree with this analysis. Crowdsourcing is actually a pretty old thing. The terminology is new. When I first saw the word I was a bit puzzled until I saw the picture explanation. The spam blocking program is the absolute best for this issue.
    Polo Shoe Fanatic would love you to read ..Polo Ranger BootsMy Profile

  26. App Developers says:

    Well I can see how much this helped me as I am one of the black listed! Why should anyone that owns a blog be able to decide what spam is or is not and then have that person black listed with Akismet? It is disturbing that a system as large as Akismet would rely on only that criteria. There must be a better way.
    Another great article. thanks!

    • Hello App,

      IMHO, that is how Akismet wants it because their system is going to progressively censor those who are most active who are also most influential. There is a huge battle going on for control of the Internet and it is up to us to ensure we find a way to retain our voice one way or another.

      There are many ways they could improve Akismet but they are not interested. They could require more than X bloggers to flag someone as a spammer, or they use a group of bloggers willing to be trained on some specific definition of spam or they could weight the votes of the most experienced, impartial bloggers heavier than hobby bloggers.

      So just as you would develop an App, we bloggers have collectively developed a better way to deal with Spam. Andy @CommentLuv created the GrowMap anti-spambot plugin and we provide instructions on best practices for blocking spam that eliminates the need for Akismet (although a few bloggers still use it too).

      growmap would love you to read ..Best of GrowMap – Our Most Important Posts All in One PlaceMy Profile

  27. cynthia says:

    I look from the angle crowdsourcing is successful but I think crowdsourcing websites are distinct from content websites.
    cynthia would love you to read ..Aerosport Ipod Nano 4g BlkMy Profile

  28. I’ve heard of crowdsourcing a few times in the recent past but I am not sure like what it is all about?

  29. reehasmith says:

    i cannot use Akismet plugin,but after reading this post i will surely try this …thanks for this informative post…keep on writing

  30. Proxyland says:

    Actually, I’m confused about Google, about Google Fairy. Many blog/sites with low quality is unblocked by Google, but many great and unique blog/sites with old domain and high quality is blocked by Google. I’am so curious How to Google makes the decision. what matters is the underlying to decide that the blog needs to be blocked or not?
    Moreover, with the new algorithm, Google makes traffic of many great and unique blog/sites is down.
    Proxyland would love you to read ..Al-Maisan Android 22 Froyo Arabic ROMMy Profile

  31. I’ve never heard of crowdsourcing before reading your post. It’s quite interesting. What is the alternative for Akisnet plugin?
    Alexis would love you to read ..Google’s Recent Algorithm changes- Are You a Winner or a LoserMy Profile

  32. I’ve heard of crowdsourcing a few times in the recent past but I wasn’t real sure what it was all about and haven’t really taken the time to research it until I read this post. What you said about the Askimet plugin is actually very disturbing. I use this plugin on all of my wordpress sites and had no idea that it could blacklist and block the best commenters out there. Is there an alternate “hands off” solution to blocking spammers but still letting the good guys in or would you have to go through each comment manually and mark as spam or approve? Also, I hope that Google doesn’t mess it up for the good guys by what they’re planning to do.
    Ken would love you to read ..Final Fantasy 14 Guide- UpdatedMy Profile

Speak Your Mind

*

CommentLuv badge