Google's Take on Search Plus Your World

A few weeks ago Googleannouncedthe launch ofSearch Plus Your World, whichdeeply integrates social sites(especially Google+) into the Google search experience to make it more personalized.

While Google claimed that the socialization was rather broad-based, the lack of inclusion of Facebook& Twitter along withthe excessive promotion of Google+raised eyebrows. While the launch was claimed to be social for personalizing results, the Google+ promotions appearedon queries where they were clearly not the most relevant resultevenwhen users are not logged into a Google account.

Google+ Over-promotion

A couple weeks ago when Google announced Google Search Plus Your World competitors collectively complained about Google over-promoting their own affiliated websites.

Twitter was perhaps theloudestcomplainer, highlighting how Google basically eats all the above-the-fold real estate with self promotion onthis @WWE search.

It is no surprise that folks likeBen Edelman,Scott Cleland&Fair Searchchimed in with complaints, as this is just a continuation of Google's path. But the complaints came from a far wider cast of characters on this move:the mainstream press like CNN,free market evangalists like the Economist,Google worshipers indoctrinated in their culture who wrote a book on Google&even ex-Googlersnowcall into question Google's transparently self serving nature:

I think Google as an organization has moved on; they’re focussed now on market position, not making the world better. Which makes me sad.

Google is too powerful, too arrogant, too entrenched to be worth our love. Let them defend themselves, I'd rather devote my emotional energy to the upstarts and startups. They deserve our passion.

The FTC'sGoogle antitrustprobe is toexpandto includea review of Google+ integration in the search results.

Facebook& Twitterlaunched a don't be evilplugin namedFocus On The User, which replaces Google+ promotion with promotion of profiles from Facebook& Twitter.

For the top tier broad social networks framing the idea of integrating promotion of their networks directly in the search results is a natural& desirable conclusion, but is that just a convenient answer to the wrong question?

• WhetherGoogle ranks any particular organic result above the corresponding Bing rankingin Google's now below-the-fold organic results is a bit irrelevant when the above the fold results are almost entirely Google.com. But is the core problem that we are under-representing social media in the search results? According to Compete.com, Facebook& YouTube combine to capture about 16% of all downstream Google clicks. Do we really need to increase that number until the web has a total of 5 websites on it? What benefit do we get out ofa web that is just a couple big walled gardens?
• If Facebook is already getting something like 20% of US pageviews& users are still looking for information elsewhere, doesn't that indicate that they probably desire something else? Absolutely Facebook should rank for Facebook navigational queries, but givenall theirnotesspam, I don't like seeing them in the search results much more than seeing a site like eHow.
• Thehe said/she saiddatadealsare also highly irrelevant. What is really needed is further context. Before Google inserted Google+ in their search resultsthe Google+ social network was far less successful than MySpace(which recentlysold for only $35 million). If social media is added as an annotation to other 3rd party listings then I think that has the opportunity to add valuable context, but where a thin"me too"styled social media post replaces the publisher content it lowers the utility of the search results& wastes searcher's time. Further, when those social media resultsare little more than human-powered content scrapersit also destroys the business models of legitimate online publishers.

Over-promotion vs"Search Spam"

At any pointGoogle can promote one of their new verticalsin a prominent location in the search results& if they are anywhere near as good as the market leader eventually they can beat them out of nothing more than the combination of superior search placement, monopoly search marketshare,account bundling& user laziness. What's more, they can make paid products free and/or partner with competitors 2 through x in an attempt to destroy the business model of anyone they couldn't acquire (talk to Groupon).

Amit Singhalis obviously a brilliant guy, but I thought some of the answers he gaveduring a recent interviewby Danny Sullivan were quite evasive& perhaps a bit inauthentic. In particular, ...

• "The overall takeaway that I have in my mind is that people are judging a product and an overall direction that we have in the first two weeks of a launch, where we are producing a product for the long term."If the product wasn't ready for prime time you were not required to mix it directly into the organic search results right off the bat. It could have been placed at the bottom of the search results,like the"Ask on Google"links were. Bing has been working on social search for 18 months& describes their moves as"being very conservative."
• "The user feedback we have been getting has been almost the other side of the reaction we’ve seen in the blogosphere."Of course publishers who see their content getting scraped& see the scraped copy outranking the original have a financial incentive to care about a free& automated scraper site displacing their work. They don't get those pageviews, they don't get that referrer data, and they don't get those ad impressions. Google's PR team isanything but impressedwhen another company dares do that to Google.
• "The users who have seen this in the wild are liking it, and our initial data analysis is showing the same."Much like the Google Webmaster Tools shows that pages with a +1 in the search results get a higher CTR, this Google+ social stuff also suffers from the same type of sampling bias& giving the listings a larger and more graphical stand out further help them pull in much more clicks.Any form of visual highlighting& listing differentiation can lift CTR. I might be likely to click on some of my own results more, but when I do so you might just be grabbing a slice of navigational searches I was going to do anyway where I was looking for something else I posted on Google+ or my Google+ account or the account of a friend& so on. Further, aggregate data hides many data points that are counter to the general trend. I have seen instances of branded searches where the #1 organic site was getting a CTR above 70% (it even had organic sitelinks, further indicating it was a navigational search) and for such a search in some cases there were 2 Adwords ads above the organic results& then the Google+ page for a brand outranked the associated brand in the SERPs for those who followed it! That is a terrible user experience, particularly since the + page hasn't even had any activity for months.
• "Every time a real user is getting those results, they really are delighted. Given how personal this product is, you can only judge it based on personal experiences or by aggregate numbers you can observe through click-through."First, publishers are not fake users. Secondly, as mentioned above, there is a sampling bias& the + listings stand out with larger& more graphical listings. If they didn't get a higher CTR that would mean they were *really* irrelevant.
• "out of the gate, whereas we had limited users to train this system with, I’m actually very happy with the outcome of the personal results."They could have been placed at the bottom of the search results or off to the side or some such until there was greater confidence in the training set.
• "People are coming to a conclusion about the product today, within the first two weeks, and they’re not fully seeing the potential where we can build this product around real identities and real relationships."If a publisher promotes a site to the top of the search results& then says something like'we will improve quality later'they are branded as spammers. In the past Google has justified penalizing a sitebased on its old content that no longer exists on the site. Investing in depth, quality& volume is a cycle. If others get prohibited from evolving through the cycles due to algorithms like Panda then it becomes quite hard to compete as a new start up when Google can just insert whatever it wants right near the top& then work on quality after the fact.
• "We don’t think of this as a promotional unit now. This is a place that you would find people with real identities who would be interesting for your queries."If this is the case then why does it only promote Google+?
• "We’re very open to incorporating information from other services, but that needs to be done on terms that wouldn’t change in a short period of time and make our products vanish."The problem is, if a company builds a reputation as a secretive one that clones the work of its partners& customers then people don't want to do open-ended transparent relationships. Naive folks might need to see the blood and tears 3 or 4 times to pick up on the trend, but even the slowest of the slow notice it after a dozen such moves.
• "I’m just very wary of building a product where the terms can be changed."Considering Google's lack of transparency& self-promotional bias on the social networking front, would you be fully transparent and open with Google? If so, then aren't the search algorithms complex enough that it would make sense to make those transparent as well? How can you ask other social networks to increase transparency at the same time Google is locking down their search data on claims of protecting user privacy?
• "It’s not just about content. It’s about identity, and when you start talking about these things and what it takes to build this, the data needed is much more than we can publicly crawl."This is where being trustworthy is so crucial. Past interactions with Yelp, TripAdvisor& Groupon likely make future potential partners more risk adverse& cautious. Outrageous"accidents"like those that happened with Mocality& Open Street Map from playing fast and loose further erode credibility. And even when Google hosts the media& has full access to user data they still rank inferior stuff sometimes (likethe recent Santorum YouTube cartoon fiasco), even on widely searched core/head keywords.

The big issue is that if people feel the game is rigged they won't have much incentive to share on Google+. I largely only share stuff that is irrelevant to tangentially relevant to our business interests& won't share stuff that is directly relevant, because I don't want to be forced to compete against an inferior version of my own work when the deck is stacked so the inferior version wins simply because it is hosted on Google.

As we move into the information age a lot of physical stores are shutting down. Borders went bust last year. Sears announced the closure of many stores. And many of the people shopping in the physical stores that remainare using cell phones for price comparisons. Given Google's mobile OS share this is another area where they can build trust or burn it. A friend today mentioned how their online prices on Google Product search almost always show a lower price near the header than the lowest price available in the list - sometimes by a substantial margin.

Identity vs Anonymous Contractors

In the past we have mentioned thattransparencyis often a self-serving& hypocritical policy by those atop power systems who want to limit the power of those whom they aim to control.

WhenGoogle was caught promoting illegal drug adsthere was no individual who took the blame for it. When the Mocality scraping& the Open Street Map vandalism issues happened, all that we were told was that Google"was mortified"and it was"a contractor."If people who did hit jobs could just place all the blame on"the contractor"then the world would be a pretty crappy place!

Eric Schmidtwarnedthat"If you have something that you don't want anyone to know, maybe you shouldn't be doing it in the first place."That sage advice came from the same Eric Schmidtthat blackballed cNetfor positing personal information about him. Around the same time Eric offered the above quote,Google was engaged in secret& illegal backdoor deals with direct competitorsto harm their own employees.

What happened to Google recruiters who dared to go against the illegal pact?They were fired on the hour:

"Can you get this stopped and let me know why this is happening?"Schmidt wrote.

Google's staffing director responded that the employee who contacted the Apple engineer"will be terminated within the hour."

When Google+ launched they demanded that you use your real nameor don't use the product. They later claimed thatyou can use a nickname on your account as well, butthere is a difference between a nickname and pseudonyms.

What is so outrageous about the claims for this need for real identities is that past studies have shown thatpseudonymous comments are best& Bruce Schneier highlighted how we lose our individualityif we are under an ever-watchful eye:

Cardinal Richelieu understood the value of surveillance when he famously said,"If one would give me six lines written by the hand of the most honest man, I would find something in them to have him hanged."Watch someone long enough, and you'll find something to arrest -- or just blackmail -- with. Privacy is important because without it, surveillance information will be abused: to peep, to sell to marketers and to spy on political enemies -- whoever they happen to be at the time.

Privacy protects us from abuses by those in power, even if we're doing nothing wrong at the time of surveillance.

In many marketsads and content are blendedin a way that is hard to distingush between them. Whenever Google wants to enterthey can demand greatertransparencyto participate(and then use the standard formatted data from that transparency to create a meta-competitor in the market.)

Increasingly Google is placingmore of their search data& their webmaster-related functionsbehind a registration wall.If you are rich& powerfulthey willsell you the data. If you arethe wrong type of webmasterthat aggregate data can be usedin *exceptionally* personal ways.

User Privacy

Ahead of Google updating their privacy policy Google has directed a large portion of their ad budget toward ads abouthow they protect users online.

What better way to ensure user privacy than to allow them to register their accounts under psydonyms? The real name policy on Google+ was part of what made Google want to stop providing referrer data for logged in users who search on Google. This has had a knock on effect where other social sites areframing everything,requiring registration to read more of public user generated content&sending outbound traffic through redirects.

Google'snewprivacy policyallows them toblend your user datafrom one service into refining the experience (and ads) on another:

If you’re signed into Google, we can do things like suggest search queries – or tailor your search results – based on the interests you’ve expressed in Google+, Gmail, and YouTube. We’ll better understand which version of Pink or Jaguar you’re searching for and get you those results faster.

Google& Facebook'swar (against) user privacyis catchingmediaandgovernmental attention. Microsoft highlighted some of Google's issues intheir"putting people first"ad campaign& the blowback has caused Google not only to publish PR-spin"get the facts"styled blog posts, but to launchyet another ad campaign.

EU regulators have asked Google topause their privacy policy changes.

Bogus Testimonials& Social Payola

Is social media a cleaner signal than links? If search engines put the same weight on social media that they put on links it would get spammed to bits. It won't be long until a firm like Ad.ly offers sponsored Google+ posts.

Some have suggested thatyou won't be able to buy Google+ followershowever Google already includes user pictures on AdWords ads (even when they desire not to be&even when they didn't endorse the product that Google suggests they endorsed). In due time I expect Google will indeed sell followers& other user interactions as ad units (just like Twitter& Facebook do).

Further, celebritiessell Tweets to advertisers. When they are hottheir rates go up:

When Ad.ly introduced self-destructing Charlie Sheen to Twitter, he was paid about $50,000 per tweet. It was worth it. Sheen’s tweet for Internships.com generated 95,333 clicks in the first hour and 450,000 clicks in 48 hours, created a worldwide trending topic out of #tigerbloodintern, attracted 82,148 internship applications from 181 countries, and added 1 million additional visits to Internships.com.

Search engines might consider these to be clean signals if those same search engines were not busy buying the manipulation of said"relevancy"signals.

Attention is purchased to create demand. It isn't comfortable to put it this way, but we are trained to obey authority&to likewhatothers like:

The average Facebook user has 130 friends, which equates with four degrees of separation to thousands of people, Mr. Fischer said. Metrics like that led him to believe that if Facebook could figure out a way to capitalize on"social endorsements,"it would be like creating a word-of-mouth campaign that could reach millions of people simultaneously. Since the campaigns would come from a friend, they would theoretically be taken more seriously than, say, a TV commercial, he said.

On an individual basisreviewsandratingsget faked everywhere. Even stodgy old slow-moving institutions like collegesgame their ranking systems.

There recently was a question raised abouthow Google's rating systems skewed high on the underlying data. Surely Overstock (the same Overstock Google penalized earlier this year) wouldn't promote Google's trusted stores aggressively on their own site if it made their business appear worse than it actually is, thus a positive bias must be baked in to the system.

Entire categories of demand are created by thosetied inwith powercost shifting to create bubbles. The federal reserve helped spark a real estate bubble with low interest rates.FBI warnings of mortgage fraudwere ignored. Consumers were constantly fed propaganda about"real estate only goes up."Then when that bubble popped, the US government bailed out those who caused it& burned trillions of Dollars propping up home prices. The government even bailed out a company that isnow shorting the housing market(when that company was about to get bailed outthe secretary of treasury leaked that material non-public information to some of his criminal investor buddies).

Does all the above sound circular, conflicting, corrupt& confusing? It should, because that is how power works& comes off as seeming semi-legitimate when acting in illigitimate ways. Theperceptionofrealityiswarpedto create profitable opportunties that are monetized onthe way upandthe way down.

Millions of kidstake drugs thataddress the symptomsof being a child full of energy, imagination& entusiasm.In some cases they may need them, but in most cases they probably don't. The solution with the highest economic return gets the largest ad budget, even if it only treats symptoms.

Web Scrape Plus+ (Now With More Scraping)

When the +1 button& Google+ launched, Google highlightedhow they would use the + button usage as a"relevancy"signal. Google recently startedinserting + pages directly into the search results for brands& right from the very startthey were using it as a scraper website that would outrank the original content source.

Google used the buy in from their promised relevancy signal to createa badge-based incentivized system which acts as a glorified PageRank funnelto further juice the rankings of these new pages on a domain name that already had a PageRank 10.

I recently read a blog post abouthow anyone could do the above& the opportunity is open to everyone. But the truth is, I can't state that something will become a relevancy signal that manipulates the search results in order to get buy in. Or, if I did something which actually had the same net effect, Google would likely chop my legs off for promoting a link scheme.

Recently the topic of Google+ as a scraper site came up yet againvia Read Write Web& on Hacker News a Googler stated thatit was"childish"to place any of the blame on Google!!!!!!

Google determineshow much information is shown near each listing& can create"relevancy"signals in ways thatthings tied to Googleget over-represented (look at the +1 count here). When they do that& it destroys other business models *of course*Google deserves100% of the blame.

Thin Content& Scraper Sites

Remember the whole justification for Panda was that thin content was a poor user experience?

In spite of sites like eHow getting hit,Google is still pre-paying them to upload content to Youtube.

Now that the (non-Google hosted) thin content has been disappeared (and the % of downstream traffic from Google to Youtube has more then doubled in the past year) it is time for Google to take another slice of the search traffic stream withSearch Plus Your World:

The Google vs Facebook locked down walled garden contest will retard innovation. As the corporate internet silos grow largerthe independent web withers. Them going after each other may leave room for Twitter, butit doesn't leave lots of room is left for others, asthe economics of publishing have to workor the publishers die.

Start ups that were on a successful trajectorywere killed by Panda:

The startup had been on a roll up until last February when Google altered its ranking algorithm with the release of “Panda.” The changes decimated TeachStreet’s traffic, and the company never quite recovered.

“We lost a lot of our traffic, and overnight we started talking to partners for biz dev, not for acquisition,” he said. However, many of the potential partners wanted to know about an outright acquisition.

About.comwas also smoked by Google:

The biggest worry, though, is that the decline of About.com itself may be irreversible. Fewer people are clicking on About ads placed by Google and the site’s own display ads have dropped in value.

The company has attributed this decline in value to Google’s decision last year to downgrade About pages in its search results. With more than 80% of traffic coming from search, the Google denigration was indeed a blow but About’s problems may be rooted in something deeper.

Keep in mind that the reason these websites were hit was that they were claimed to be thin& thus a poor user experience. When the NYT bought About.com one of the top competing bidderswas Google!

Now that the"thin content"has been demoted in the search results Google can integrate deep content silos from Google+, like this one:

That is an 8-word Google+ post about how short another blog post is. I like Todd& do like to read his writings, but here Google is clearly favoring the same sort of content they would have torched if it was done on an independent webmaster's website.

How Google has raters view other websites that redirect traffic is based upon those sites having a substantial value add. Clearly in the above example there was nothing added to the interaction beyond sharing a bookmark with a punchy tagline.

If Google wants to use the + notation to pull up that other referenced page then perhaps that can make sense, but to list an 8-word Google+ page in the search results nearly a year after the Panda algorithm is outrageous. This sort of casual mention integration in the search results occurs on expensive keywords as well. Not only do they list your own Google+ posts...

...but they also list them from anyone you follow...

In addition to information pollution, the other big issue here is time. Google wants to make forms more standardizedto make filling them out faster&they give regular sermons on the importance of fast search results. Yet when I do a navigational search, Google delivers two AdWords ads, a huge Google+ promotion, and then the navigational search result barely above the fold.*

*Since I thought the above was obnoxious, I renamed our Google+ company page toS_E_O Bookto help Google fix their relevancy problems.

Can anyone explain how Google's speed bias is aligned with putting plus junk right at the top, even on brand searches? Yahoo! has been pretty aggressive with putting shopping ads in the search results, but their implementation is still a better user experience than what Google did above.

And Bing offers an even cleaner experience than that.

Due tohow Google integrates Google+ in such a parasitic wayI seeno incentive for participating on their networkexcept when I have something that is outside of my domain of expertise, something that I am not targeting commercially, something that is thin, or something irrelevant to say! That incentive structure combined withGoogle's photo meme featurewill ensure thatcontent marketerswill help plenty of people see Star Wars stuff ranking formortgage loansearch queries.

When you own search/navigation you own language. that position can easily be extended into any other direction/marketin a way a social graph can not:

"The only technology I’d rather own than Windows would be English,"McNealy said."All of those who use English would have to pay me a couple hundred dollars a year just for the right to speak English. And then I can charge you upgrades when I add new alphabet characters like ‘n’ and ‘t.’ It would be a wonderful business."

Further, Google can chose at any point torespond toorignoremarket regulations in accordance with whatever makes them the most money. They can also fund 3rd parties doing the same (like undermining copyright) to force others to strike an official deal with Google to be"open."

A lot of businesses live on small profit margins, so Google's ability to insert itself& fund criminal 3rd parties aligned with Google's internal longterm interests is a big big big deal. Companies will learn that you either work with Google on Google's terms or you die.

When a public relations issue brews they canquickly change their approachand again position themselves as the white knight.

Brand Equity& Forcing the Brand Buy

Yahoo! put outa research paper highlighting activity bias, stating that the efficacy of online advertising is often over-stated because people who see ads about a topic were already more closely tied in with that particular network& that particular topic before they even saw the ad. As an example, any person who sees an AdWords ad forhemorrhoid treatmentwas already searching for hemorrhoid-related topics before they saw your ad (thus they were in the subset of individuals that might have came across your site in some way if you were in the search ad ecosystem or not).

This sort of activity bias-driven selection bias (homophily) exists on social networksonline&offline.

Google didresearch on incrementality of ads& they came to the opposite conclusion as Yahoo! did. Google suggested you should buy, buy, buy, even on your own branded keywords. They suggested that testing was expensive (no mention that the only reason it is expensive is because Google chooses not to make such tools easily accessible to advertisers)& that the clicks were so cheap on branded keywords that you should buy, buy, buy. Many advertisers who mix brand& non-brand keywords together don't realize thatthey are using the"returns"from bidding on their own brand to subsidize over-paying for other keywords.

Google Analytics is the leading& most widely used web analytics program. They can sharewhatever metricshelp them sell more ads (defaulting to crediting the last click for conversions, even if it was on a navigational search to your site)& pull back on features that are not aligned with their business interests (SEO referral data anyone?)

This goes back toScott McNealy's quote:"The only technology I’d rather own than Windows would be English. All of those who use English would have to pay me a couple hundred dollars a year just for the right to speak English. And then I can charge you upgrades when I add new alphabet characters like ‘n’ and ‘t.’ It would be a wonderful business."

Analystsdidn't understandwhyGoogle CPC rates were down8%last quarterwhile overall search clicks were up 34%. The biggest single reason was likely more clicks on adlinks on branded AdWords ads. While a brand buying its own keyword typically pays far less per click thanwhat some of the biggest keywords go for, the branded keywords typically have an exceptionally high CTR.Those additional clicksdragged down Google's average CPC, but the extra revenue they offered was a big par of the reason why Google was about to grow at 25% even though their display network only grew at 15%.

That slow growth of display is in spite of Youtube now serving over 4 billion video streams per day& Google adding display ads to log out pages.

Online viewsare not the same as TV views. A comScore study found that31% of display ads are never seen. In spite of that, US online advertising willreach nearly $40 billion this year.

Google wants to insert itself as a needed cost of business in the same waycredit card companies have.

On Google Maps they put an ad inside your location box.

Even ifmost people don't participate on Google+, Google canstill force advertiser buy inthrough over-promotion of the network in the search results. On your branded keywords they may drive your organic listing below the fold& put Google+ front& center.

Facebook earningsare still growing much faster than Google's& Facebook encourages advertisers toadvertise their Facebook pages, so even when you pay for the click Facebook still keeps the user. Facebook isadding apps to the timeline&is trying to win VEVO music video hosting from YouTube.

While Google is primarily known as a search company, it is getting harder to get off of Google though any channel other than a toll booth. Google keepsdriving the organic search results downward, while Google verticals fill up many of the organic results that remain. Many companies already buy Google ads on their own YouTube content. Some buy ads on Google to drive them to their Youtube videos& then buy ads on their own Youtube video to promote their websites. Soon Google will try to push you to buy them on your Google+ page as well.Google is becoming a walled garden:

Google wants to control more elements of your social world now. They don’t just want to be a search engine.

Is that so bad? Maybe not. It’s certainly no different from how other companies, from AOL, to Microsoft, to Apple, to Disney, to Facebook, have viewed the world — as ideally a walled garden, an all-consuming platform that most people use for pretty much every form of entertainment and social interaction.

A lot of people thought that Google was somehow different. They were, of course, wrong.
...
To move forward either as the old Google or Google+, Google needs to be capable of making fair deals with the partner ecosystem. It needs to curb its instinct to kill competing media companies that were actually producing great content that Google helped you find.

I suspect there will be plenty of bloodshed before Google figures that one out.

"This is the path we’re headed down – a single unified, ‘beautiful’ product across everything. If you don’t get that, then you should probably work somewhere else."-Larry Page

Google no longer believes in the concept of the open web. Blame it on Larry Page becoming the CEO, blame it on him talking to Steve Jobs& Steve telling him to make fewer and tighter products, blame it on Google funding eHow, or blame it on basically anything. But if you go back far enough, much of the stuff that is going on nowwas clearly envisioned a decade ago:

I was lucky enough to chat with Larry one-to- one about his expectations for Google back in 2002. He laid out far-reaching views that had nothing to do with short-term revenue goals, but raised questions about how Google would anticipate the day sensors and memory became so cheap that individuals would record every moment of their lives.He wondered how Google could become like a better version of the RIAA- not just a mediator of digital music licensing - but a marketplace for fair distribution of all forms of digitized content. I left that meeting with a sense that Larry was thinking far more deeply about the future than I was, and I was convinced he would play a large role in shaping it. I would rather jump on board that bullet train than ride a local that never missed a revenue stop but never."- Douglas Edwards

What happens when the Google+ version of your content outranks the version on your own site? And what happens when your branded channel and/or your fans become a vertical ad silo Google sells to your competitors?

I tested submitting a couple posts to Google+ with a Wordtracker top keywords list& valuable keywords (on a cpc*traffic) basis in posts about top keywords. Those posts rank #2 or #3 in Google for many people that follows me. No harm to me since those posts were irrelevant to this site, but if they were about my theme& topic I just would have out-competed myself. When Google outranks you (even with a copy of your content) they get to taste the data again and sell off the attention another time. You only get a slice of that monetization, even when it is your work that is being monetized. Maybe it is great for stuff that is somewhat less relevant and/or keywords that are so competitive that you otherwise wouldn't score for them, but we have to be really careful we don't out-compete ourselves. Though if Googke keeps this up they won't be the only ones monetizing it. Give it a few months and celebrities will be selling sponsored Google+ posts based on some metric created by multiplying search volume, CPC& how many followers they have.

Is Bing Better? Will Enough People Ask That Question to Matter?

For years Google built their reputation as being the search engine that offered the cleanest& fastest search results. They were known for monetizing less aggressively than the competition. But over the past couple yearsGoogle has dialed up their adsto wherethey now send a greater ratio of ad traffic than organic search traffic. One Google engineer recently described the ability to rank highly in Google without buying their ads asbeing a bug that was getting fixed!

Google's big risk in their coupling of aggressive monetization,aggressive self-promotion& changing how users feel about user privacy is that they can create the perception that users should go elsewhere for for an honest or trustworthy search. This not only builds momentum for smaller search services likeDuckDuckGo&Blekko, but has also won praise for Bing fromGizmodo,Dave Winer&The Next Web.

You can learn a lot more about what Google really thinks by reading what their new hires say. They are not yet skilled in the arts of public relations& make major gaffs like this one:

Instead of being able to SEO the entire Internet, businesses can now only affect the search results for a tiny percentage of users. That's a good thing because SEO can't scale, andSEO isn't good for users or the Internet at large.

If you look at the Google experience from the standpoint of customers, it's pretty good. Users get relevant search results and ads. Advertisers get their content on top of everything else. It's a good compromise between advertising and usability, and it works really well.It's a bug that you could rank highly in Google without buying ads, and Google is trying to fix the bug.Manipulating Google results shouldn't be something you feel entitled to be able to do. If you want to rank highly in Google, be relevant for the user currently searching. Engage him in social media or email, provide relevant information about what you're selling, and, generally, be a"good match"for what the user wants. -Googley Jon Rockway

Would love to hear someone more senior confirm this as the official Google company position, however they are too skilled at public relations to make that blunder (at least outside offoreign AdWords adsthat tell you to"forget SEO").

Ahrefs is the newest entry into the link research tool space. They use their own bot and their own index (which they state is based on information from a trillion website connections).

They claim their index is updated every 30 minutes and the fresh data is available to their users within 30 minutes of the actual index refresh.

Ahrefs also has a ranking database of roughly 45 million keywords from 9 different countries (US GB FR RU DE ES IT AU BR). The tools within their membership are:

• Site Explorer
• SERPs Analysis
• Reports
• Labs/Tools

Their pricing is very straight-forward and only increases or decreases based on volume of data you have access to. You can check out the easy to understand pricing on theirpricing page(and they offer SEO Book readers a 50% discount on the first month).

Site Explorer

Ahref's Site Explorer functions in a similar way to Majestic's Site Explorer and SeoMoz's Site Explorer. You can choose a specific URL, the domain without subdomains, or domain with all its subdomains:

If we look at the Site Explorer results, you'll see an overview of the last 45 days or so from Ahref's crawl history:

On the left you can see some interesting stats like the total number of backlinks, different referring IP's and subnets (class c blocks and such), unique domains, and the types of backlinks the site has (text, image, redirects, and so on).

In addition to the overview report, you have other research options to chose from:

• New Links
• Lost Links (great opportunity for you to swoop in and alert the linker + sell them on linking to you and your resources)
• Anchor Text Profile
• Pages Crawled on the Site
• Referring Domain Breakdown
• SERP Positions (organic ranking report)
• Raw Export of the Data (up to your limits based on your pricing plan)

New Links

In the New Link tab you can go back to a previous month, or work inside the current month, and find newly discovered links by the day. Here is what that looks like:

Click on whatever day you want and you'll get a list of linking urls, the target link page, and the anchor text used for the link:

This report can help you reverse engineer, down to the day, a link building campaign that your competitor is running (always good to be out in front of a big link push by a competitor) and can also help you evaluate your own link campaign or even help you spot a link growth issue that may have resulted in some kind of penalty or over-optimization filter.

Now keep in mind that, based on their stated crawling guidelines, the stronger links from stronger sites tend to get crawled more frequently so the spammiest of the spammy link approaches might not get picked up on. For that level of deep research a historical report from Majestic SEO and a link status checker, like Advanced Link Manager, is likely a better bet.

You can export this report to Excel or .CSV format.

Lost Links

The Lost Links tab has the same interface as the New Links report does. For your own domain you might want to consider tracking your own links in something like Raven or Buzzstream but this tool does report dropped links down to the day. Combine that with their crawling preferences (better links = quicker attention) and you can spot drops of substance quickly.

You can use this report to find links that a competitor has lost, off of which you can contact the webmaster and see if you can't promote your site or similar content to earn the link your competitor was previously getting.

You can export this report to Excel or .CSV format.

Anchor Text

The anchor text report is exactly what you expect it to be. It lists the anchor texts of external links, the number of occurrences, as well as an expandable dropdown menu to see the pages being linked from and the pages being linked to on the site you are researching.

You can export to Excel or .CSV and choose to export everything, up to your limit, or just the current page.

Crawled Pages

This report will show you all the pages crawled by Ahrefs with the following stats:

• Page URL and Title
• Crawl Date
• Page Size
• Internal Links
• External Links

I would likely use this report (on competitors) for checking some of their more popular internally linked-to pages as well as checking out how they structure their site. You can also jump right to a site explorer report for any of the URL's listed on that report as well as check the SERP positions for any of them.

Referring Domains

One thing I like about Ahrefs is that it's straight and to the point. It's very easy to get in, get your data, and get out. Each report does pretty much what you expect it to. This report shows the referring domains + number links coming from that domain. You can access the links from each domain by clicking the Expand button next to the referring domain:

SERP Positioning

Similar to SemRush, Ahref's provides estimated ranking data for keyword sets on both Google and Bing/Yahoo in multiple countries (US, UK, AU, DE, FR, ES, IT, BR, RU). The tool shows the:

• Position
• Keyword
• CPC
• Estimated cost
• Ranking url
• Global search volume
• Advertiser competition
• Last date checked
• Rating (estimated visitors per month based on assumed traffic distribution)

The other cool thing about this report is that it will tell you the change from the last time they checked the ranking.

SERPs Analysis

This is similar to the SERP positioning report. Essentially, you enter a URL and you get the Google + Bing& Yahoo ranking data with those same metrics as stated above:

• Position
• Keyword
• CPC
• Estimated cost
• Ranking url
• Global search volume
• Advertiser competition
• Last date checked
• Rating (estimated visitors per month based on assumed traffic distribution)

In addition to that, you also have the following reports:

• Daily Stats
• History of Changes
• Ads Analysis

Daily Stats

This report shows you, on a daily basis, the following data points:

• New Keywords
• Lost Keywords
• Total Keywords that moved up
• Total Keywords that moved down
• Total Positions up
• Total Positions down
• Rating Change (estimated percentage of traffic gained or lost)
• Cost Change (rating change * CPC)

There are graphical charts for:

• Search Engine Traffic (shown above)
• Keyword Trend (total keywords ranking)
• Traffic Cost
• Bar Graph for New and Lost Keywords
• Estimated Traffic Changes
• Estimated Traffic Cost Changes

History of Changes

This report breaks down the keyword changes by day and how much the specific keyword moved up/down (and the corresponding page that is ranking).

You can look at a daily report, a 7 day report, 30 days, or a custom range.

Ads Analysis

Ahrefs also incorporates Google (and Bing/Yahoo but I had a hard time getting figures for Bing/Yahoo) PPC data. You can pull in the ranking of the ad, the ad text, volume& CPC data, as well as last updated date& competition levels.

You can look at just the keyword/ranking data or choose from their other 2 reports; keywords/ranking + ad text (Table + Ads) or just the PPC ad text itself (Ads Preview).

Reports

You can create reports for your own domain for free or a any other site as a part of your subscription. Each domain counts as a separate report, so you can enter as many as you are entitled to in this interface but they do count against your monthly allowance.

The report overview looks like this:

Each tab represents a data point you can review. In any tab you can choose to export the visible page or the entire report.

There are quite a few filtering options here, as you can see below:

Your filtering options, report-wide, are:

• URLs from - you can include or exclude based on user-defined data (exclude by word(s), domain extension, and so on)
• Backlinks Type - you can choose to show, specifically, different backlink types (nofollow, image, frame, redirect, form, deleted)
• Pages - show links only to a specific page
• Subdomain - show links only to a specific subdomain
• Countries - show links from specific countries
• Anchors - show or exclude specific anchor text links
• Referring Domains - show links from a specific domain, or set of domains only
• IP - show links from a specific IP or range of IP's only
• Subnets - show backlinks only from specific subnets
• TLD - show links from specific TLD's only
• Date - show links based on specific crawl period

The cool thing here is that you can layer on the filters as you wish. The following screenshot shows all filters selected and available:

The reporting is really quite powerful and provides numerous ways to quickly filter out junk links so you can focus on the good stuff.

Labs

There are 3 additional tools in their Labs section.

• Ahrefs Top - Top 1 million domains by number of backlinks, completely searchable
• Domain Comparison - compare up to 5 domains for different link metrics (see below)
• Batch Domains - (see below) dump in a bunch of URLs and get a total count of backlinks, referring domains, and IP's. Unsure of the limit here but I did about 25 with no problem.

Here is a screenshot of the domain comparison feature:

The Batch Domains feature looks like this (and is completely exportable!):

Ahrefs is Worth a Spin

I was impressed with the speed of this tool, the exportability of the data, and the report filtering capabilities. It hardly hurts to have another link database to pull from, especially one that is updated every 30 minutes.

The tool is quite easy to use and it does pretty much what you expect it to. If you are into link research you shouldgive this tool a try. The database appears to be a fairly good size for a new database and the ability to slice and dice that data from right within the web interface is a solid feature. If you do try it out, let us know what you think! We are also adding their link data to SEO for Firefox& the SEO Toolbar today.