Matt, Would it be possible/financially practically for Google to offer a premium support option that would pay for itself?

Figure out how much it costs to respond to someone, offer that as a flat fee with a clear indication, that paying only gets you a consult on what's wrong with your site, and not any promise that it will get you reinstated. Basically what you did in your comments above.

I would personally make it expensive to make sure people don't try continually working the system by paying for the consult to get inside information about how to rank higher on Google.

I work in a support center, so I definitely recognize that it's expensive and difficult to do well at scale, so I appreciate that you're looking into how to do it!

It would be possible. I think Google has a bit of a historical aversion to it because it reminds us of pay-for-inclusion, in which someone would pay for their web page to be crawled/indexed.

Larry Page was especially critical of pay-for-inclusion because it skews your incentives: if you don't crawl the web well, then people pay you to fix your own shortcomings, which in turn encourages you to have more shortcomings.

I think Google also comes from the perspective of self-service AdWords being successful, so the idea of self-service (free) diagnostics really appeals to us. That's why we've put a lot of effort into our free webmaster tools.

I wouldn't be philosophically opposed to a pay-for-support system if it were done well, but it would be a tricky thing to get right. Normally when we consider it, we end up saying things like "Why don't we just try to make it so that people don't need that option?"

>Larry Page was especially critical of pay-for-inclusion because it skews your incentives: if you don't crawl the web well, then people pay you to fix your own shortcomings, which in turn encourages you to have more shortcomings.

It would be fine provided you explicitly didn't turn it into a profit center, and just made it pay for the people's time.

Torching people's websites and shrouding the reasons why in mystery skews webmasters' incentives - away from creating high quality content and towards figuring out how to circumvent the latest change to the search algo.

>I wouldn't be philosophically opposed to a pay-for-support system if it were done well, but it would be a tricky thing to get right. Normally when we consider it, we end up saying things like "Why don't we just try to make it so that people don't need that option?"

I'm sure you could do that tomorrow if you wanted, but explaining in perfect detail exactly why somebody's (legitimate) website got torched would open the details of your algorithm right up, which would not only open it up to gaming, but would open it up to being copied.

You really could use a team of humans who can explain in human terms (as opposed to algorithmic) exactly why webmasters' sites got torched for violating the spirit of your "high quality content rule". Those same humans could equally feed back data to the search team where in their opinion an algorithm accidentally torched something it probably shouldn't have.

> I wouldn't be philosophically opposed to a pay-for-support system if it were done well, but it would be a tricky thing to get right. Normally when we consider it, we end up saying things like "Why don't we just try to make it so that people don't need that option?"

This is going on the wall here at the office. Thank you.

Hi Matt,

We don't know each other but I think we know of each other. I'm rather immersed in webspam detection and found this incredibly interesting.

You imply that the challenge is finding a solution that scales. Yet it sounds to me from your response that this site was flagged via manual review. Did I misunderstand?

If I heard you correctly, then is manual review a significant equation in the webspam detection methodology? You guys are boiling the ocean so I find that rather hard to swallow.

The more likely conclusion I can draw is that he had a significant number of (auto-generated) pages on his site flagged as spam and that in turn raised some eyebrows.

BTW, you and your team are doing some amazing work. I wish the paid side was up to the standards you set.

The site was flagged both algorithmically and also escalated to a member of the manual webspam team.

The basic philosophy is to do as much as we can algorithmically, but there will always be a residual of hard cases that computers might not do as well at (e.g. spotting hacked sites and identifying the parts of a site that have been hacked). That's where the manual webspam team really adds a lot of value.

In addition to things like removing sites, the data from the manual webspam team is also used to train the next generations of our algorithms. For example, the hacked site data that our manual team produced not only helped webmasters directly, we also used that data to produce an automatic hacked site detector.

If you're interested, I made a video about the interaction between algorithmic and manual spamfighting here: http://www.youtube.com/watch?v=ES01L4xjSXE

I don't think it is really possible here to have a solution that provides a good outcome for all users and also scales. For every person who does a proper write up and gets some attention places like here there would no doubt be many others who have had the same issues but got no attention.

What would really help is a clear reasoning given for the ban and a set of steps a site can take to get back in. From what I see often web masters are more than willing to make changes, they just don't know which to make. Often to the bans are said to be final, denying someone the chance to make it better.

I guess it is tempting to provide minimal support for organic search as people aren't paying and generally there are plenty of other sites to cover ones absence. One thing that would be good is say a yearly fee which guarantees that you can get one on one support if something happens.

Hi Matt, thanks for chiming on on this thread.

From your reply, I understand that your primary issue is indeed with scaling any manner of support issues. As I understand it, the current thinking at Google is to seek automated solutions over technical to reduce costs and improve issue volume handling. With that in mind, is there a system of triggers that can flag issues for manual review or intervention instead of being handled completely automatically?

I think the majority of issues that have people up in arms aren't really that big, but there are the exceptions. Instead of monitoring G+/Twitter/etc., there could instead be a series of internal checks that look for particular criteria:

1. Age of campaign/ad - was it super old and not really applicable or not even running much budget? It may be worth a lighter hand than a complete ban because it isn't indicative of a trend or pattern on the part of the advertiser.

2. Amount of impressions/clicks/spend over time - what is the overall severity based on actual impact? Are users actively clicking on the ad, returning to the site and continuing to use it? As each click and therefore organic link is logged, it would be good to cross reference them and go "well, this site raises flags but has high return-user rates and decent sentiment in organic results"

3. Traffic vs. Content ratio - does the site have fairly thin content that still gets dramatic numbers of users? Tying in with #2, it basically can help tell if a site is offering something thin on content but highly unique and/or valuable. If people are using it, they may be on to something that the QS algorithm misses.

The one thing I'd love to see is just a little bit more of a verbose notification or warning for the people that get hit. While many only have themselves to blame, many want to make sure they stay on the right side and any chance they have to do so is of great benefit. Then again, it is also important to not give to much information to the black-hatters that just make use of it to get even better at gaming the system.

These are just thoughts, but it has long been on my mind considering the metric ass-ton I've run through AdWords for myself and clients over the years. Hope this helps spark a few ideas :)

"With that in mind, is there a system of triggers that can flag issues for manual review or intervention instead of being handled completely automatically?"

I can't speak for the ads folks at all, unfortunately, but on the websearch side we certainly strive to do as much as we can algorithmically, but also to use our manual cycles effectively. The site in question was flagged algorithmically, but also sent to a member of the manual webspam team who concluded that it violated our quality guidelines. When a reconsideration request came in, it went back to a manual person for review again.

To the extent we can figure out ways to do it without compromising our systems, I think both websearch and ads would like to be as transparent as we can.

Matt, I think the main problem (having seen what it looks and feels like on the receiving end) is that Google's solution and treatment of the site owner is, for lack of a better term, rather violent.

Put yourself in the shoes of someone who is not a scammer and is trying to do a good job. Perhaps someone how is learning the ropes. Getting hit with an unilateral, violent and --for all intents and purposes-- permanent action such as seems common in these cases can be devastating.

I'd like to relate a case that I witnessed that was truly perplexing. It involved about 200 domains that were registered with GoDaddy and placed in their "cash parking" program.

They were there for months with no issues whatsoever. One day, the domain owner realized that this cash parking service was Google AdSense with GoDaddy taking a bite of the minimal action. So...he moved all the domains to a product called "AdSense for Sites" (I don't remember the exact name). This services was marketed by Google as a place to park your domains and earn some money through advertising that Google would automagically place on these domains. Same service that had been on these domains for ages through GoDaddy.

The transfer went well and all domains were accepted. No problems. The domains go "live", if you will, and ads start showing as predicted.

Two days later the account is cancelled and the domains are taken off the program citing "suspicious activity". The irony is that the sites --and their content-- were handled entirely by Google through this "AdSense for Domains" product. No self-clicking activity took place whatsoever. No nefarious activity of any kind. This person was far too busy with real business to go around clicking on ads across 200 domains to make $0.25 at the end of the day.

This was simply a transfer to Google for the same service that Google had been providing through GoDaddy for many months, years in some cases.

There was no recourse. No way to speak to anyone. No way to even try to understand what this "suspicious activity" was all about. The account was banned, closed, done...forever.

Now, here's a person who had plans for legitimate and valuable real sites to be launched on some of these domains later on. The whole experience scared him to a point of simply rejecting the idea of doing anything with Google if he could avoid it.

We had no way to provide any kind of an argument to the contrary because of the violent and totalitarian nature of the cutoff. Who would want to do business under those conditions?

The fact of the matter is that AdWords/AdSense generated revenue could evaporate overnight and with no recourse whatsoever. That's a tough pill to swallow for anyone who is a legitimate entrepreneur looking to build value and make some money or earn a living through their efforts.

Scammers are a different matter. However, you seem to treat both groups with the same hammer which, in my humble opinion, is not right.

I now have to advise anyone we work with that any income that relies on Google for either lead generation or direct income (AdSense) has to be treated as though it could evaporate at any time and for any reason without any real opportunity given to restore it in a timely fashion. Without that caveat on the table I couldn't personally advise anyone to use your products.

In a normal business one would engage with ones vendors in order to resolve issues in mutually beneficial ways. In this case it is a one-way street with violent and severe consequences for your customers and partners. That's what you have to fix.

I take your points, and in my group we've worked hard in the last year to improve our transparency and our appeals process, e.g. see http://googlewebmastercentral.blogspot.com/2011/09/reconside... from a few weeks ago. Likewise, AdWords now has phone support for all AdWords advertisers.

I can't speak for the AdSense (for Domains) team other than to say that when they shut down an account, they think that they have good reason for it. And unfortunately, that's typically a situation where they can't give many details--if the team sees abuse, providing information about how the abuse was flagged would help spammers quite a bit.

I know that Google can seem abrupt sometimes, and I dislike that, but part of the issue is also scale. See https://plus.google.com/117377434815709898403/posts/1hRWj489... that notes that if each Google user had a single 10 minute issue every three years, that would need 20,000+ support people to handle that load. Or consider that there's 200M domain names, and all those webmasters want to talk to Google and ask questions.

Even this link is discouraging: http://www.theatlantic.com/magazine/archive/2011/11/hacked/8... It mentions that several thousand people get their Gmail account hijacked every day. Trying to support all the people who want to interact with Google in a scalable way is a really hard problem.

Support for all Google users would obviously be painful.

I think it would be fair if there was some personal support for people who either spend a lot of money on Adwords (I think there already is?) or make a lot of money on Adsense. These people probably have the highest levels of stress due to "insta-evaporation probability", and no doubt the amount of support people would be several magnitudes less if you used a threshold like that.

There is phone support for everyone who uses AdWords now. I don't know the support situation for AdSense though.

Sorry it took a few days to reply to this. Too busy writing code...

With regards to the argument that describes the resources required for customer support even to a small percentage of your audience I can only say this: It's your chosen business model. I am paraphrasing one of my favorite answers when someone complains about their job ("It's your chosen profession").

The point is that Google's business is about doing what it does for a huge number of people. If supporting them is overwhelming either get out of that business or figure out how to do it correctly. I can't really accept the "it's too many people" argument as a valid reason for not doing it well or for applying the "criminal algorithm" to everyone.

There's a side thread there that says that "providing information about how the abuse was flagged would help spammers quite a bit". While true, I, again, find myself not agreeing with the idea of punishing legitimate customers for this reason. I would like to think that the vast majority of Google customers fall under the "legitimate" category. If spammers get better because you are providing detail flagging information you will simply have to get better at detecting and blocking spammers. This would trigger an evolutionary phase which, at some point, should make it very difficult for a spammer to game the system, even with "full source" if you will. Much like security algorithms become more secure if the source is released and is tested with full knowledge of the internals, yours should do the same.

Conversely, honest and legitimate customers would gain the huge benefit of now understanding how to behave or how to do things and why a certain approach might not do well in Google's ecosystem.

One of the most frustrating things I have seen is someone full of drive to launch an internet business only to be shot down by a Google shutdown. And, when no reason or actionable information for the punishment is given this entrepreneur simply had to throw their hands up and give up on that tack. Needless to say, their next attempt ignored Google products completely and they are doing OK. My guess is that they could have done very well and much grief could have been prevented had Google said: "Your site has the following problems ..." and then, rather than cut them off schedule them for a review in, say, thirty days (or variable length based on the seriousness of the issue). That would have been far more civilized and far more conducive to helping your community grow and evolve in the right direction.

New and inexperienced internet entrepreneurs (and some experienced ones) need a way to learn how to behave. What works and what does not. What is acceptable and what isn't. It is only reasonable to assume that they will make many mistakes in their zeal to get an idea off the ground. Penalizing them with a permanent hammer blow to the head is not conducive to growing better netizens. Guiding them with actionable feedback is.

The current process can only be characterized as violent. From the perspective of an honest business person it is tantamount to getting hit with a bullet while walking your dog. The reasons could have been many. Maybe the shooter objected to your walking your dog in front of their home. Had the shooter at least attempted to communicate with the dog-walker it is far more likely that violence could have been averted.

Above all, if your "Do no evil" is sincere, then you have to change the way this works right away. The way this hits honest entrepreneurs is nothing less than pure evil. Again, you take a bullet and you don't know why.

I do appreciate your visibility here in HN. In the past I have simply given up trying to raise these and other points with anyone at Google that might remotely have the ability to at least elevate the conversation internally. I hope you might be that person. I mean all of the above in the vein of constructive criticism. We all want to see the ecosystem becoming more conducive to the exploration of new ideas. Google, at this time, has taken a rather totalitarian position of being the "moral authority", if you will. With that, and so long as you want to be a benevolent dictator, I think, you inherit the responsibility to not cause harm through your actions.

Having said that, until things change I have no choice but to treat your offerings as something that one simply cannot rely on to build a business. The "Google love" can disappear from your site overnight and you'll have no practical way to fix it. That's not a business, that's going to Vegas.