I don't buy the point that once source code is published, you have to do all the "customer support" stuff. You don't have any obligation to do customer support so that you should be able to decide when you'd like to reply the request or whether reply it at all.
Source code is a very precise way to exhibit the work you have done. It is so precise that even particular machine can understand it. I am really happy to see that recently in NIPS more and more people published their MATLAB code along with their papers.
Oh, you must do "customer support"! Otherwise, they will use your code and report 1) that your code does not work or that 2) their code is better because they did not use your code well.
And for code that is not easy to install (e.g., requires many libraries, frameworks, dev tools, etc.), you need to spend time on documentation and even then, you often need to answer questions like "how can i create an eclipse plug-in to use your code or lxml failed to install"...
"Otherwise, they will use your code and report 1) that your code does not work or that 2) their code is better because they did not use your code well."
This x 1000! This is a topic near to my heart as I've actually had people use models & software we provided to them, not making an effort to understand fundamental concepts and theory behind it (or just being incapable of understanding) and then publish papers saying our models are crap! (not even providing better ways, nor comparing to anything else, just saying 'I put in some data and the results were unrealistic, this model is bad).
I agree. I've released my code, and there are several examples of people taking it and using it to do stuff that is totally outside its range of reliability because they just ran the code and didn't bother to learn enough about the problem. Then they publish crap and the crap comes back to me because MY code gets a bad rep for creating incorrect results.
Mostly you only concern about the criticism from within scientific community. Since your result is published, generally, people with background will spend time to figure out how to use it by themselves. And their criticism is only relevant when it gets published. At that point, you are surely obligated to response such criticism. Most of the time, it is really not relevant that some random people send you email and have such "eclipse plug-in" problem.
"some random people send you email and have such "eclipse plug-in" problem" -> They are not random people, they are Ph.D. students working on their thesis and you can be assured that they will find a way to publish their comparison one way or the other. Some Ph.D. students will go at great length to figure out other's code before asking a question, but some will suck your energy big time and still do a poor job at representing your work. Now compare this to those who do not release their code: they can only be evaluated on their published results and they can spend their time on their next study/research work.
My point is that there is not a lot of incentive to release code: it requires a lot of your time, it is not always clear that granting agencies/employers consider this as being more positive than publications, and it exposes your work to all sorts of unfair replication/comparison.
Now, if research work without released code were never cited, that would be a terrific incentive to release your code :-)
P.S. I speak from experience and I release the code of my most important research work.
"Mostly you only concern about the criticism from within scientific community"
Ha. That doesn't apply to climate scientists. And there was the case of Andy Schlafly of Conservapedia (a creationist lawyer) demanding all sorts of nonsense from a scientist who documented evolution through many, many generations of bacteria.
At its worst, you get ideological cranks ganging up and demanding information, possibly through FOIA requests, pretty much for the sole purpose of using up all the scientist's time responding. Then when they don't respond, and/or it turns out they aren't subject to FOIA requests, the cranks freak out, break into a server, steal emails, and declare a conspiracy.
Source code is a very precise way to exhibit the work you have done. It is so precise that even particular machine can understand it. I am really happy to see that recently in NIPS more and more people published their MATLAB code along with their papers.