The problem I had with Haskell when trying it out was that every time I wanted to use a package with dependencies I ran into "cabal: Error: some packages failed to install. The exception was: ExitFailure 1.". The solution was most often trying earlier versions of the package until it worked or not use the package at all. Cabal just seemed so broken.
Not to diminish the Cabal problems here, but you run into similar problems with Maven as well. Version management is a real hard problem. Especially when there is little awareness in producing compatible successors. Especially this problem made my experiences with Haskell somewhat bitter. But I have the same problem (in really productive code) with Maven. Writing a working Maven plugin can be made pretty challenging by that as especially the stuff you want to use in Maven plugins is burdened with incompatibilities.
Haskell packages like to specify requirements using both a lower bound and an upper bound. So package A depends on B being version 0.55 <= B < 0.99. Package C depends on B being version 0.90 <= B <= 1.35. So if you installed C before A you will get version 1.35 of B and will now need to either downgrade B or have two versions installed. Chaos ensues.
Why Hackage packagers like to specify upper bounds I'm not sure of. Backwards compatibility either is very hard to achieve in Haskell or maintainers just don't care that much about it. I don't know about Maven so I can't say whether that is as bad. But at least pip, npm and gems are a breeze compared to cabal.
Packages are supposed to specify upper bounds because without them you can end up with spontaneous breakages when a transitive dependency several steps away changes in a backwards-incompatible way. Haskell has the PVP which can prevent this. It is pretty similar to Semantic Versioning (http://semver.org/). The upper bounds are not specified willy-nilly. They are specified in a way that includes backwards-compatible revisions, but excludes backwards-incompatible ones.
I've been using Haskell for more than 6 years and in my experience the worst build problems happen with parts of the ecosystem that don't specify upper bounds. There's been a lot of discussion about this in the community. We're aware of the issues and we're working on solving them. One attractive possibility being discussed is adding a flag to cabal that allows one to ignore upper bounds. In theory this seems like it can preserve the benefits we get from the added information in an upper bound while eliminating the pain. An extension to this idea is to make two operators (say < and <!) that specify a "soft" upper bound that is just the highest you've been able to test against and a "hard" upper bound where you're telling cabal that your package is known to break. This extra information would make the solver much better when you tell it to ignore upper bounds because it would only ignore the ones that are safe to ignore.
I suspect that this problem is more visible in Haskell than in other languages. Haskell's type system gives you a way to bound the number of things that you must understand about an API in order to use it. The confidence that we get from having these guarantees makes code reuse orders of magnitude easier in Haskell than it is in pretty much any other language in mainstream use today. Others have made this same observation. Here (https://www.youtube.com/watch?v=BveDrw9CwEg#t=903) is a presentation given at CUFP last year by a company using Haskell in production that gives the actual reuse numbers that they've seen for several different languages.
There's also been more recent mailing list traffic on it, but my Google skills are failing me here and the arguments haven't changed all that much.
My opinion is that we are relying too heavily on upper bounds to solve a whole bevy of interrelated, but separate, problems. Other solutions are evolving to help deal with some of the problems. cabal sandboxes allow you to build a package separately from your others, reducing dependency issues. cabal freeze will help you specify exact dependencies, including transitive ones. Another project, Stackage, aims to maintain a whole slew of packages in a buildable state:
It's an incredibly stupid policy. As others have noted, other package managers also has the capability of setting upper bounds on package versions and pip also has the ability to freeze your dependencies to specific versions.
However those features aren't necessary to productively use pip for example. Because Python package maintainers are very good at maintaining backwards compatibility (or using deprecation cycles when those are needed) because people that depend on their packages get upset with them. The rules are different for beta releases because it's your own fault if you depend on them and their api changes.
Precisely because specifying upper bounds is encouraged, Haskell maintainers seem to act as if they have a free card when it comes to backwards incompatible changes. In the long run that causes much more packaging troubles then specifying upper bounds saves. Perhaps there's nothing wrong with Haskell nor Cabal, but the Haskell community's way to handle backwards compatibility is wrong imo.
Upper bounds are information about what you know your package works with. It makes no sense to throw this away. What does make sense is improving our tooling to allow this information to be ignored in certain circumstances.
I've been on the other side of the fence with a large app written a long time ago that did not specify upper bounds. Now that app is essentially unbuildable given the time I'm willing to put in. If I and all my dependencies had followed the PVP and specified correct upper bounds everywhere, I would still be able to build my app today. No, I wouldn't be able to build it with the latest version of other libraries, but that's not what I want to do.
If you don't specify upper bounds, the probability of your package building goes to ZERO in the long term. Not epsilon...zero. That's unacceptable for me. If it works today, I want it to work for years to come.
"Upper bounds are information about what you know your package works with."
Not really; the upper bounds generally are speculative. They specify that the package will not work with particular versions when typically those versions have not yet been released. Often when those versions are released they work fine, which prompts a useless flurry of dependency bumping.
"If I and all my dependencies had followed the PVP and specified correct upper bounds everywhere, I would still be able to build my app today."
I now distribute my packages with the output of the ghc-pkg command so you can see the exact versions the library builds with. That solves your problem without using speculative upper bounds.
I wouldn't expect it to work for "years to come" in any event, though: Haskell changes too much. You probably won't be able to get today's package to work with the compiler and base package that will exist years hence. It's hard to even get an old GHC to build: today's GHC won't build it because of...dependency problems.
> Not really; the upper bounds generally are speculative.
Yes, an upper bound is still useful information. It says that a package is known to work with a certain range of dependency versions. This is why I think cabal should support something like < for speculative upper bounds and <! for known failure upper bounds. Then we could have a flag for optionally ignoring speculative upper bounds when desired.
> I wouldn't expect it to work for "years to come" in any event, though
I can still download a binary of GHC 6.12.3. Hackage also keeps the entire package history so there's no reason things shouldn't build for years to come. I'm all for the fast moving nature of the Haskell ecosystem, but we also need things to be able to build over the long term and there's no reason we shouldn't be able to do that.
> Which won't even work on current versions of Debian GNU/Linux because the soname for libgmp changed.
Again, my argument assumes that you're not upgrading to current versions of things. Enterprise production-grade deployments often fix their OS version because of this. There's a reason RHEL is so far behind the most recent versions. If you are upgrading, then you'll be fixing these problems incrementally as you go along and it won't get out of hand.
> No it doesn't. I had specified a dependency on a particular version of text
You're making my point for me. It looks like you were probably depending on 0.11.1.4, which doesn't exist now. Your problem would not have happened if you had used an upper bound of "< 0.12" like the PVP says instead of locking down to one specific set of versions. If you had done that, cabal would have picked up subsequent 0.11 point releases which should have worked fine. Also, I can still download text-0.1 which is more than 5 years old.
The old adage "Haskellers doesn't know how it feels to shot themselves in the foot because they are walking around with a bleeding flesh wound in their feet" fits aptly. The problem upper bounds tries to solve just does not exist in other languages because packagers are expected not to break backwards compatibility. I can almost without exception run the same apps I developed for Django 1.0 (csrf protection introduced in 1.2 caused backwards incompatible changes but that was an exception) almost ten years ago with current software versions.
In fact, if any of the packages demanded an upper bound it would suck because I don't want to use a legacy Django web server containing tons of exploits which no one is ever going to fix because it's not maintained anymore.
> The problem upper bounds tries to solve just does not exist in other languages because packagers are expected not to break backwards compatibility.
I'll agree that Haskell moves faster and has more breaking changes, but that statement is just wrong. Look at http://semver.org/ (not related to Haskell at all) and you'll see that the very first point on the page is about incompatible API changes. So clearly this issue exists outside of Haskell and people have developed methods for managing it with version bound schemes. You are arguing against rapidly changing software, not against upper bounds. In your example where the Django server is not making backwards-incompatible changes, upper bounds wouldn't hurt you at all because the bounds are on the major version number, but exploit fixes that don't break backwards compatibility will only bump the minor version number.
Comparing Haskell to any other mainstream language in this discussion is invalid because the other languages have been around a lot longer and have reached a more stable state. Python appeared in 1991. The first Haskell standard appeared in 1998. So that means Python has at least 7 years of stability on Haskell. I would argue that Haskell gained adoption much more slowly because it is much less similar to any mainstream language that came before it, so the actual number should be larger. Paul Graham's essay "The Python Paradox" came out in 2004. I would suggest that Haskell is just now getting close to the point that Python was at when PG wrote that essay. That means that Python has at least 10 years on Haskell. So if you're comparing breaking changes in Haskell today with Python, you need to compare it with Python as it was 10 years ago. If you think the breaking changes are not worth that much pain for you, then don't use Haskell right now. But you shouldn't make that decision without educating yourself about the benefits the language has to offer. For me, it is a small price to pay compared to the benefits I get from Haskell.
Only time will tell, but I predict that companies based solely on Haskell will emerge in a few years dominating their competition because they can write better quality software, iterate faster, with fewer people, more reuse, fewer bugs, and easier maintenance than companies not using Haskell.
>Why Hackage packagers like to specify upper bounds I'm not sure of
Because having working code spontaneously break is very bad. You can't just blindly pretend any future versions of all your dependencies will work without updating your code. APIs change. So you specify the version range your code works with.
>Backwards compatibility either is very hard to achieve in Haskell
Backwards compatibility is the whole point of upper bounds.
>But at least pip, npm and gems are a breeze compared to cabal.
I had constant problems with pip. Because it will blindly install the latest 'foo' dependency even though the API changed and now my app is broken.
You can't just blindly pretend any future versions of all your dependencies will work without updating your code. APIs change.
If you have a risk of arbitrary changes in APIs happening in any minor update, your infrastructure and dependency management are fundamentally flawed and ongoing development will always be difficult. There is nothing unique to Haskell about this, it's just software development 101, separating interface from implementation and all that jazz.
There does seem to be a surprising... I'm not sure how to describe it, maybe a lack of respect?... for this principle in parts of the Haskell community, given the same community's obvious general preference for correctness and being sure everything fits together in a controlled way.
Of course sometimes you want to make major changes to an API and maybe they break backward compatibility, but usually that is a Big Deal in software development, it doesn't normally just get done in some quick point release along with a couple of bug fixes and a security patch.
I had constant problems with pip. Because it will blindly install the latest 'foo' dependency even though the API changed and now my app is broken.
Not that I'm doubting you, but I do find that experience surprising. I work with Python all the time, on different kinds of projects but typically always using pip for package management, and I can't immediately think of a single time that's happened to me. Of course, you can also tell pip to install only within a specific version range if you ever need to.
Perhaps my initial choice was words was poor. Allow me to clarify: I don't think it matters very much what level of version numbering indicates a breaking change, because the main problem is how often those breaking changes happen. I've used libraries that have had a total of zero breaking API changes over a period of years in professional production code written in many other languages. In Haskell, I seem to have run into one ever-changing package or another via Cabal every time I've tried to do something moderately serious in the language. Given the emphasis on relatively small, general and highly reusable parts in Haskell code, I find this surprising, and unless I've been remarkably unlucky in the projects/packages I've tried, it must be a significant drag on development efficiency.
There are some parts of the ecosystem that are incredibly stable (base, containers, parsec) and some aren't (lens). Of course, a lot of new hotness winds up being somewhat experimental, and so we're stuck with bad decisions made early on or wind up faced with the choice of when to cause breakage - in which case "earlier is better" might not be wrong... there have been some efforts to isolate more stable subsets, where that's wanted.
Fair point, though I'd suggest that it's not just general/utilitarian/language support areas like lenses where rapid evolution happens, but also quite a few of the practical, real world things like dealing with comms protocols or user interfaces. This is true to some extent for every mainstream language I know as well, but I think perhaps because Haskell's ecosystem isn't as large and there aren't as many (or sometimes any) ready-made alternatives, you feel the effect more when you run into it. For better or worse, I think this rapid evolution does lend some weight to claims that Haskell isn't ready for use on most mainstream projects yet and to the idea that it's more of an academic's or theoretician's language, even if a few such people happen to work in industrial software development rather than in research now.
I suppose it's a similar situation to what we see in web development. There are always people who want to push the envelope with the latest HTML7/CSS5/SomethingScript technologies, and it's helpful for advancing the state of the art for some people to do that. At the same time, if you're trying to build real web sites for paying clients, what you really need is solid foundations that are as standardised and portable as possible.
"general/utilitarian/language support areas like lenses where rapid evolution happens"
Oh, no question. It just was an example I had off the top of my head of somewhere there has been a lot of instability recently.
"For better or worse, I think this rapid evolution does lend some weight to claims that Haskell isn't ready for use on most mainstream projects yet and to the idea that it's more of an academic's or theoretician's language, even if a few such people happen to work in industrial software development rather than in research now."
To an extent, but I'm not sure I accept the dichotomy. The great thing about Haskell over HTML7/CSS5/SomethingScript is that the people doing experimental stuff are doing it in the same language you're working in, and it will bleed directly back into "things that are stable enough to use" as they get stable. The thing that Haskell-in-industry has over academia is that those experimental innovations scratch itches found by people working in industry. The presence of Haskell in academia means those industrial innovators have more things to reach for. Where it works, it works very well. Because the Haskell community is small, we don't have the large mass of stable libraries that something like Python has, but how much that matters depends both on how stable you need things and on what you need in the first place.
"At the same time, if you're trying to build real web sites for paying clients, what you really need is solid foundations that are as standardised and portable as possible."
Absolutely. If you need things stable, stick to stable packages. There are efforts (stackage, Haskell Platform) toward making this easy. If you need things stable, and you have needs that stable libraries don't meet, and it doesn't make sense for you to get involved in maintaining less stable libraries, then Haskell is probably not a good fit for your project.
That is the most elaborate strawman I have seen in a while. You are now literally saying "haskell is dumb because they do what I think they should do. They should do what they already do instead."
>If you have a risk of arbitrary changes in APIs happening in any minor update
You don't. API changes require changing the major version. Read the PVP.
>There is nothing unique to Haskell about this
Precisely my point.
>Of course, you can also tell pip to install only within a specific version range if you ever need to.
That is the most elaborate strawman I have seen in a while. You are now literally saying "haskell is dumb because they do what I think they should do. They should do what they already do instead."
No, I'm not literally saying any such thing.
There are two points here. One is whether or not your version numbering scheme and package/dependency management tools systematically track breaking changes. The other is how often breaking changes happen.
As far as I'm aware, Haskell doesn't generally have a big problem with the first point. As you say, PVP takes care of that as long as people follow it.
But breaking changes, even if properly acknowledged, seem to happen with amazing (not in a good way) frequency in the Haskell ecosystem. No doubt there's some variation between packages, but I'm another example of someone whose early experiments with the language proved very frustrating because of the constant maintenance headaches with Cabal etc.
If your complaint is "some haskell libraries move too fast for my liking" then you should say that in the first place. If you frame your complaint as "cabal sux and everything else solves this wtf is wrong with you guys" then you have to reasonably expect that is what people will respond to.
My point is that the Haskell community seems (or at least parts of it seem) more tolerant of frequent API changes, including backwards-incompatible ones, than we would normally see in mainstream programming languages. I think this is a drag on development efficiency.
You wrote yourself, "having working code spontaneously break is very bad". You can protect against dependencies breaking by adding upper limits on version numbers for your dependencies, but that doesn't fix the problem, it just conceals it, and it has a cost: every time someone makes a breaking change in a library API, everyone using that library needs to check through what actually changed to see whether it affects their particular situation and then update their dependency management again.
In my original post, I never even mentioned version numbering schemes, and in two different posts now I've reiterated that this isn't what I'm concerned about.
I wasn't having a dig at Cabal for how it supports controlled dependencies either, nor suggesting that other package managers do that job better. That would be like blaming the UI for every bug in software because the UI is where the consequences of bugs are visible to the user. Cabal isn't the problem, the implied need to use certain features of Cabal routinely because otherwise things will break is the problem.
I wasn't even objecting to Haskell libraries developing "too fast for my liking". I think the innovations coming out of the Haskell community are some of the most interesting and promising ideas in the programming world right now, and in general I'm a big fan. I'm just saying that, other things being equal, breaking changes in APIs are undesirable, and that the Haskell ecosystem seems a lot more tolerant of such changes than most, whether because of that desire to move forward quickly or otherwise. That's great in a research language, but I don't think it is helpful for mainstream industrial development, because it does have practical consequences particularly in terms of maintenance overheads.
>You can protect against dependencies breaking by adding upper limits
As every sane system does. So again, why are you acting like this is a haskell thing?
>but that doesn't fix the problem
Yes it does.
>every time someone makes a breaking change in a library API, everyone using that library needs to check through what actually changed to see whether it affects their particular situation
That is the case no matter what. If I write an SSL application in C, I need to pay attention if a new version of openssl comes out. Once again, this isn't even remotely a haskell issue.
>but I don't think it is helpful for mainstream industrial development
You are not required to update to every new version that is released. If you do not want to track the bleeding edge then don't. That's why we have upper bounds.
It's not really cabal but libraries that are broken. They tend to define dependencies too liberally with >=, which obviously breaks things when the depended library makes non-backwards compatible changes.
Also, the primary language implementation itself makes rather large changes sometimes.
Cabal sandboxes have fixed a lot of these problems, usually this results from having installed all packages globally, and then cabal tries to find a satisfying version between all global packages and fails. You can also take Ed Kmett's approach and just do --force --reinstall every time. I prefer the sandboxes though, when something breaks you can always nuke the sandbox and quickly reinstall. I would recommend upgrading to Cabal 1.18.x and try the sandboxes out.
Cabal has been the focus of a ton of work lately, much of which is still ongoing. It has been improving rapidly, notably with sandboxes in 1.18. It still has a long way to go, admittedly, but I find myself having trouble with it way less frequently now.
