I've been doing the Coursera ML course the past few weeks(started on the 22th of August). I can only give it my highest recommendation, it really is a great course. I did computer science in university(2.5 years) and have a decent grasp of calculus and linear algebra. I've also been programming for 9 or so years. I've found the course to be surprisingly simple and intuitive, but this probably varies with how comfortable you are with the math in the course.
>Coursera ML course the past few weeks(started on the 22th of August). I can only give it my highest recommendation.
I'm taking the edX series of courses on Machine Learning with Apache Spark. It is a pretty good class and covers linear algebra and the basic ETL workflow, plus selection of models, model parameters tweaking parameters etc., study cases involving user ad clicks prediction and PCA analysis of neural synapse data of jellyfishes.
However, my biggest fear is that re-learning different matrix manipulations, walk-through of logistics regression, PCA and SVM will be similar to learning Spanish in high school without immersion, learning guitar scales without improvising with it, learning mappings of different Madden key-combo's; I remember learning all of these mathematical syntactical operations in high school, and have an eerie feeling of once grasping these concepts concretely once and yet only knowing it vaguely. It is nice to re-learn these things, like refreshing myself with a rolodex of Spanish verb conjugation, only to go back to the CRUD work I deal with on a daily basis and fade back to oblivion.
Not sure about the Coursera class, the structure of the edX courses are presented in Python Juypter notebooks where you fill in the code snippets. It is fun and addictive to solve each exercise as a mini-puzzle; but not sure how much it'll stick vs. if one had to take an concrete problem and take its pieces and puzzles from start to finish, without the hands-holding.
I think perhaps Cousera courses have "capstone" projects - curious if anyone have had experience doing one? In lieu of one on the edX course, I think I plan to grok some papers with some large genomic dataset and/or financial time-series and try to replicate their result.
Yeah this is definitely a fear of mine, it does feel like you can get through the course and mostly understand the concept without getting the tools to fully apply the learnings. Like drothlis points out the Coursera course it very hand-holdey too. For the exercises you are given a set of Matlab/Octave files and your objective is to fill in a few of them with the correct implementation of specific steps. It doesn't feel like you ever get to perform the full task, I plan to redo some of the exercises with Python after I finishes the course.
I'm also doing this course right now. Having MSc degree in computer science helps a lot, although I got my diploma as long as 10 years ago, but still some math sticked with me.
My biggest regret is that my day job as developer does not leave me with much time for more Octave programming. Of course I do excercises, but that's not enough to be good ML developer.
However, I think that just having a lot of sample code would be helpfull if I ever make ML my full time job. Especially considering that the excercises themselves are pretty interesting (handwriting recognition, image compression, etc).
I totally agree. I took that course as well, and thought his explanations were amazing, but because I'm just a rails/angular developer and do not have the required math background, I definitely got lost and could not keep up the pace. I'm currently taking math now so I will be able to participate in something like this in the future.
I'm taking this course as well, and I'm really pleased with how practical it is, and how much "here's how to figure out how to do better" advice and techniques there are.
Really, it's a great model for teaching technical subjects in general.
Did they get you with the email that the first assignment was due as well?! They sure got me. :D I had been meaning to do the course for quite a while and had started it once before, but now with the deadlines I am finally following along. :)
I know enough about machine learning (and scikit-learn) to make simple models with production deployments. The results are pretty good, but they come almost exclusively through trial and error—testing different algorithms and approaches until something works.
Unfortunately, I really don't understand the underlying theory. Every time I start diving into it, I realize it'll probably take a huge amount of time to understand (particularly because I'll likely have to do an intensive refresher on linear algebra and multivariable calculus).
How dangerous is it to be practicing machine learning without deeply understanding the theory? Is it worth the substantial time investment?
Not dangerous, in my experience. It doesn't take deep theoretical knowledge to provide value (i.e. value to a customer or to a business) through machine learning. Assuming here that one knows how to cross validate, check for overfitting, etc., and not shoot themselves in the foot.
EDIT: Note that it's still EXCEPTIONALLY difficult to provide true, lasting value to a given organization with this stuff and takes years of experience (note I didn't mention deep knowledge and experience with the latest techniques aboard the hype train).
I think the main skill is getting an intuitive grasp of expressing a problem in machine-learning language, which usually means "I have this data set, what are the most salient features I should be using to train my algorithm to tell me things". The main challenge to solving a machine learning problem in my experience is the curse of dimensionality.
Aside from that it's still trial and error. As a sibling pointed out the best you can do is get a good grounding in stats and sampling theory so that you ensure that you're not overfitting your data.
There is no rule-book for what kind of classifier works well on your problem, aside from reading the literature and seeing how well other people got on. There are exceptions, for instance if you're doing image classification then nowadays it's convnet or bust. But still you're going on instinct rather than "If you have problem <X> you should solve it with machine learning algorithm <Y>".
As with all these sorts of things, knowing the theory is useful if you want to start implementing your own classifiers from scratch, but nowadays it's approaching wheel-inventing territory. Machine learning libraries are mature, powerful and increasingly easy to use (like sklearn). Some of the libraries like Tensorflow are specifically geared so that you don't need to know about the implementation, you just read a description of the CNN and you can express it in a few lines of Python.
Otherwise using them is still pot luck:
- How many training samples will be enough for your problem to avoid overfitting/poor results? You won't know this until you try.
- What features are the best for your problem? There are algorithms which will try to learn for you, but in most cases that's a niche application and will require a lot of twiddling.
- You still (usually) have do a grid-search to find the optimal parameters for a particular method.
- Do you care which method you use, say an SVM vs Random Forest, if you get 90% accuracy consistently?
- Be mindful that for many problems in machine learning, 80% accuracy is considered exceptionally good. You shouldn't expect perfect performance so sooner or later you have to say good is good enough.
FWIW you can learn how these algorithms work without a very deep understanding of calculus or linear algebra. I did it, although I'm trying to catch up on the math now. And really for the average ML user, you don't need to learn the exact math or calculations, just a general understanding of what it's doing.
Above all have a good understanding of over fitting and best practices for test and validation sets. Then you can use algorithms as black boxes and at least know you aren't doing any harm.
Nothing to get excited about, especially since you aren't yet invested. However, should the odds indeed be high that ML will become automated by 2025 then another field choice might be wiser.
morgante: "Here's a different question: is it worth it?"
Only you can be the judge of that.
A better question is "What is the opportunity cost?". Is there something else that one could do with that same time that would be more valuable/useful?
I suspect that most persons have a number of things that would be more valuable to them than learning ML to some vague degree and undefined purpose. One might take an economics course, exercise more and pay more attention to one's diet to get into better shape, get more sleep, sell the boat, read Nicholas Taleb's second and third books, etc.
The part about the Coursera course's steep learning curve feels so familiar. I'm doing the course as well, but it's pretty tough for me. I suck at maths and I don't know Octave or Python. It's a very good course though, I still try to watch some lessons every week, but I'm way behind schedule.
I'm impressed by the author's ability to recognize his own failure in completing a course, because for me it often just keeps dragging on while I still think "some day I will finish it". I can't accept the fact that I just failed it. Sometimes I wish I was still in university, being able to dedicate 40 hours per week to just learning this kind of stuff and having teachers around.
It is very hard to complete a course without having the prerequisites. If you took some courses in math and Python you would probably be able to pass it with much more ease.
For Calculus check Coursera - the Ohio State course if you really don't know absolutelly anything, then do the Penn State course(s) Calculus in a Single Variable
For Linear Algebra check Gilbert Strang's course on MIT OCW
For Probability/Statistics either John Tsitsiklis' course on MIT OCW (or even better on edX when it re-runs) or Harvard's Statistics 110: Probability course on youtube
The level of multivar calculus you'll need is really minimal, just google these terms "partial derivative", "total derivative", "differentials and integrals commute" ("Symmetry of second derivatives", "Fubini's theorem") or take the vector calculus course from MIT OCW
Indeed you don't need to know much vector calculus for machine learning. Stokes theorem and the like are mainly relevant for physics, but the Jacobian and Hessian are good to know.
"Linear Algebra from Foundations to Frontiers" on EdX[1] is an introductory course in linear algebra with a manageable learning curve. There are many calculus courses on Coursera and EdX.
Machine learning is an interesting case, in that the gap between theory and praxis is quite wide.
You can basically create, run and evaluate a model within 10 minutes (e.g. with scikit-learn or Tensorflow). It would even be an acceptable model that might solve your company's problem.
But you have to plan in a few years of deliberate practice to work through statistical learning theory and its prerequisites to get an overview and to be able contribute to the scientific or open source efforts in that field.
And in my experience the opposite applies as well. Having a PhD in a field providing the necessary background in theory doesn't necessarily imply the skills and experience required to produce useful models.
That's a really good question. The best people I've seen so far have had a willingness to rigorously align their models with reality, plus the knowledge and/or experience to know what to check. Verifying assumptions, picking evaluation metrics appropriate for the problem, checking for model interpretability, checking that model decisions are sane, and so on.
I disagree wrt contributing to open source. There's still a lot of general software engineering that goes into these systems. I've found d it pretty easy to contribute implementations since the core of many ideas are actually quite simple. Of course they're only simple in hindsight, but implementation is just that.
I see everyone talking about Andrew Ng's ML course and having some difficulty with octave. Why is no one trying the new ML with Python series by UW on Coursera? It seems pretty well structured.
Honestly Octave/Matlab isn't a problem I've found. If you are vaguely familiar with any programming languages it shouldn't be a problem. To me Matlab is just another tool to express my thoughts just like any other programming language. Sure it as has some things to learn, but it's not like it's a whole new paradigm or anything. Personally I struggled more with getting the linear algebra right than the syntax of Matlab
> "It’s possible to get a good machine learning teacher for around 50 USD per hour"
How/where did you find a teacher like this? Finding a machine learning subject expert is already pretty difficult; knowing how to find one who is also a good teacher would be a useful skill in its own right.
The article is helpful for all of us who are struggling with ML.
Since your story is a success, no matter how you personally feel about it, can you tell us how many hours per week in that year did you dedicate your time to ML? A rough estimate will be good.
What are your plans for future? Find a ML-only job?
Hmm.. hard to say, but in 2016 I've probably spent around 5 hours per week on average.
Most of the times I've struggled with ML has been related to the fact that I've been trying courses/tasks that have been above my level. Once I find content which is tailored to my level, ML isn't that hard after all.
I definitely want to do more ML in the future, one way or the other.
I'm testing my knowledge a bit, but this looks like it has 2-3 layers? One input, one hidden, and one output? (nn.py) When might you add additional layers to this particular problem?
Would turning the results into a probability be useful? (e.g. "how likely are we to get a customer based on this description?") Your staff could generate test data for that.
That neural network is just an experiment I did the other day, and it doesn't work properly. I'm going to fix it when I have time. Regarding the results: I don't know how they'll be, that's why I'm experimenting with it ;)
I've attempted to do Coursera's ML course, but I am not very good at learning from videos - I much prefer reading a text book and then attempting the problems in my own time.
Are there any recommended text books that one can read from start to finish in order to get a similar introduction to ML?
If you already know partial derivatives, gradient descent algorithms, and some linear algebra, it's easy. The assignments are not challenging either, even compared to other coursera classes. But for people that don't have this background, it can be hard.
I think the problem with Coursera is that they try to accommodate as many students as possible, which means the classes aren't very deep. I don't think this should be the role of a ML class to teach about partial derivatives or matrix factorization for instance. It's great that the class is accessible to many people, but sometimes you feel that you just scratch the surface, and that you're not at the level of university students that went through more challenging classes.
As a comparison, I did the labs of 6.828 and 6.824 (OS and distributed systems) on MIT opencourseware and it was much harder and much more rewarding than any coursera class. And I did only the labs, there is much more material to cover to do it seriously (quizzes, lot of articles to read...). Kudos to MIT students! (I wonder how many such classes they take during one semester). Too bad opencourseware classes aren't always properly set up for outside students. There may be some material missing, no forum to ask questions, video quality can be very bad and so on...
Yeah, when I did the Coursera course I supplemented it with the CS229 materials to get more of the theory behind everything, which worked great. If you want more math rather than less, it's the way to go.
But there seem to be a few famous ones (such as cs231n) missing. Any current stanford students want to chime in on what else is hot nowadays? In addition what's a recommended course sequence that will take you from cs229 to a bleeding edge deep learning expert?
Assuming you're taking about Andrew Ng's Stanford ML course, I had the same impression until I realized that the actual Stanford ML course is a lot harder and theory-heavy. Basically, the Coursera thing is a more practical, watered-down (less math) version of the real Stanford course (which btw you can still find on Youtube, though it may be a bit dated)...
I have a bachelors degree in economics, so some university math with calculus and linear algebra. The reason I found it hard was because I had to learn Octave while also understanding the algorithms and general ml concepts.
That being said, I think it's a great course, and would highly recommend it to anyone who thinks they're up for it. But there are other courses that provides an easier gateway into ML.
> The reason I found it hard was because I had to learn Octave while also understanding the algorithms and general ml concepts.
I had the same experience, even after using Matlab some in grad school. I wish I had seen this before trying Ng's course (all of the exercises in Python):
I love the idea of using data to train a machine to recognize patterns and classify things. It's kind of a bummer though that the application is to help sales people do their job.
Can anyone tell us how hands-on is Stanford's CS224D please? I found Andrew Ng's course too theoretical and not so practical.
Right now I'm almost halfway through Udacity's Intro to Data Science and it's great, so I'm deciding if I should take Udacity's Deep Learning or CS224D
There was a great ML presentation at RustConf yesterday where a couple of guys broke down building a classifier using Python and Rust. It really demystified the process.
