Hacker Newsnew | past | comments | ask | show | jobs | submitlogin
Yaegi – Yet Another Go Interpreter (containo.us)
220 points by emilevauge on July 24, 2019 | hide | past | favorite | 72 comments


Very cool idea. Doesn't seem ready for prime time yet -- tried running it on a small Go program to score boggle boards, and got errors in the interpreter.

Playing around, it seems to have problems assigning int literals to int8 variables, and it seems to sometimes have problems re-assigning pointer values.


Bench-marking a fat native call, rather than a purely-interpreted function, is a bit bait-and-switch.


They do explain it very clearly though, and give a reason why that is what they are interested in for business purposes. Benchmark and optimise for what you're running, right? Better than a Mandelbrot?


They explain it, but it's still deceptive.

I would have expected a honest benchmark result. Which will be much worse than 6%.

Which is totally fine! an interpreter will of course be much slower. But this seems like the wrong kind of marketing by omission to me.


Yeah, but they're just benchmarking a wrapper around gzip. It looks to be around 300 LOC and does very little.. It's a bit deceptive. A large table of benchmarks across different domains and workloads would be more honest.


Programming in a language lacking a repl is always uncomfortable. I always wind up making a 'scratch.{ext}' file and running it to test things (scratch.* is in my global gitignore). Happy an interpreter+repl now exists for Go.


You can do similar things by writing tests as you go, plus then you can keep the tests if they seem useful.


In my experience, the transition from running the code to see if it works to running the tests to see if it works, is a major milestone in producing high quality software.


For one meaning of "test", sort of, but repls are useful for far more than "do outputs match what I expect". Test writing is not very good for exploratory programming.


Tests are perfect for "do outputs match what I expect". If you don't know what you are expecting yet just to `t.Error("")`. When you are done fiddling around and want to solidify what you expect add the `if foo != expectation`.


To add to this, editor commands to quickly switch to your test file, auto generate test boilerplate and run the currently highlighted test are super helpful to speed up this loop.


This sounds really helpful, can you provide any links to read more about setting these helpers up?


It completely changed the way I write Go. This is editor specific though.

I use Vim for programming Go and have a few custom scripts built on top of https://github.com/fatih/vim-go.


I actually did a pair programming session with JBrains on TDD to enhance my skill. What he explained to me was exactly this. You need to test something, for example you want to test how regexp works. You start writing tests with your expectations and go on. After you are satisfied, you actually keep this in a package for reference and documentation on works the library.


I disagree with this to some extent. Unit tests are a pretty good poor-mans REPL in languages that don't have REPLs. For instance, I've seen people write fake unit tests to do things like query a database for analysis purposes; it was just the fastest way they could come up with to do the exploratory work. What makes a unit test a good poor-mans REPL in a language like java with an IDE is:

* Java IDEs, especially eclipse, are amazing at fast, incremental compilation. You make one change, and then only recompile what you need to.

* Java has no relatively expensive link stage like C++, C, even C# to a lesser extent, when running tests from an IDE. Your IDE can run your unit tests and also be smart enough to only load the classes it needs to run the test.

Most dynamic languages already have a REPL, so the technique of using unit tests for exploratory work isn't necessary, but I think it's very useful for Java at least.


Unit tests are a pretty good poor-mans REPL in languages that don't have REPLs.

Java IDEs, especially eclipse, are amazing at fast, incremental compilation. You make one change, and then only recompile what you need to.

I always found this lugubrious compared to the experience in many Smalltalks. The Java vs. Smalltalk USENET flame wars in the 90's were partly motivated by the downgrade in developer UX caused by the taking of the Smalltalk runtime model and deliberately crippling it.

Granted, Java has made amazing progress since then, as have IDEs generally. Is there a non-Smalltalk environment that lets one "code in the debugger" to the same extent as Smalltalk? I know of Clojure and the kind of experience shown with LightTable. (To elaborate, to "code in the debugger" in Smalltalk is very nimble and powerful. It's kind of like the turn-based version of LightTable but with pausing in the debugger and manual tinkering/time-travel with the context stack.)


The JVM is less dynamic in its ability to reload code, but the payoff is that java code can be 10-100x faster than smalltalk code. I personally think the performance gain is worth it. Hotspot has some limited support for hot code reloading, which is a pre-requisite to being able to "code in your debugger", but it's limited enough that I don't bother with it. Alternatively, you can code in a dynamic language on the JVM such as JRuby or Clojure and have full support for "coding in the debugger". These languages are also a lot slower than Java.


We are talking Go here, which has a fast compiler. The IDE integration isn't always as good as Java, though.


Go has a fast compiler but to my knowledge has zero support for incremental compilation. It has to recompile the entire project, transitively with all its dependencies, before you can run your code modification. I haven't code in go, but IMO this does not scale well as projects grow in size.


I find GoLand to be quite good.


Seconded, GoLand isn't just quite good, it is phenomenal. If you aren't running it, you need to have a compelling reason.

https://www.jetbrains.com/go/


This tends to be more important in dynamic languages where "the outputs" are often hilariously far from what is reasonable to expect. At least that's where I find I use a REPL in Python--load in some poorly-documented 3rd party library to see what the return type of a function is (for some given inputs) and what properties are available on that object (since the "type" isn't much good given the aforementioned poor state of the docs). In Go, however, you get all the information you need from the type and godoc.org.


This tends to be more important in dynamic languages where "the outputs" are often hilariously far from what is reasonable to expect.

Most of my experience with dynamic language programming is far from what you describe above. About the only place where I've encountered the like is with Javascript.

https://archive.org/details/wat_destroyallsoftware

I once experienced "magic" jumps in the debugger in Smalltalk, but that was because of some liberties taken with custom methods implemented in C, or because of missing_method style metaprogramming. (#doesNotUndedrstand: in Smalltalk)


This experience is commonplace in Python. Can’t speak so much for other languages.


I use the go playground for this. Only problem I have run into is lack of sane time values.


Me too but if I had a repl I’d probably prefer that.


I think this is a matter of personal preference. At work my primary language is Python and I rarely use the REPL, and when I do, it's a regrettable experience--can't use up arrow to get previous command, can't use arrow keys to move cursor, entering tabs after the `. . .` makes it hard to figure out how many tabs to enter, etc. And of course you can't visualize the whole "program" at a glance like you can in a text editor. Not sure if Python's REPL is just subpar among repls or if I just don't understand how to leverage REPLs for my benefit, but `$EDITOR /tmp/foo.py` just works so much better for me.


> when I do, it's a regrettable experience--can't use up arrow to get previous command, can't use arrow keys to move cursor

Dunno what's wrong with your readline but what you describe is default Python behavior. Try installing ipython, you'll get all that and more (such as syntax highlighting, multi-line editing, and much more).


Yeah, I'm not sure why, but this happens in VS Code's integrated terminal, which also has other readline-related issues (alt+backspace deletes to the previous space instead of deleting the previous word, for example). In any case, I still don't use the REPL except when I need to dynamically inspect some value (which isn't necessary in Go, since it's statically typed).


Windows is crippled somewhat on that front, but there are workarounds. Often easier and way more functional to just install a wrapper like ptpython, bpython, ipython, etc.


I'm using a Mac.


Sounds like a broken terminal configuration. Try running python at the standard terminal and see what it does.


Yeah, like I said, it works at a standard terminal, not in the VS Code integrated terminal.


I have a keyboard shortcut Ctrl+Alt+i mapped to this: alacritty -e /home/mbarkhau/bin/ipy

and /home/mbarkhau/bin/ipy is this

/home/mbarkhau/miniconda3/envs/py37/bin/ipython -i -c import itertools as it; from statistics import ;import functools as ft;import operator as op;import io;import pathlib as pl;import pandas as pd;import numpy as np;import typing as typ;from math import ;import enum;import re;import sys;import os;import ujson as json;import collections;import random;import decimal;import fractions;import time;import datetime as dt;"

I must use it dozens of times a day.


For about a year now I've used ipython as my day-to-day python REPL. I'll type a bit in ipython then save that session to a file, tidy up that file, and voila, a full script, now I can repeat that session at will.

Ipython has interactive tab completion (including for imports and locally defined symbols), syntax highlighting, (sane) auto indenting, all in the REPL. It's the foundation of jupyter notebooks.


Bpython is much nicer experience than the native repl, but I still really only use it to call help(), __slots__(), etc and test minor things like forgotten syntaxes; the lack of automatic reloading just makes it to irritating to work on the script itself that way


I’ve often had similar feelings to you.

Something that blew my mind recently was using a Lisp repl which was integrated with my text editor.

I just wrote out the code I wanted to toy with in my text editor buffer, and then highlighted it and hit a key command to evaluate it at the repl without even needing to switch over to the repl window.

It would be so cool if we had great support for something similar in non Lisp languages.


I prefer a scratch file and send-to-repl. Common in e.f. F#


Ah, this famous REPL driven development. After which we are left with highly coupled codebase that has almost no tests. After doing clojure for 6+ years I forced myself at some point to disable REPL, it allows one to go fast in short term and creates way too much debt long term.

Nowadays I find repl driven approach to be not so sustainable for mid to large sizes of teams.

It is fun on small/personal projects though.


Why is REPL a cause of a coupled codebase and lack of tests?

Surely those are programming practices that don't need to go hand-in-hand.


Around a ~6MB interpreter for those curious, not bad.


Generally the Go runtime (garbage collector, scheduler, allocator, etc) alone is 2-5MB, for additional context.


a bit less:

    $ cat > t.go
    package main
    func main() {}
    $ go build t.go
    $ ls -l t
    -rwxr-xr-x  1 f2f  staff  1101432 24 Jul 23:19 t


And you can drop that down to ~775k by using `strip`.

    [rjp@hostname tmp]$ ls -l gosize 
    -rwxr-xr-x 1 rjp rjp 1126905 Jul 25 07:00 gosize 
    [rjp@hostname tmp]$ strip gosize
    [rjp@hostname tmp]$ ls -l gosize 
    -rwxr-xr-x 1 rjp rjp 793704 Jul 25 07:00 gosize
    [rjp@hostname tmp]$ go version
    go version go1.12.7 linux/amd64


Not sure, but is it possible that the runtime (or parts thereof) isn’t linked in in that example?


From the "another" archives, I remember this one, that looks abandoned:

https://news.ycombinator.com/item?id=15622894

https://neugram.io/

https://github.com/neugram/ng


It seems to be from the creators of Traefik, which is another great tool.

https://traefik.io/


I'd be interested in an opinion of the dev on what was the hardest part to build. I could imagine reflection features or the incremental type checking would be hard to do. But I always see these projects (like cling/ROOT from cern) and think that some subproblems are really hard to do if done incrementally with only local knowledge.


Why would I need a Go Interpreter? What advantage does it provide over using a precompiled binary?


The most obvious use for this is to distribute an application with a scripting interface.

Users can write simple scripts which change behaviour, or react to events, without needing to learn a toy-language, or even necessarily have a compiler setup.


There are some circumstances where an interpreter is useful: For example, it should be possible to make better golang repls with an interpreter than with a binary.

In a "scripting" scenario, you can load some golang code at runtime without needing to bring the full golang compiler toolchain along, which keeps the nice "single binary" deploys of golang. It may be possible to sandbox scripts as well (though I haven't looked into how that could work with this implementation), which could reduce the risk of running scripts (less likely to shoot yourself in the foot. Probably not a security boundary -just look at how much work goes into safe JS in browsers to have that).


Sandboxing is possible but only if it brings it's own standard library (I think... I'm not familiar with how go implements io at it's roots)


I'm working on an approach for Golang sandboxing which works through whitelisting imports, and munging all references, casting operations, and function calls, which lets one whitelist those as well. I would disallow all io and network access.


Seems like a wasm interpreter with WASI might be a better approach.


I've also given that some thought as well. Actually, all of the above could be combined.


No IO? As in at all?

xor eax, eax here we come.


I would be providing an API and handling IO for the client. I'm not disallowing all IO and network access. I'm restricting it to going through my API.


Could you not just hijack the system calls/c runtime? Then you can still do "safe" I/O without a specific API (Or are you rewriting the stdlib on top of the API? I've never actually used go so I don't really know anything about how it does or doesn't work usually)


I didn't see any while reading through the blog post...


Does it have any pretty printing capabilities? Repo doesn't seem to...


Now, you can play with it in the Go Playground https://play.golang.org/p/P--lUj4Vbax


Wasn't clear to me: will it run say a serial port without additional steps?


That's pretty cool. Definitely going to use this on my projects.


Like runhaskell for Go?


YAGNI?


Please add generics at least on Yaegi! :P



is gomacro better for repl on golang comparing to yaegi? both look well


gophernotes is sort of like a repl under jupyter


I love you


Yaeji is going to be very happy that an interpreter was named after her! https://soundcloud.com/kraejiyaeji


I was thinking the same thing :)




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: