Translation to a single language back and forth seems to be really really good.
Although quite impressive, it still suffers from the same problem that most other translator service have if you keep translating the same text between random languages.
I translate the following text from English to various other languages (without going back to english) 6 times and then I went back to English.
The original text is
I wonder if the test you gave me was biased. My belief is that because I'm an elf, some questions are inherently biased. Water dwarfs would not have a problem answering the question: Are unicorns wet? But elves do.
The final English text is
I wonder if the test they gave me was biased? I guess being an elf, there is a bias in the question. I'm sure the water gnomes would have no problem answering the question of whether the unicorn is wet. ? But the elves.
Notice how drarfs became gnomes. The last sentence is not a sentence. Various other problems like a "?" by itself etc.
Translation to a single language back and forth seems to be really really good.
This could be considered a "feature" if used for comedic effect. Anyone who has young children that are into Disney movies should check out this video https://www.youtube.com/watch?v=2bVAoVlFYf0. This woman runs "Let It Go" from the movie Frozen through Google Translate a bunch of times and then sings the result.
You might also be testing the inherent difficulty of lossless translation between many languages.
It would be interesting to see the same tasks with professional translators. I'm guessing some of the errors might still be there at the end, like the gnome one, whereas the last sentence would probably be folded in the previous sentences.
Do you remember which 6 languages you went through before going back to English?
The final English text is I wonder if the test they gave me was biased? I guess being an elf, there is a bias in the question. I'm sure the water gnomes would have no problem answering the question of whether the unicorn is wet. ? But the elves.