Uh - woah! This is incredibly unexpected. I'm really glad people are interested in my database and in Japanese prints!
From a technical perspective there've been a lot of interesting open source projects that I've made to support this site:
https://johnresig.com/projects/ukiyoe/
I've also written a number of papers and given presentations on the work that I've done here (and elsewhere in the digital humanities).
https://johnresig.com/research/
I've since worked on a number of other projects like a database for a consortium of Art History Photo Archives: http://images.pharosartresearch.org/
And I'm building another one now on Japanese Picture Books (from the 19th century and older).
These are all my "spare time" fun hobby projects, my day job is still at Khan Academy as a Frontend Architect. I just enjoy getting to explore other types of applications and problem spaces!
I found this to be an incredible resource, not only for Japanese prints (a relatively new but definitely cool acquaintance) but also for the various open source technologies used.
Especially interesting to me is the https://github.com/mongaku/mongaku library for image searching. Haven't used it in a project yet, but expect I will. Thanks for this (and the other amazing things of immeasurable impact like jQuery you authored)!
This is an amazing site! I haven't really gotten into ukiyo-e but this is so great for exploration and discovery. I think there are a lot of other genres/topics as well that I would love to have a site like this.
I appreciate the work put into making the site light and responsive, even with so many images. And I find the mouseover functionality really neat, too - even if it's not the most practical way to view hundreds (or thousands!) of works at a time, it's completely mesmerizing.
Just to clarify: YUI and ExtJS both came out after jQuery, jQuery came out in 2006, and jQuery is actually more popular now than it was a year ago (or in 2010 - see the figures cited in the blog post).
Not exactly. It's linking to the old SAT practice material that's for the old (current) SAT exam. The new material, for the upcoming 2016 exam, which was just launched today, is available at:
After doing some more poking it appears as if Avisynth (and thus NNEDI3) is Windows-only. Do you happen to know if there are ways to run it in Linux or OSX? Or if there's a comparable set of software for those platforms?
Avisynth should run in Wine, but there is also Vapoursynth[1] (which works natively on OSX & Linux) and a NNEDI3 port[2] for it. After getting both of them up and running, a script like this[3] ran with the vspipe program that comes with Vapoursynth should do the trick. It's a bit cumbersome since Avisynth and Vapoursynth are primarily intended for processing video, not images, but it gets the job done in absence of a dedicated NNEDI3 resizing tool. I'm actually using this exact setup at work myself when I need to do any image upscaling.
NNEDI3 is fantastic - thank you for providing a link and some samples!
You're absolutely right that I shouldn't have said "normal". I update the post to clarify that this was using "OSX Preview". I did some hunting but didn't find any obvious pointers as to which algorithm they're using. If anyone knows offhand I'll be happy to include it!
Yeah, why they're going through all that hardware effort, I dunno. Simpler developer workflow I guess. Would be interesting to do a cost/benefit vs. just using a Linux stack.
The 'hardware effort' is to get dramatically improved processing time by using the GPU since they're trying to do it on a much larger scale.
I have/continue to use imagemagick and similar software-based solutions and they're pretty slow for multi-MB images (but most servers don't have good GPUs so it's the only solution unless you're building custom racks as imgix does).
Yeah, I'm not super sure about the dramatically improved processing time. Especially compared to a SIMD-optimized scaler. You have to spend some time sending the image to the GPU and reading it back too.
Especially if you set imagemagick to use the much worse scaler that imgix uses, I imagine it'd be pretty fast.
On the other hand, if you replaced imgix's stack with the high quality scalers from mpv (written as OpenGL pixel shaders), and then compared to expensive CPU scalers, I would expect a GPU solution to be a win.
Note that imgix also has to recompress the image as PNG or JPEG at the end. This has to be done on the CPU and is probably more resource intensive than any of the scaling.
You can upload 100s of MBs of texture data to a GPU in milliseconds. Sending and receiving from GPU doesn't actually take that long in comparison to the time it takes to process a multi-MB file in software.
Well, OSX Preview seems to be doing something interesting as I can't seem to find an exact match with some quick attempts, but whatever method they use it looks rather similar to Lanczos scaling.
Yeah, on second thought, after seeing the low noise reduction result again, I suspect that may be an even better result for what I'm looking to achieve. Many of the details in his rope are preserved and the calligraphy appears to be in better shape.
One thing I should note is that when looking at prints (at least for when it comes to technical analysis) being able to see accurate representations of the lines is far more important than the uniformity of colored regions. Color is almost always at the whim of the printer on any given day, whereas the black lines (from the keyblock) should always remain the same. Granted you're going to have issues either way (using this tool or doing normal scaling) as the source material is inherently compromised.
Although it's not clear what scenes exactly the upscaler was trained up, I suspect that it's currently best suited towards scenes that have lots of large bold lines and not lots of tiny details.
I imagine it was trained on 'typical' anime-style art of black/bold character outlines and mostly flat colors.
A common solution to resizing anime characters is to create a colored vector of an image. The differences between these vectors and the original stills are minimal and usually 'satisfactory'. There is an entire scene of people who create these vectors and another scene of people who use the vectors to create wallpapers and other graphics. [0] Waifu2x can help replace the need to vector these images by increasing the quality of upscaling them.
This is the prevalent 'style' for anime - at least from the past 8-10 years or so. There are a few outliers and I imagine Waifu2x would work poorly on them. For example, I do not see it working well on a still from "The Garden of Words". [1]
One potential point of improvement is training the neural network in prints with Japanese text, this seems to be the weakest point in using the Anime trained one.
Yeah - the background looking like crumpled-up wrapping paper is definitely not ideal. I suspect that it's having trouble with mostly-uniform areas of color that have slight variations. It appears to be extrapolating and creating these larger effects.
Have you tried running the original through a high pass filter (to get the textures) and applying it over the vectorized version? It might work for the background texture, though it would probably suck for the text.
There are well-known techniques to clean JPEG artifacts, but they need the tables from inside the JPEG to work best.
Since JPEG uses an 8x8 block transform, you can find artifacts by shifting the image a few pixels over and looking for how the transformed block changes, basically.
Also, using a better chroma upscale can help for small images. libjpeg just uses nearest-neighbor (no "real" resizing) and hardly anyone notices, but it helps with lines and edges.
I have not - although that's an interesting idea, thank you! Relative to my other projects this is a very low-priority exploration. I was very interested to see if there could be a "cheap win" for this particular sub-problem that I will be dealing with, should I get around to digitizing these books.
From a technical perspective there've been a lot of interesting open source projects that I've made to support this site: https://johnresig.com/projects/ukiyoe/
I've also written a number of papers and given presentations on the work that I've done here (and elsewhere in the digital humanities). https://johnresig.com/research/
This is a talk that I gave in 2014 about the construction of the site and my reasons for building it: https://codeascraft.com/speakers/john-resig-analyzing-japane...
I've since worked on a number of other projects like a database for a consortium of Art History Photo Archives: http://images.pharosartresearch.org/
And I'm building another one now on Japanese Picture Books (from the 19th century and older).
These are all my "spare time" fun hobby projects, my day job is still at Khan Academy as a Frontend Architect. I just enjoy getting to explore other types of applications and problem spaces!
Happy to answer any questions anyone might have!