Sounds a bit like running out of memory, followed by the OOM killer going after the process... though I'm not sure the OOM killer would act that quickly. Do you get any interesting messages in `dmesg`?
You're spot on - dmesg shows it's OOMing, and pretty fast too. Turns out that it happens even if I just open the web page, yet alone start trying to remove the background of an image, so I'd guess it's something to do with loading whatever model is being used.