To push back on this, I consider pandas/numpy so crucial to Python as a whole they are effectively stdlib to me. I wouldn't blink at this because it would happen sooner or later.

Unless is was absolutely critical the server have as small as a footprint as humanly possible and it was absolutely guaranteed there would never need to be included in the future of course. However, that first constraint is the main one.