This is the basis of our work with Facebook which can capture a light field using 6 or 24 cameras. We run a Job on cloud and it comes back as a 3D scene ORBX asset you can load into Octane for Nuke, Unity etc.

https://twitter.com/otoy/status/894972675679965184

Can anyone do it? Is the software to do that open source?

You need the camera. The software to process is hosted by us on the cloud, I don't if FB can oss it. At least the ORBX file is open and loads in Unity Nuke and AE

Well, Lytro is still in business and they are doing precisely that. https://www.lytro.com/immerge

It's a true plenoptic (light-field) camera.

Well, does that scale well? how much does it cost? Can a camgirl use it to "stream" (not live) her shows, for example?