Microsoft has a blog with more details [0] but the Axios article does a good job of summarizing the main points. The full report is also available to read [1].
If it uses a hash of an image, does that mean an edited image (eg cropped or resized) wouldn't be detected? Or is there a way to extend a single hash to multiple image variants? I imagine the answer is no, but I would also like to be optimistic and think even detecting the original version would stop a lot of NCII sharing (ie it's not a perfect solution, but still helpful).
Here's an article on how a few different image hashing algorithms work https://content-blockchain.org/research/testing-different-im.... I think in general they still function with resizing (and a few other image manipulations), but I don't think they handle cropping.
Cropping does seem like a harder problem, in the sense that there's no "clever" (high compute efficiency) solution available. The only thing that comes to mind is the brute-force approach of comparing normalized small tiles instead of whole images, which would involve exponentially more compute effort. Interestingly, the article you cite mentions rotation and skewing, which make both clever and brute-force approaches even more expensive and/or less effective. Certainly seems like a target-rich environment for research.
> does that mean an edited image (eg cropped or resized) wouldn't be detected?
Depends on the hash, and I also wouldn't be surprised if "fingerprint" turned out to be a more accurate term. This is a PR document, after all. ;) You can normalize things like resolution and color balance, and create a hash/fingerprint that's a concatenation of hashes for portions of the image, which would allow some degree of useful comparison despite many kinds of changes. Similar techniques have been used to detect copyright violations as well, so there are technological precedents. The real question is implementation quality.
While this article is on sound fingerprinting it does talk about image fingerprinting, the algorithm finds "significant" points in the image and as such can still work because it looks at their positions respective to each other, which will still hold after scaling or rotating. And cropping is likely to remove content lacking such features, so it can handle that as well.
Yup, PDQ (https://github.com/facebook/ThreatExchange/tree/main/pdq) and MD5- https://stopncii.org/faq/. MD5 is a cryptographic hash, which means that even a single bit changed gives you an entirely different hash. This usually precludes any attacks to try and deliberately generate collisions, but also means it's harder to match “benign” changes - for example, many platforms re-encode videos as part of the upload process (reducing resolution, changing formats, etc). However, many platforms skip re-encoding, so MD5 can have better results than you might think at first glance. The more bits you take from the original content, the harder you can make it to bypass, but the more you might have to worry about capturing enough that you might compromise the privacy of the original submitter. For the program, StopNCII picked a set of tradeoffs between those two tensions, and are keeping a close eye on how effective it is, and will iterate if need be.
Quite an interesting review, with lots of useful learnings for practitioners (in my opinion). I also liked the use of the term "graphical literacy" in the abstract, which is something I've seen cause challenges first hand (though I never thought to use that term):
"Effectively designed data visualizations allow viewers to use their powerful visual systems to understand patterns in data across science, education, health, and public policy. But ineffectively designed visualizations can cause confusion, misunderstanding, or even distrust—especially among viewers with low graphical literacy."
In case it's helpful, here's the first paragraph from the post:
"The original KiCad domain name (kicad-pcb.org) was recently sold to an unnamed third party that is not affiliated with the KiCad Project or members of the KiCad Development Team. This sale was unexpected and may pose a risk to KiCad users. The new owners may simply post advertisements or (worst-case scenario) they may host malicious versions of the KiCad software for download."
> they may host malicious versions of the KiCad software for download
Imagine someone using this as an attack vector to smuggle in exfiltration appliances in the form of hardware circuits in the KiCad software itself, so that the generated PCB files contain spyware in hardware form.
That would be a first of its kind: Supply chain attack at hardware level.
Extremely difficult to achieve undetected. Firstly just successfully smuggling a working design for spyware into the PCB design would require an impressive feat of CAD engineering, then there's no way to achieve this just with copper traces, you would need to add new components to the board. Both changes in the traces and new components would be extremely obvious in all but the most automated workflows (getting a PCB manufactured is a very manual process). It's typical to inspect the output gerber files manually for any errors, as well as for PCB manufacturers to offer a preview of their interpretation of the gerber. Similarly with the BOM, which has a different flow with a lot of human touch. Also, the techniques which would allow hiding the components within the PCB itself (which do exist) are extremely rare. The only places with the slightest chance of this working (massive organisations with a huge amount of siloing) aren't using KiCAD as a tool.
You're correct of course. It's wildly infeasible, prone to rapid detection, and there's no plausible profit to sneaking subtle backdoors into random KiCAD projects. Now let's brainstorm possible vectors anyway...
You could manipulate PCB traces in such a way as to leak data over RF? No topological change to the circuit and very hard to pin down.
Perhaps there's some chip that has some functionality enabled by a logic input. The designer intended it to be off but KiCad tied it high or low or open to turn it on instead. That functionality might enable a serial interface or reading or writing some sensitive memory or whatever that might become a vulnerability.
I imagine it would be much more likely to have a "backdoor"-ed version of kicad that would phone in with intellectual property rather than PCB spyware. Making a PCB is too much like writing assembly for it to go unnoticed...
Exfiltrating KiCAD project files and manually designing a subtly backdoored version is a much more likely attack. An automatically inserted backdoor has a high risk of detection, dragging the entire plot out into the open. Stealthily phoning home is harder to detect and most small operations that would use KiCAD are probably not well equipped to detect such network traffic. The downside is of course that physical access to the product is required to install the backdoored electronics.
> pcb's today contain too much 'made by hand' to hide much without the designer noticing...
Doesn't mean the layers have to be rendered with the modified/malicious version of KiCAD. They could just try to hide it if they detect a layer with id=spyware.
I'm just saying that this would be a very sneaky way to infiltrate the hardware industry, because currently all installed versions rely on the old domain - and that's where they will pull their updates from, too. So pushing out a newer release with that "spyware" modification would be super easy to realize.
Adding an extra layer is expensive (and done in pairs). As others mentioned, when you go to manufacture your boards, you export each layer into its own file, zip them all up and then send/upload the files to your manufacturer. Either you will notice the extra files, or when you put in your order, the manufacturer will reject the order as you gave them the wrong number of files (eg: you paid for 4 layers, but sent them 6). The file format of the layers is basically a vector in ASCII of each trace, so there is little opportunity to hide extra stuff.
More and more manufacturers with online ordering show you images of what they think each layer looks like, and any modifications unless they are very slight will be detected then. You always review each layer in the manufacturers tool as there is a host of things that can go wrong (layer ordering, mirroring, alignment, copper vs solder mask vs silkscreen layer types).
Adding extra components is out too, as the Bill of Materials is exported to CSV, then imported into several component suppliers websites. Any non-basic component is carefully scrutinized for need as they are expensive (and these days hard to get) and to make sure you have everything you need to actually build the board as any non-trivial board requires multiple suppliers to provide all of the components. Even if you missed it then, assembly charges a significant amount per unique component they have to place on the board (eg: placing same resistor twice is cheaper than 2 different resistors).
Once the board is assembled, it will then likely undergo EMI testing to comply with various countries limits on how much RF can leak out of the product. In quite a few cases, final testing is done by a 3rd party lab. This basically limits whatever data exfiltration method to be short range.
If someone wanted to be evil, they would have much better luck on the software side of the product rather than at the board level.
As a part of a highly targeted attack on a single device, maybe. It would have to involve specialist knowledge and manual work to do it. For example, something simple like sabotaging a "mic on" or "camera on" LED on might be possible on a complicated design without anyone noticing for a while. Seems a lot of work for little effect though.
Blanket automated modifying of hardware designs to add "spyware in hardware form"? I would say even "automated" part is impossible at the moment. I've never heard of automation that would understand a hardware design on a level that would be required for that.
In the end, the only modification with a good chance of being missed by the designer is the copper artwork on existing PCB layers. You might add an extra trace, or break an existing trace somewhere. But as soon as your hacked Kicad starts adding BOM items (like an extra "spyware" microprocessor or something) or even extra layers, I guarantee someone is going to notice very soon. If not for other reasons then because these things will add extra $ on someone's bill.
There is no way this could work. The EDA tool (e.g. KiCAD) is only the first step of the fabrication process - you generate Gerbers from your KiCAD board layout, which are a highly standardized format that can be viewed in a variety of independent pieces of software. There are also multiple stages of design validation - both in the design phase, and then in the physical phase of actually inspecting your fabricated boards. The chances of any "malicious" circuitry slipping in are next to nil.
Well, depends on how the Q&A process is in the pipeline after the design.
What I thought of is maybe it might be feasible to sneak in some circuits that reroute e.g. a network port's traffic to a specific public IP/CnC. Depending on how complex the PCB layout is, it could be feasible to encode or modify the modulation of easy network busses (aside from ethernet).
But I guess that would involve deployment of malicious firmware or availability of a specific "malicious" chipset, too, because ethernet is quite complex in the sense that there are too many physical parts necessary to implement it in hardware form.
I was just thinking about the Q&A pipelines in the industrial process. Usually they never validate anything because of proprietary/protected intellectual property contracts, so suppliers down the line always claim it's according to specifications and that is blindly trusted by the manufacturers.
Identifying something like this is much harder in the organizational sense, because it involves a lot of time for verification down the line, and involves a lot of organizational blamestorm before anything really happens to fix it.
I wish the subtitle could have been included in the HN posting: "Scientists scramble to harvest ice cores as glaciers melt"
The next paragraph provides more context: "Ice provides historical records about climate and shows the impact humanity has had. But many glaciers are now melting, prompting renewed urgency among scientists."
[0] https://www.microsoft.com/en-us/worklab/work-trend-index/bre...
[1] https://www.microsoft.com/en-us/worklab/work-trend-index/202...