Wow. It appears that movaxbx.ru copied multiple of my blog posts as well[1][2][3][4].
I wonder how hard it would be to take legal action (in particular to stop them from doing this altogether, as opposed to just taking down a single article). The site appears to be hosted in Russia[5]. From a quick glance over the Wikipedia page[6], it seems to me that Russian copyright law is quite similar to copyright law in most developed countries.
LGPL or not, if I remember correctly, F-Secure does not enforce a valid signature of the 7-Zip library, so you can replace it yourself. Don't quote me on this though.
However, F-Secure applies several patches to harden 7-Zip and to fix bugs that are not yet fixed in the public 7-Zip version. So it is not clear whether it is always such a good idea to do this.
I think this is correct. Since _solidAllowed is set to false at the beginning of Code(), it will remain false if an exception occurs in the middle of decoding (CVE-2018-5996). This will enforce PpmError being set to true for the next item, which in turn will enforce the (possibly broken) PPMD state to be reinitialized. In some sense, this means that the new bug fix is a generalization of the first one, fixing both CVE-2018-5996 and CVE-2018-10115.
That's right, they patched CVE-2017-17969, which affected ZIP decompression. Interestingly, I believe they didn't patch CVE-2018-5996 (affecting RAR), which I published [0] on January 23 together with CVE-2017-17969.
Note that the standard 'p7zip' package from Debian/Ubuntu doesn't support RAR. However, they have an additional package 'p7zip-full' or 'p7zip-rar' for RAR support. I didn't check explicitly, but I assume these are affected.
I checked the versions in buster (p7zip-full 16.02+dfsg-6, p7zip-rar 16.02-2) and they look unaffected to me. Turns out that the Debian maintainers patch upstream sources to include hardening flags, e.g. -fstack-protector-strong -D_FORTIFY_SOURCE=2 -Wl,-z,relro. You can use hardening-check to check the binaries on your system.
Okay, so these packages come with more mitigations than 7-Zip on Windows. However, looking at the source code, I am pretty sure they are affected by the same bug.
"Unaffected" was probably the wrong word to use. What I meant is that one of the mitigations (making the executables position-independent) should prevent the bug from being exploitable for remote code execution on Debian.
There were some misunderstandings that I want to clear up (maybe I will add them in an update to the blog post):
1. Some people mentioned that this would "only affect RAR files" and it would be safe to extract 7z files with 7-Zip prior to version 18.05. This is wrong, because 7-Zip detects the file type from the magic numbers at the beginning of the file. So the exploit can be renamed to 'exploit.7z' and it works just as well.
On /r/sysadmin, someone even mentioned that a temporary solution might be to block RAR files. By the same argument, this is unlikely to be effective.
2. Almost all versions prior to 18.05 are affected. I manually checked version 15.05 and 17.01, and they are definitely affected.
3. Not only 7-Zip itself is affected, but essentially all software that uses 7z.dll as library to extract files. This includes various anti-virus software. However, exploitation may be more difficult (though not impossible) if ASLR&DEP is properly enabled (on all modules).
It's fascinating that this category of equipment, which searches for viruses by running untrusted code, is still regularly installed in all corners of valuable networks.
As far as I understand the bug, this is not about running untrusted code (but: a parsing error resulting in state corruption). Unless you refer to the 3rd party lib (7z), used by virus scanners, to analyse rar files. But typically "untrusted code" means code that was supplied "at runtime", not at compile time (like a lib), so e.g. if a virus scanner would actually execute a .exe to evaluate its effects, or run javascript found in a webpage.
To be fair to virus scanner vendors, the only way to mitigate this kind of bug is NIH: don't use 3rd party libs, implement everything yourself. But then, of course, without bugs yourself, as well :)
> To be fair to virus scanner vendors, the only way to mitigate this kind of bug is NIH: don't use 3rd party libs, implement everything yourself. But then, of course, without bugs yourself, as well :)
That is not the only way to mitigate such vulnerabilities. AV vendors have had plenty of time to work on sandboxing parts of their scan engines that have repeatedly been found to have vulnerabilities like this one. Somehow we've arrived at a point where no one would recommend using a browser that doesn't utilize sandboxing to some degree, but when it comes to security products, you can count yourself lucky if they don't just run that code as SYSTEM. That should really tell you all you need to know about the state of the AV industry.
It is a very blurry line between running strange code and running library code against strange data. Mostly though, I'm trying to say that AV boxes should be outside the firewall.
What AVs use this? I keep hearing horrible things about AVs but little proof. I know Tavis from Google finds bugs and AV in rare cases can reduce security, but that's a far cry from them embedding FOSS projects to save time. For AV companies writing a rar parser is a single-day activity, its hard to believe they need this app.
The few places I've seen 7z used is in installers where the input is known (installer archive in 7z format) and I'm assuming signed in many cases so you can't feed it random inputs. I find it hard to believe Sophos and Symantec and Trend are copying and pasting 7zip.dll into their apps.
I don't know about 7-Zip specifically, but AV vendors use plenty of FOSS code. Here's some findings just from Google's P0 showing that Symantec[1], Bitdefender[2], Microsoft[3] and Avast[4] all use unrar in their products. It wouldn't be far-fetched to assume some might use 7-Zip for other archive formats.
Maybe I'm just way overestimating how complex RAR archives are (I admit, I have not looked into this), but I think you're out of your mind if you think that someone could write a parser to analyze RAR archives in a day. That it would be better tested, debugged, and more secure than a tool that has probably seen more widespread use than any single AV.
DEP was previously disabled because Igor used to compile 7-Zip with VC6, which doesn't support the /NXCOMPAT flag. I convinced him back in January to enable it for 7-Zip 18.01. Note, however, that 64-bit versions of Windows enforce DEP even if the /NXCOMPAT flag is missing. Since Windows 10, the 32-bit version does this as well.
ASLR was primarily disabled because Igor wanted to strip the relocation from the binaries in order to save about 0.5-1% in file size. I have discussed this with him, and convinced him to enabled full ASLR for 7-Zip 18.05.
So now we have 7-Zip 18.05 with full ASLR and DEP. Stack canaries (/GS) are still disabled though.
If you can convince him to use Control Flow Guard, Stack Canaries, and HE-ALSR then you should be nominated for for whatever the security community has as an equivalent to a Nobel prize.
If you can convince him to get rid of his custom garbage Stdlib replacements and use ISO C++ then you're a hero to maintainability (and would probably improve the performance because the stdlib has move support).
HE-ASLR I am discussing with him right now, and I think we will get this.
But honestly, I don't think we will ever see a 7-Zip with /GS or CFG. Not only would this cost about 1% in binary size, it would cost an additional 1% in runtime performance loss. Additionally, it would require compiling 7-Zip with a modern compiler like VS2017. You're just asking for too much.
Like blibble says, this is an absurd tradeoff given that 7zip's primary use case is unarchiving files downloaded from the public internet. Between that, the willful ignorance and dismissiveness of security measures displayed https://sourceforge.net/p/sevenzip/feature-requests/1270/, and the apparent need for you to keep pressuring him to enable each countermeasure one-by-one, I question the prudence of using 7zip at all. He is clearly a great developer, and I have no reason to suspect ulterior motives, but his actions don't engender trust.
I know this probably isn't the answer you're looking for, but it might be worthwhile to maintain a parallel fork to enable these options for yourself/others. That way, folks that feel the same way you do can have their cake and eat it too. I realize that it's not an ideal solution though.
Why am I getting downvoted? I sincerely wonder what is so objectionable in my comment. Most uses of 7-zip are one-off events that last around a minute long. Most users aren't decompressing terabyte-range files. It makes no fracking difference to users if they have to wait 5 minutes or 1 minute. But it makes a difference if they receive 7z archive and get their bank account cleaned because the author prematurely optimized his program on a 20 year old compiler. If there was ever a reasonable situation where security flags could be disable then client-side decompression utilities aren't one. Anyone who needs that 5% performance improvement can compile the library themselves. This is what is wrong with this industry.
Consider that it was misplaced to suggest a 500% performance drop. It's used to extract large archives, backups, videos, documents. Folks don't want it to take half an hour instead of a few minutes.
Thanks for replying, this explains things. I guess I extrapolated from my needs too much, I didn't consider users who do daily 10+ GB compressed backups.
I still wonder if there's an ethical dilemma that lies underneath this - is it fair to prioritize needs of users who use your tool in a different way (for backup jobs) over users that use your tool only occasionally, perhaps not even out of their own initiative (they were sent 7z file from a ostensibly trusted source).
> Additionally, it would require compiling 7-Zip with a modern compiler like VS2017
Ahhh, but does say VS2017 produce a smaller executable file, or a faster executable?
I am pleased Igor cares about individual 1% improvements - they stack up to significant savings. However I agree for our work usage security is more important.
> does say VS2017 produce a smaller executable file, or a faster executable?
If I recall correctly, Igor once said that he tested the new VS compiler and it produced neither smaller nor faster executables. I believe there was almost no difference.
Given what I've seen of his code, I'm not surprised. When I mentioned the stdlib I wasn't joking. His code is unable to use modern compiler optimizations such as move, RVO, and constexpr because he's using idioms that prohibit it. It would be nice if he would at least release a solution that others can use to build with VS2017.
Why is that asking too much? I realize paying for software is not something everyone wants, but doesnt the free versions of VS work for compiling 7-zip?
Dunno, just as with FB/CA, users have agreed to what befell them. Relevant extract from LGPL 2.1: "THE ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE OF THE LIBRARY IS WITH YOU."
I agree with others here on the security vs. performance and security vs. binary size trade-offs. As a thought experiment I wondered at what point it would become a hard decision if I were in his shoes. I think a 10% performance hit would make it a harder decision for me, but at that point I would probably still err on the side of caution and prioritize security.
Of course it's possible this is exactly one of the reasons Igor owns something like 7-Zip, and I don't :-)
Developer cares about efficiency. It seems strange in today's world of Electron apps and text editors that take up a GB of ram and have hundreds of ms of latency, but that's clearly where his priorities lie and I, for one, can respect that.
If people are so up in arms about the security problems of 7-Zip, they're free to fork it.
> Some people mentioned that this would "only affect RAR files" and it would be safe to extract 7z files with 7-Zip prior to version 18.05. This is wrong, because 7-Zip detects the file type from the magic numbers at the beginning of the file. So the exploit can be renamed to 'exploit.7z' and it works just as well.
But the contents of a file is what really determines the actual file type, though.
So if you decide to block RAR files, you should do that by looking at the magic number. Then you are good to go. And it's not like you have to write a lot of code or anything to do this. The standard 'file' utility in Unix is able to determine the type of a file -- often by looking exactly at the magic number.
Just like you would inspect the actual contents of a file if you were running an image hosting site -- and not trust whatever arbitrary filename the uploader told you that the file had.
Of course, but I would still strongly advise against this.
If you really cannot avoid implementing something like this, you should inspect the 7-Zip code in order to be 100% sure that the magic number detection in your filter is identical (or matches a superset) to the one from 7-Zip.
> you should inspect the 7-Zip code in order to be 100% sure that the magic number detection in your filter is identical (or matches a superset) to the one from 7-Zip
Those are the two magic numbers for RAR archive version 1.50 onwards and RAR archive version 5.0 onwards respectively, and those are the places they are referenced in the 7-zip source code. I looked at the source archive of the 18.05 version, downloaded from https://www.7-zip.org/a/7z1805-src.7z. I guess if you wanted to be really rigorous you'd look at previous versions as well. If the 7-zip project makes use of a version control system and that vcs supports the equivalent of git blame then that should not be too difficult of a task for whoever wanted to go to that extent of investigation.
And here is where one copy of the 'file' command identifies the same magic numbers as RAR archives:
You missed my point. It is trivial to find out what the magic number is. What is more important though: How exactly is the magic number matched? From what you have written, one might be tempted to simply check whether a file begins with this magic number. And this would be wrong. If you take a look at the matching in CPP/7zip/Archive/Rar/RarHandler.cpp:
7-Zip finds the magic number if it appears within some searchHeaderSizeLimit, i.e., the file does not need to start (at offset 0) with the magic number. For example, 7-Zip will extract a RAR file which begins with [00 52 61 72 21 1A 07 00] (instead of [52 61 72 21 1A 07 00]) just fine.
So I just tried to compile 7-Zip with VS2017 and /DYNAMICBASE. The main binary 7z.dll is 1,569,792 bytes in total, 9344 bytes (0.595%) of which are used by the relocation table. Enabling stack canaries (/GS) gives me a 1,578,496 byte binary (including the relocation table), so another 8704 bytes more.
I assume you mean a performance comparison? The runtime performance cost of ASLR on Windows is zero once a binary has been loaded, since the code is relocated at load time.
Stack canaries might cause a slight performance hit, but it is usually below one percent, since it creates only a small cost per function call for a fraction of all functions.
I wonder how hard it would be to take legal action (in particular to stop them from doing this altogether, as opposed to just taking down a single article). The site appears to be hosted in Russia[5]. From a quick glance over the Wikipedia page[6], it seems to me that Russian copyright law is quite similar to copyright law in most developed countries.
[1]: https://movaxbx.**/2018/06/05/f-secure-anti-virus-remote-cod...
[2]: https://movaxbx.**/2018/05/04/7-zip-from-uninitialized-memor...
[3]: https://movaxbx.**/2018/05/04/7-zip-multiple-memory-corrupti...
[4]: https://movaxbx.**/2018/05/04/bitdefender-heap-buffer-overfl...
[5]: http://ip-api.com/#movaxbx.ru
[6]: https://en.wikipedia.org/wiki/Copyright_law_of_Russia