I was surprised by how dismissive these comments are. Former staff members, engineers included, are claiming that their former company's unsafe development culture contributed to a colossal world-wide outage & other previous outages. These employee's allegations ought to be seen as credible, or at least as informative. Instead, many seem to be attacking the UX designer commenting on 'Quality control was not part of our process'.
My guess is that people are identifying with sentence said just before: "Speed [of shipping] is everything." Aka "Move fast and break things."
The culture described by this article must mirror many of our lived experiences. The pure pleasure of shipping code, putting out fires, making an impact (positive or negative)... and then leaving it to the next engineers & managers to sort out, ignoring the mess until it explodes. Even when it does, no one gets blamed for the outage and soon everyone goes back to building features that get them promoted, regardless of quality.
Through that ZIRP light, these process failures must look like a feature, not a bug. The emphasis on "quality" must also look like annoying roadblocks in the way of having fun on the customer's dime.
This is not a game. I would normally agree but not when it comes to low-level kernel drivers. They're a cyber security company making it even worse.
Not very long ago we had this client who ordered a custom high security solution (using a kernel driver). I can't reveal too much but basically they had this offline computer running this critical database and they needed a way to account for every single system call to guarantee that any data could have not been changed without the security system alerting and logging the exact change. No backups etc were allowed to leave the computer ever. We were even required to check ntdll (this was on Windows) for hooks before installing the driver on-site & other safety precautions. Exceptions, freezes or a deadlock? No way. Any system call missed = disaster.
We took this seriously. Whenever we made a change to the driver code we had to re-test the driver on 7 different computers (in-office) running completely different hardware doing a set test procedure. Last test before release entailed an even more extensive test procedure.
This may sound harsh but CrowdStrike are total amateurs, always been. Besides, what have they contributed to the cyber security community? - Nothing! Their research are at a level of a junior cyber security researcher. They are willing to outright lie and jump to wild conclusions which is very frowned upon in the community. Also heard others comment on how CS really doesn't really fit the mold of a standard cyber security company.
Nah, CS should take a close look at true professional companies like Kaspersky and Checkpoint; industry leaders who've created proven top notch security solutions (software/services) but not least actually contributed their valuable research to the community for free, catching zero-days, reporting them before no one even had a chance of exploiting them.
Absolutely. Some people are born firefighters. Nothing wrong with that.
I once worked with a senior engineer who loved running incidents. He felt it was real engineering. He loved debugging thorny problems on a strict timeline, getting every engineer in a room and ordering them about, while also communicating widely to the company. Then, there's the rush of the all-clear and the kudos from stakeholders.
Specific to his situation, I think he enjoyed the inflated ownership that the sudden urgency demanded. The system we owned was largely taken for granted by the org; a dead-end for a career. Calling incidents was a good way to get visibility at low-cost, i.e., no one would follow-up on our postmortem action items.
It eventually became a problem, though, when the system we owned was essentially put into maintenance mode, aka zero development velocity. Then I estimate (balancing for other variables) the rate the senior engineer called an incident for not-incidents went up by 3x...
I agree that enjoying firefighting is not inherently harmful. However, the situation you describe afterward irks me in some way I can't quite put my finger on. A lot of words (toxic, dishonest, marketing, counterproductive, bus factor) come to mind, but none of them quite fit.
Some people rise to the occasion during crises and find it rewarding. There's a lot of pop science around COMT (the "warrior gene" associated with stress resilience), which I take with a grain of salt. There does seem to be something there, though, and it overlaps with my personal experience that many great security operations people tend to have ADHD traits.
I've volunteered to fight a share of fires from people who check things in untested, change infrastructure randomly, etc.
What I've learned is that fixing things for these people (and even having entire teams fixing things for weeks) just leads to a continued lax attitude to testing, and leaving the fallout for others to deal with. To them, it all worked out in the end, and they get kudos for rapidly getting a solution in place.
I'm done fixing their work. I'd rather work on my own tasks than fix all the problems with theirs. I'm strongly considering moving on, as this has become an entrenched pattern.
Former QA engineer here, and can confirm quality is seen as an annoying roadblock in the way of self-interested workers, disguised as in the way of having fun on the customers dime.
My favorite repeated reorg strategy over the years is “that we will train everyone in engineering to be hot swappable in their domains”. Talk about spinning wheels.
My guess is that people are identifying with sentence said just before: "Speed [of shipping] is everything." Aka "Move fast and break things."
The culture described by this article must mirror many of our lived experiences. The pure pleasure of shipping code, putting out fires, making an impact (positive or negative)... and then leaving it to the next engineers & managers to sort out, ignoring the mess until it explodes. Even when it does, no one gets blamed for the outage and soon everyone goes back to building features that get them promoted, regardless of quality.
Through that ZIRP light, these process failures must look like a feature, not a bug. The emphasis on "quality" must also look like annoying roadblocks in the way of having fun on the customer's dime.