Chrome cripples movie studio Mac Pros

It’s not often that a single software bug can bring an entire industry to a virtual standstill, but it happened this week – and experts finally found an unlikely culprit.

The problem began on Monday 22 September when reports emerged of a problem with Macs running Avid software.

Avid is an editing suite that production companies use to put movies and TV programs together. A few days ago, movie editors started reporting that Mac Pros running Avid software were crashing. If users tried to restart their machines, they wouldn’t reboot. It left production studios ringing their hair as they lost valuable editing time.

Here’s one tweet from Shane Ross, staff editor at Prometheus Entertainment, as the situation broke:

Can anyone confirm issues with Macs running Avid (mainly reports of Mac Pro Trash Cans) shutting down, but not comi… twitter.com/i/web/status/1…

—
SHANE ROSS ⚔ (@comebackshane) September 24, 2019

And here’s Michael Kamens, assistant editor on Modern Family, complaining of the same thing:

@Monica_Edits Happened to our whole team at modern family. Our vendor is moviola. Its all systems there and other v… twitter.com/i/web/status/1…

—
Michael Kamens (@TheMBrand) September 24, 2019

Imagine how you’d be feeling if you were working on something with a deadline of hours, like a news segment.

Props to Avid, which was all over this problem from the beginning, dropping everything to work out the issue, in a perfect example of how to handle a technical issue properly. The company even put up a video:

What was going on? Was it a virus? Was it another crazy attack from hackers upset about a movie that they didn’t like?

Yesterday, we found out. Google did it.

The problem wasn’t with Avid, or with macOS, but with the Chrome browser. Google’s latest Chrome update had borked the system with a bug.

When Mac users install Chrome, they’re not just getting the browser. Google also installs another module under the hood called Keystone. It’s an update manager that regularly checks to see if there are new versions of Google programs and updates them behind the scenes. Doesn’t that make you feel safe? Well, it does, until it goes wrong. The latest version of Keystone was broken.

According to a Google post explaining the incident, Chrome damaged the file system on macOS machines. It said little more than that, other than providing some command line code to fix the issue. A Chrome bug report shed more light on the matter, though.

Chrome removed a symbolic link (symlink), which is a shortcut to a linked object. The system treats the symlink as the linked object. Keystone removed the /var symlink, which threw the affected Macs into disarray. Several online commentators have already labelled the bug a ‘varsectomy’. Geddit?

Macs are supposed to prevent programs from tinkering with the system by default, using a projective measure called System Integrity Protection (SIP). Also known as rootless, it’s a feature introduced in the El Capitan version of macOS that protects system-owned files from alteration. It even protects them from sudo, which is the Linux command that people use when they’re doing dangerous stuff on the system and need to escalate their privileges.

SIP is switched on by default, but programs wanting deep access to graphics cards, like, say, a movie editing program, often need it turned off. That’s why Avid users were so vulnerable to the issue, but it also affects pre-El Capitan versions of macOS that didn’t have SIP installed.

This just goes to show how much trouble a simple software bug can cause. The problem wasn’t with Avid or macOS at all, but with a completely different third-party app.

Google was still working on a patch at the time of writing. In the meantime, follow the instructions to fix the problem.

This is one situation where the classic question “have you tried turning it off and on again” most definitely would not have been advisable.