Cruise, General Motors’ robotaxi subsidiary, has announced the resignation of its founder/CEO, Kyle Vogt, in the wake of a serious road incident which led the California DMV to order Cruise’s vehicles off the road, and for Cruise to voluntarily stop service in other US locations. Mo Elshenawy, a current EVP of Engineering at Cruise, will take the role of President and CTO. Craig Glidden, who recently took a role as Chief Administrator at Cruise, will add President to his titles, making two men in the role of President. Some other management changes were announced in the blog post. This seems to have been an eventful weekend for CEOs of world-changing AI companies.
This caps a stunning month for Cruise and Vogt, who founded the company as a startup then sold it to GM. For a period, GM had appointed Dan Amman as CEO of Cruise, but he left in a conflict with Mary Barra, CEO of GM, over Cruise’s plans and IPO potential, and Vogt returned to the role (while continuing as CTO). Under Vogt’s returned leadership, Cruise underwent a rapid expansion, deploying live service to the public in multiple cities, and announcing several more that would go live soon. They also expanded service in San Francisco from evening-only to all day.
With the expanded service came a number of incidents, including traffic blockages from paralyzed cars, minor and moderate crashes, and interactions with emergency vehicles and scenes which raised the ire of the Fire Chief, the Muni transit agency and others in the city. A battle arose between the city and both Cruise and its competitor Waymo, but both companies seemed to be doing OK, because the city does not have jurisdiction over driving—that belongs to the state of California.
It all came to a head however during a serious incident which began when an unknown human hit-and-run driver hit a jaywalking pedestrian while driving in the lane next to a Cruise vehicle. The victim was bounced off the first car and into the lane in front of the Cruise, which could not avoid hitting her, though it tried to stop. That would have been unfortunate, but with no fault to Cruise, except for what happened next.
After stopping, the Cruise vehicle reportedly realized it had hit a pedestrian. However, it incorrectly concluded it had hit her on the side of the car, rather than having run her over with the front wheels. It was apparently unaware she was under the vehicle. It followed programming to clear the road and so attempted to pull over by driving forward 20 feet and slightly to the right. In doing so, it pulled the victim along and ended up with the rear tires resting on her leg. Emergency crews advised that the vehicle should not move off of her, and instead jacked the vehicle up to extract her and take her to hospital, where she remains. There have been minimal updates on her condition, though some suggestions are that she is improving slowly.
This, however, is not all (or even most) of what sunk Vogt and Cruise with the DMV. When Cruise offered reports on this incident to the press (including myself) and the DMV, they did not mention the dragging after the vehicle had stopped. The DMV states they were not shown the video of this dragging, though Cruise insists they were shown it multiple times. However, a written statement from Cruise to the DMV the morning after the crash omitted this very important detail. This was also the case in my discussions with Cruise—while I asked the video be stopped before the incident, I asked a number of questions that should have elicited information about the dragging, but did not. Other press reports describe a similar situation.
The DMV accused Cruise of being misleading, and so pulled their permit to operate in California. The California PUC automatically withdrew their permits to operate a taxi service. Cruise voluntarily stopped operation in other cities in the USA.
In the intervening period, a number of damaging leaks and revelations have come out of the company. Morale is reportedly extremely low, as can be seen by how frequently staff are leaking internal information.
GM paused production of Cruise’s custom vehicle, known as the Origin, which is key to their strategy. They put a hold on employee stock redemptions, though these have been partially restored. The letter to the DMV from Cruise’s senior counsel which omitted the dragging was disclosed, and more.
Analysis
Cruise’s growth this year has been very rapid. There has been frequent speculation that they were pushing too hard, too fast. They published results and research showing their vehicles were having fewer safety incidents than other ridehail drivers (such as Uber) which would suggest they were meeting acceptable safety goals and not pushing too fast, but their non-safety incidents (traffic blockages) were inhuman in nature and raised ire in the city. Even so, they received a permit to expand service and quickly did so, but then rapidly had a series of safety incidents, culminating in the pedestrian dragging.
In evaluating if a robotaxi pilot is going too fast, the public will tend to focus on incidents, and so the public’s representatives also will do this. The real answer, however, comes from data and statistics. No robotaxi service, and certainly no pilot, will be without incidents, including serious ones. The question is rather whether the level of these reaches a level of being unacceptably unsafe. The human drivers these robotaxis are replacing are unsafe, but we (for now) deem them acceptably unsafe. It would be an error to halt development of robotaxis that will make the roads safer if they meet the same”acceptably unsafe” level, or beat it as Cruise claims.
While we haven’t seen the final data after the flurry of incidents Cruise engendered after they expanded service, they might or might not have reached a level that would be deemed unacceptable. If they did, they should have stopped on their own. Cruise had trouble appointing a permanent Chief Safety Officer—somebody with strong authority to enforce safety requirements in the company—and only did so after the incident.
The real failure, though, was the creation of a culture of keeping details close to the vest. Cruise had a disturbing pattern of revealing only what they felt they had to. This probably arose from fear of the public’s instinct to look at incidents rather than statistics, but in the end it was not sustainable. When the dragging incident happened, they worked hard to get out the message that another driver was at fault. That’s an unsurprising desire, since any company lives in fear of the crashes where they are at fault. Any such crash can sink them. Somehow this led to the non-disclosure of the mistake their software did make in this incident.
It is likely that this pattern of low disclosure was led by Vogt. In the end responsibility stops with him. If he had wanted to create a more transparent project he could have. But the way the public reacts to incidents holds some blame in making such transparency frightening. Even Waymo, which tries to be more transparent than Cruise, has trouble doing it. The same is true of most other companies. In a field where there are only two major players, it may not be much to say that “Cruise was the worst at it” but it is nonetheless true. I regularly advised their PR department that in my judgment they were taking the wrong approach. But I say the same to Waymo, where I worked a decade ago.
Even so, Cruise’s problems are a lesson for the industry and for society. We face a dilemma. Robocar technology offers the promise of making our roads safer. In fact, it offers a guarantee, because if the companies can’t demonstrate that they are a fair bit safer than human drivers, they will get shut down, and permanently. All their billions of investment will be wasted. It’s deliver that safety or don’t go at all.
With that guarantee, the companies must face the public’s difficulty in comparing human unsafety with machine unsafety. We’re much more afraid of being harmed by robots than by people. We’re picky and don’t want to be run over by robots, but accept being run over by drunks.
Our legal and regulatory systems have a bug. With Cruise’s dragging incident, some fixes were immediately obvious. The first one Cruise deployed was to stop the car from trying to pull over in situations like this. With some irony, it seems likely that the reason the car was so quick to pull over was due to all the pressure from city officials that the cars were freezing and blocking streets and emergency vehicles in uncertain situations. This car was in an uncertain situation where it definitely should have not tried to move until humans gave the OK. (Cruise was also recently criticized for leaks suggesting they were relying too much on remote human advice to resolve confusing situations, something they will wish had happened here.)
Our legal systems usually ask in these situations, “Could this have been foreseen and prevented?” If so, we may decide the action was negligent. The problem is that almost every incident will have a fairly obvious fix when examined after-the-fact. In fact, it’s one of the great things about robotaxis that unlike humans, none of the cars in the fleet (or competitor’s fleets) will make the same mistake again.
So every incident might be negligent or unacceptable in hindsight. But there is no way to prevent them all in advance. That’s not within human competence yet. We have not conquered the challenge of bug free software, even in much more simple realms. They will happen, and be obvious in hindsight, and if we are unwise, we will pull them off the roads, and delay the eventual delivery of much safer roads. While we delay it, drunks and others will continue the carnage we are used to.
This doesn’t mean that everything’s acceptable. In fact, in this case, it is not just a question of hindsight. On Sept 30, just a few days before this incident, I happened to write, “A nightmare would be a car hitting and dragging somebody, oblivious. I have encouraged that there be sensors which will detect this. Sensors that see the sides and rear can help, some cars don’t have those. A car could also detect unusual acceleration.” This was not the first time I spoke of this, I had previously advised a client of this issue, though not Cruise. (I advised Cruise briefly when it was getting started, but not for compensation or at this level of detail.) As such I consider both dragging, and hiding it, to be on the “this shouldn’t happen, even by bad luck” list. A few other of Cruise’s incidents, including hitting the back of a bus, turning from the wrong lane and being hit by a fire truck are also on that list for me, but others may disagree.
If there is an objective list of such things, and a company is failing at too many of them, it may be a sign they are not ready. I don’t have access to the internal details of all these incidents as would be needed to make a full judgement, but it could be it would say Cruise pushed too hard.
This doesn’t mean they should be shut down. If they can find good technical and strategic leadership, their current statistics suggest they could deliver. Waymo is doing a better job, but they should not be without competition. (Zoox and Motional hope to be that competition as well, along with several Chinese companies who will have difficulty deploying in the USA, and there are also a number of startups, and even Tesla has aspirations in spite of being so far behind.)
In spite of its mistakes, Cruise has also been the subject of unfair treatment. The San Francisco Fire Department accused them of blocking an ambulance, delaying the journey of a patient to hospital, where they died. The department had to walk that back when video recordings showed this rather nasty accusation to be false. Both Waymo and Cruise have to be thankful for their video records which have also shown the reality of situations where accusers had poor perception, or possibly were slandering. Hard as it is, in the end we should pay attention to statistics, and Cruise has good statistics.
Or so they tell us, and that’s their biggest problem. For quite some time, we’ll think “Cruise alleges…” in front of any statement from Cruise. They have earned that impression, and must earn their way out of it. I hope that as Cruise moves forward, we’ll see the real truth, and as long as that truth is good, if they aren’t worse than the people they are substituting for and guarantee to get better or die, they should definitely continue and become a major player, backed with the resources of one of the world’s leading automakers. No other automaker is nearly as far along, all the other leaders are high-tech companies.
Vogt’s fall is also sad because it is founder-led companies which are often the real movers and shakers in the world. Cruise will no longer be founder-led. It will not have somebody at the helm who can make things happen in the company not simply because they have authority from the board of directors, but because they are the person who brought the company to where it is. That’s amazingly important, especially in a startup or a company trying to do what has never been done before.
Alphabet is still founder-led, though much less so than before, and even less so at Waymo, though one of the CEO’s is from the original team. Zoox lost one of its two founders but has the other in the CTO role. Motional is founder-led, and several of the Chinese players are. What lies ahead for Cruise is a story yet to be written.
Read the full article here