Golden Keys are vulnerable. They’ll have to be shared between a number of agencies — meaning people — who need to access suspect communications. Even assuming “acceptable” diligence by their keepers, the keys can still be stolen by determined hackers working for other governments, terrorist organizations, or even domestic corporations who want a leg up on the competition. If banks and other financial organizations are forced to abandon unbreakable encryption, motivated criminals will go to great lengths to steal a Golden Key.
Backdoors in encryption are always a bad idea.