1 Comment
User's avatar
Brian Marick's avatar

Without taking a position on the whole issue:

Married to a professor of veterinary medicine, it seems to me comparing the results of AI coding to the results of evolution ought to make you nervous. https://wiki.oddly-influenced.dev/view/welcome-visitors/view/your-body-is-a-gross-kludge

The lesson of medicine is that bodies are not designed for understandability or non-planned-for maintenance. You're essentially conceding that humans working on AI code will be as expensive and failure-prone as physicians working on bodies.

It's worth considering how bad humans are at predicting nonlinear effects (Dörner's /The Logic of Failure/ is good at that, https://www.hachettebookgroup.com/titles/dietrich-dorner/the-logic-of-failure/9780201479485/?lens=basic-books) That's the root of the oft-quoted Hemingway bit:

“How did you go bankrupt?” Bill asked.

“Two ways,” Mike said. “Gradually and then suddenly.”

That's an effect of how profoundly bad we are at understanding exponential growth.

If it were my money at stake, I'd want a some assurance that if AI hits a wall, I won't be left with the equivalent of a nasty autoimmune disease. That is: what are the risks? how will you monitor them? what's your disaster recovery plan?

The problem with a ZIRP is that those questions are b-o-r-i-n-g and you can't compete with those who skip them. You're out of business before they crash. ("The market can remain irrational longer than you can remain solvent.")

Similarly, there's a collective action problem. Our society is structured such that when the optimists' predictions go wrong, they don't pay for their mistakes – rather society as a whole does. See housing derivatives in 2008, the Asian financial crisis of the late '90s, etc. ZIRP makes it cheaper to be an optimist, but someone else pays the bill for failure (Silicon Valley Bank, Savings and Loan crisis)

It's weird to see ZIRP touted as a model, given the incredible overspending that took place, which had to be clawed back once ZIRP went away. (Most notably in tech layoffs, but I'm more concerned about all the small companies that were crushed because of financials, not because of the merit of their products.)

Please extend your analogy to the end of the AI ZIRP environment. Or will line go up forever?

Expand full comment