One alleged evil of "excessive competition" here is that with open competition you might get a "race to the bottom" wherein cabs that aren't well maintained or that overcharge customers make the most money. The initial apparent advantage of the medallion system is that it gives the regulators quite a lot of control over what cab operators do and it's easy to imagine that power might be used to do good. Standardizing cab appearance and fares so customers aren't confused, standardizing cab behavior in various ways, that sort of thing.
The downside of this system is that - like nearly every other regulatory system - it is subject to "regulatory capture". Over the years, the established players in any regulated industry have a huge incentive to convince the regulators to rule in ways that keep out competitors. Eventually market competition no longer serves as a check on bad behavior at all - all that matters is keeping the regulators happy. And the regulators are in the pocket of industry, so customers get screwed.
The initial apparent advantage of the medallion system is that it gives the regulators quite a lot of control over what cab operators do and it's easy to imagine that power might be used to do good.
We don't to limit competition for that. All we need to do is spend 50% of medallion fees on inspections (with the remainder going to general wasteful NYC spending, e.g. redistributing wealth based on random chance [1]).
In a comment on this topic a couple of years ago, I did some calculations showing that by spending only 50% of the yearly medallion fee, we could subject every taxi to an undercover inspection every single day.
That's a little harsh - yes, a lottery is involved, but only because the supply falls far short of the demand. The lottery is conducted only among eligible people anyway (or at least, ineligible people are filtered out after the lottery), and only low-income people receive the apartments in the end.
The downside of this system is that - like nearly every other regulatory system - it is subject to "regulatory capture". Over the years, the established players in any regulated industry have a huge incentive to convince the regulators to rule in ways that keep out competitors. Eventually market competition no longer serves as a check on bad behavior at all - all that matters is keeping the regulators happy. And the regulators are in the pocket of industry, so customers get screwed.