While this is undoubtedly a well-worn topic (and likely will continue to be as the prevailing philosophies of governing ebb and flow), I want to share some of my thoughts given my experience working alongside US Federal agencies and civil servants. I expect this will not be the last time I write on this subject.
My tl;dr take is this - government can be run like a business (operative word being "like"), however, that proposition requires a more nuanced perspective that may not be readily apparent.
I believe the crux of the pro-"run it like a business" partisans' argument is that the strategic focus should be on the bottom line and efficiency, similar to how a private sector business would look to maximize its operating margin. However, this focus on maximizing an agency's bottom line should focus on how those savings (i.e., the additional margin gained) can be put back into the agency's operations. The underlying assumption of this position seems to be that any optimization that's achieved can result in lower budgets going forward.
In truth, the opposite perspective should be taken. Any savings that are achieved through more efficient / optimal operations, should be invested in ways that improve the agency's operations. Ideally, any funds saved should be invested into a working capital (or revolving) fund. Doing this would achieve two goals - 1) it should incentivize agency executives and staff to find ways to save money (since this money could then be carried forward past the fiscal year) and 2) these savings would be available for "rainy day" reasons. One promising initiative along these lines is the US Federal government's Technology Modernization Fund.
I realize that this is only a cursory take on this topic and, as I said above, I'm sure I'll have more to say on this.