Athletic director Doug Woolard made the announcement Sunday after meeting with Holtz individually and then accompanying the former coach to a gathering with players. A search for a replacement will begin immediately, with no definitive timetable for naming the successor.
"It was a very difficult meeting and one I will tell you that Skip handled very professionally and very classy as he always does," Woolard said during a news conference.
"It was a matter of just not having the production that we needed to have over the last couple of years on the field. ... Every program experiences highs and lows on the way to national prominence," Woolard added. "Brighter days are ahead."
The son of former Notre Dame coach Lou Holtz went 16-21 at the Big East school, concluding a three-year run with the worst season in school history. The Bulls lost nine of 10 to finish 3-9, 1-6 in the conference, following a 2-0 start.
The firing came a year after Holtz was given a contract extension through 2017 despite going 5-7 in his second season. He will receive a $2.5 million buyout paid over five years.
In a statement released by USF, Holtz thanked his assistant coaches and players for their dedication and loyalty.
"I'm extremely proud of how they fought through adversity during this time. Throughout my time here the young men on this team never gave up, and that reflects on their character as individuals and as a team," Holtz said.
"I believe we made some positive strides, most notably in our academics, that were helping to build a foundation for this program and I would have liked the opportunity to see it through," the coach added. "But I understand the administration's decision and wish them nothing but success in the future."
Holtz inherited a program that Jim Leavitt built from scratch, signing a $9.1 million, five-year contract in January 2010 after Leavitt was fired for mistreating a player who had accused the former coach of grabbing him by the throat and slapping him in the face during halftime of a game.
The 48-year-old Holtz came to USF from East Carolina, where he had guided the Pirates to a pair of Conference USA championships. He welcomed the challenge of stepping into the Big East and trying to transform the Bulls, ranked as high as No. 2 in the country in 2007, into an elite program.
But it did not happen.
The Bulls went 8-5 and appeared in a bowl game for the sixth consecutive season two years ago, but a pattern of underachieving that actually began under Leavitt returned in 2011, when USF won four straight to climb into the Top 25 only to drop seven of eight down the stretch and tumble to the bottom of the conference.
USF has lost 14 of its past 16 against Big East opponents and were 5-16 overall in league play under Holtz, who stated boldly after taking the job: "We can win conference championships here. We can win national championships here."
The dismissal came the day after the Bulls concluded with a 27-3 loss to Pittsburgh. Holtz said afterward that he hoped to retain his job, reiterating that the program had made strides under his direction, including academics, that weren't necessarily reflected in the win-loss record.
Woolard acknowledged as much, lauding the fired coach for setting a "new USF standard for team GPA" and working tirelessly to "mold our football players into outstanding young men."
"However my responsibility ... is clear," Woolard added. "We must strive to put a more successful football program on the field."