The organization made the announcement Friday morning.
"After several productive conversations, we made the difficult decision to part ways with Scott Pioli and allow him to pursue other opportunities," Chiefs Chairman and CEO Clark Hunt said.
The announcement comes four days after Hunt fired head coach Romeo Crennel and said there had not been a "final determination" on Pioli's status with the organization.
Multiple reports say the Chiefs are nearing a deal to bring Andy Reid in as the next head coach of the team, and the former Eagles coach is said to be interested in having former Browns general manager Tom Heckert or the Packers' Director of Operations John Dorsey take over in Kansas City.
The Chiefs posted a 23-41 record during Pioli's tenure.
"The bottom line is that I did not accomplish all of what I set out to do," Pioli said in a statement. "To the Hunt family - to the great fans of the Kansas City Chiefs - to the players, all employees and alumni, I truly apologize for not getting the job done."