The predominantly black Protestant denomination, headquartered in Memphis, claims 6 million members worldwide and traces its origins to the 1870s.
The church said in a statement that Patterson died at Methodist University Hospital in Memphis.
Patterson was hospitalized in January for an undisclosed illness. He told his followers in 2005 that he suffered from prostate cancer.
At the 99th annual COGIC Holy Convocation in November, he said he had considered stepping down from the post he held since 2000, but changed his mind after receiving an outpouring of support.
"If my body being afflicted can get us back to where God wants us to be, then I'm willing to suffer," Patterson said at the time. Following his comments, Patterson received a standing ovation.
"It was a direct result of his work, such as his TV ministries, that allowed people to see COGIC as it should have been projected," said jurisdictional Bishop Jerry Maynard, the chief operating officer of the national church who had worked with Patterson since 1992.
"He always said if you represent God, you should represent him in a very positive way and not in a way that cause people to question God or his ways."
Maynard said the church would begin selecting a successor after memorial services are completed.
"Many people know him because of his worldwide telecast and the kind of compassion and love and faithfulness that he demonstrated," said San Diego jurisdictional Bishop George D. McKinney, a member of the COGIC General Board.
"America lost an angel today," former U.S. Rep. Harold Ford Jr. said in a statement. "He was favored by the Almighty and he used his favor to instruct us all on how to be better servants."
He was born in Humboldt, Tenn., and was ordained as an elder in COGIC in 1957 in Detroit.
Patterson attended Lemoyne-Owen College in Memphis, Detroit Bible Institute and held an honorary doctorate from Oral Roberts University.
Patterson is survived by his wife, Louise Patterson. The couple had no children. Funeral arrangements were pending.