Neumann was born in Prachatice, Bohemia (modern-day Czech Republic), then within the Austrian Empire. He attended school in Budweis before entering seminary there in 1831. Two years later he transferred to the University of Prague, where he studied theology. He was interested in astronomy and botany. He intended to be ordained, but his bishop, in 1835, decided there would be no more ordinations, as Bohemia had a high number of priests already.
Neumann, who spoke eight languages then wrote to other bishops in Europe, but they all replied that they also had too many priests already. He was inspired by the missionary writings of Bishop Frederic Baraga in America, and because he had learned English by working in a factory with English-speaking workers, Neumann wrote to bishops in America, requesting to be ordained in the United States. In 1836, he arrived in the United States with very little money, and was ordained to the priesthood there. He was assigned by the Bishop of New York to work with recent German immigrants in mission churches in the Niagara Falls area, where he visited the sick, taught catechism, and trained teachers to take over when he left. After four years of service there, he realized his own need for support and came to realize the importance of communal activity in his work. He thus applied to the Redemptorists. He was accepted, and entered the novitiate of the order in Pittsburgh, Pennsylvania. In January, 1842, he took the vows to enter the order in Baltimore, Maryland, and became the first Redemptorist in the New World. After six years of difficult but fruitful work with the order, he was appointed the order's provincial superior in the United States. Neumann was naturalized as a citizen of the United States in Baltimore on February 10, 1848.
In March 1852, Neumann was consecrated in Baltimore, as Bishop of Philadelphia, Pennsylvania. He was the first to organize a Catholic diocesan school system and increased the number of Catholic schools in his diocese from one to two hundred. He also introduced the School Sisters of Notre Dame to the New World to assist in religious instruction and staffing the orphanage. In 1853, he established Saint Peter's Parish in Riverside, New Jersey.
Neumann was not a popular bishop and received criticism. He had to deal with the Know Nothings, a political group determined to deprive foreigners and Catholics of their civil rights; the group burnt down convents and schools. Discouraged, Neumann unsuccessfully wrote to Rome and asked for someone else to take his place.
Neumann wrote in many Catholic newspaper and magazine articles. He also published two catechisms and a Bible history in German. There were also many teaching orders brought in by him.
In 1860, Neumann died due to a stroke at the age of 48 while walking down a street in Philadelphia. After his death people began to talk of how great he had been.