Paul Rougnon was born in Poitiers the son of Louis Rougnon and Claire Clotilde Robin. A student at the Lycée Bonaparte (now the Lycée Condorcet), he entered the Conservatoire de Paris in 1861 as an auditor, then a full-time student in 1862. He studied piano, music theory and composition with such masters as Édouard Batiste, François Bazin, Ambroise Thomas, Antoine François Marmontel and César Franck. He received degrees in music theory in 1865, harmony in 1868 and counterpoint in 1870.

In 1873, at age 27, Rougnon became a professor at the Conservatoire, then under the directorship of Ambroise Thomas. He taught music theory, counterpoint and fugue until his retirement in 1921. His students include Alfred Cortot, Yves Nat, Fernand Oubradous, Noël Gallon, and Henri Mulet. In addition to teaching, he was an administrator for the choral societies Orphéon and Sociétés musicales mutuelles. He composed choral works especially for these ensembles. After the turn of the century, he began to work with various magazines including Piano-Soleil, Le Monde Musical, Le Ménestrel, Le Monde Orphéonique and L’Harmonie.

As a prolific composer and writer, he composed more than 300 musical works in addition to literary and pedagogical volumes. He composed hundreds of piano pieces, two operas, some chamber music, and also vocal and choral works. In 1896, under the tenure of Théodore Dubois, the Conservatoire began the practice of charging composers to write contest pieces. Rougnon began composing works for this purpose, particularly for piano, viola, flute, and trumpet. His treatises on music theory and piano pedagogy are still in use today.

Rougnon received a gold medal for his collective works of music education at the 1900 Exposition Universelle in Paris. In 1911, he was made a Chevalier of the Légion d’honneur.

Rougnon married Marie-Louise de Beurmann in 1887 and had five children. He died on 11 December 1934 at his home in Saint-Germain-en-Laye.