Albert was born at Schloss Rosenau, near Coburg, Germany, the second son of Ernest III, Duke of Saxe-Coburg-Saalfeld, and his first wife, Louise of Saxe-Gotha-Altenburg. Albert's future wife, Victoria, was born earlier in the same year with the assistance of the same midwife. His godparents were his paternal grandmother, the Dowager Duchess of Saxe-Coburg-Saalfeld; his maternal grandfather, the Duke of Saxe-Gotha-Altenburg; the Emperor of Austria; the Duke of Teschen; and Emanuel, Count of Mensdorff-Pouilly. In 1825, Albert's great-uncle, Frederick IV, Duke of Saxe-Gotha-Altenburg, died. His death led to a realignment of Saxon duchies the following year and Albert's father became the first reigning duke of Saxe-Coburg and Gotha.
Albert and his elder brother, Ernest, spent their youth in a close companionship marred by their parents' turbulent marriage and eventual separation and divorce. After their mother was exiled from court in 1824, she married her lover, Alexander von Hanstein, Count of Polzig and Beiersdorf. She presumably never saw her children again, and died of cancer at the age of 30 in 1831. The following year, their father married his own niece, his sons' cousin Princess Antoinette Marie of Württemberg; their marriage was not close, however, and Antoinette Marie had little—if any—impact on her stepchildren's lives.