Rubinstein was born to Jewish parents in the village of Vikhvatinets in the Podolia Governorate, Russian Empire (now known as Ofatinți in Transnistria, Republic of Moldova), on the Dniestr River, about 150 kilometres northwest of Odessa. Before he was 5 years old, his paternal grandfather ordered all members of the Rubinstein family to convert from Judaism to Russian Orthodoxy. Although he was raised as a Christian, Rubinstein would later return to being atheist.
Rubinstein's father opened a pencil factory in Moscow. His mother, a competent musician, began giving him piano lessons at five, until the teacher Alexander Villoing heard and accepted Rubinstein as a non-paying student. Rubinstein made his first public appearance at a charity benefit concert at the age of nine. Later that year Rubinstein's mother sent him, accompanied by Villoing, to Paris where he sought unsuccessfully to enroll at the Paris Conservatoire.
Rubinstein and Villoing remained in Paris for a year. In December 1840, Rubinstein played in the Salle Érard for an audience that included Frédéric Chopin and Franz Liszt. Chopin invited Rubinstein to his studio and played for him. Liszt advised Villoing to take him to Germany to study composition; however, Villoing took Rubinstein on an extended concert tour of Europe and Western Russia. They finally returned to Moscow in June 1843. Determined to raise money to further the musical careers of both Anton and his younger brother Nikolai, their mother sent Rubinstein and Villoing on a tour of Russia, following which the brothers were dispatched to Saint Petersburg to play for Tsar Nicholas I and the Imperial family at the Winter Palace. Anton was 14 years old; Nikolai was eight.
In spring 1844, Rubinstein, Nikolai, his mother and his sister Luba travelled to Berlin. Here he met with, and was supported by, Felix Mendelssohn and Giacomo Meyerbeer. Mendelssohn, who had heard Rubinstein when he had toured with Villoing, said he needed no further piano study but sent Nikolai to Theodor Kullak for instruction. Meyerbeer directed both boys to Siegfried Dehn for work in composition and theory.
Word came in the summer of 1846 that Rubinstein's father was gravely ill. Rubinstein was left in Berlin while his mother, sister and brother returned to Russia. At first he continued his studies with Dehn, then with Adolf Bernhard Marx, while composing in earnest. Now 17, he knew he could no longer pass as a child prodigy. He sought out Liszt in Vienna, hoping Liszt would accept him as a pupil. However, after Rubinstein had played his audition, Liszt is reported to have said, "A talented man must win the goal of his ambition by his own unassisted efforts." At this point, Rubinstein was living in acute poverty. Liszt did nothing to help him. Other calls Rubinstein made to potential patrons came to no avail. After an unsuccessful year in Vienna and a concert tour of Hungary, he returned to Berlin and continued giving lessons.
The Revolution of 1848 forced Rubinstein back to Russia. Spending the next five years mainly in Saint Petersburg, Rubinstein taught, gave concerts and performed frequently at the Imperial court. The Grand Duchess Elena Pavlovna, sister-in-law to Tsar Nicholas I, became his most devoted patroness. By 1852, he had become a leading figure in Saint Petersburg's musical life, performing as a soloist and collaborating with some of the outstanding instrumentalists and vocalists who came to the Russian capital.
He also composed assiduously. After a number of delays, including some difficulties with the censor, Rubinstein's first opera, Dmitry Donskoy (now lost except for the overture), was performed at the Bolshoy Theater in St. Petersburg in 1852. Three one-act operas written for Elena Pavlovna followed. He also played and conducted several of his works, including the Ocean Symphony in its original four-movement form, his Second Piano Concerto and several solo works. It was partly his lack of success on the Russian opera stage that led Rubinstein to consider going abroad once more to secure his reputation as a serious artist.
In 1854, Rubinstein began a four-year concert tour of Europe. This was his first major concert tour in a decade. Now 24, he felt ready to offer himself to the public as a fully developed pianist as well as a composer of worth. He very shortly reestablished his reputation as a virtuoso. Ignaz Moscheles wrote in 1855 what would become a widespread opinion about Rubinstein: "In power and execution he is inferior to no one."
As was the penchant at the time, much of what Rubinstein played were his own compositions. At several concerts, Rubinstein alternated between conducting his orchestral works and playing as soloist in one of his piano concertos. One high point for him was leading the Leipzig Gewandhaus orchestra in his Ocean Symphony on November 16, 1854. Although reviews were mixed about Rubinstein's merits as a composer, they were more favorable about him as a performer when he played a solo recital a few weeks later.
Rubinstein spent one tour break, in the winter of 1856–7, with Elena Pavlovna and much of the Imperial royal family at Nice. Rubinstein participated in discussions with Elena Pavlova on plans to raise the level of musical education in their homeland; these bore initial fruit with the founding of the Russian Musical Society (RMS) in 1859.
The opening of the Saint Petersburg Conservatory, the first music school in Russia and a development from the RMS per its charter, followed in 1862. Rubinstein not only founded it and was its first director but also recruited an imposing pool of talent for its faculty.
By 1867, ongoing tensions with the Balakirev camp, along with related matters, led to intense dissension within the Conservatory's faculty. Rubinstein resigned and returned to touring throughout Europe. Unlike his previous tours, he began increasingly featuring the works of other composers. In previous tours, Rubinstein had played primarily his own works.
Rubinstein continued to make tours as a pianist and give appearances as a conductor. In 1887, he returned to the Saint Petersburg Conservatory with the goal of improving overall standards. He removed inferior students, fired and demoted many professors, made entrance and examination requirements more stringent and revised the curriculum. He led semi-weekly teachers' classes through the whole keyboard literature and gave some of the more gifted piano students personal coaching. During the 1889–90 academic year he gave weekly lecture-recitals for the students. He resigned again and left Russia in 1891 over Imperial demands that Conservatory admittance, and later annual prizes to students, be awarded along racial quotas instead of purely by merit. These quotas were effectively to disadvantage Jews. Rubinstein resettled in Dresden and started giving concerts again in Germany and Austria. Nearly all of these concerts were charity benefit events.
Rubinstein also coached a few pianists and taught his only private piano student, Josef Hofmann. Hofmann would become one of the finest keyboard artists of the 20th century.
Despite his sentiments on ethnic politics in Russia, Rubinstein returned there occasionally to visit friends and family. He gave his final concert in Saint Petersburg on January 14, 1894. With his health failing rapidly, Rubinstein moved back to Peterhof in the summer of 1894. He died there on November 20 of that year, having suffered from heart disease for some time.
The former Troitskaya street in Saint Petersburg where he lived is now named after him.