Born in Cape Colony, Goldman was a uitlander who spent much of his life in the Transvaal. As a young man he built up a fortune in mining, using some of the profits to purchase an extensive estate known as Schoongezicht in the Middelburg District. During the Second Boer War, Goldman was a war correspondent for The Standard.
Initially attached to Sir Redvers Buller"s relief force, he traveled with them as far as Ladysmith after which he transferred to the cavalry advancing north in order to report on their endeavours.
These experiences served as the foundation for Goldman"s subsequent book With General French and the Cavalry in South Africa. After the war, in an effort to gain political influence Goldman purchased the struggling weekly journal, The Outlook.
Seeking to develop it into a prominent platform for the cause of tariff reformers, he hired the journalist J. L. Garvin as its editors Garvin quickly transformed the journal into a publication of note, yet in spite of an increase in both prominence and circulation the paper failed to turn a profit.
After a series of disagreements between the two men over business matters, Goldman sold the paper to Lord Iveagh in October 1906.
Still desiring a political role, Goldman involved himself in politics directly by entering Parliament, winning the Penryn and Falmouth seat in the January 1910 general election as a Unionist and serving until the borough was abolished in 1918 (the name was transferred to a new county division). During the First World War, Goldman served as a major in the Cornwall Royal Garrison Artillery. In 1919 he purchased the Nicola Ranch and Town site in British Columbia which grew to some 300,000 acres (1,200 km2).
In England Goldman lived at Trefusis House, Falmouth until about 1929, after which he moved to the Jacobean mansion at Yaverland Manor.
Goldman married, in 1899, Honourable Agnes Mary Peel (1871-1959), daughter of liberal politician Arthur Peel, 1st Viscount Peel.