Gilmour joined the army, and served in the Anglo-Zulu War (1879) and in the Sudanese campaign 1884-1885. He was later a Captain of the Royal Company of Archers, and was on 29 July 1926 created a Baronet, of Liberton and Craigmillar in the County of Midlothian. Gilmour married on 19 October 1889 Lady Susan Lygon (24 May 1870 – 28 January 1962), 2nd daughter of the 6th Earl of Beauchamp.
They had three children:
Margaret Gilmour Dugdale
Sir John Little Gilmour, 2nd Baronet
Lady Susan Gilmour
Lady Susan Lygon Gilmour (24 May 1870 – 28 January 1962) was created a Dame Commander of the Order of the British Empire in 1936 "or services in connection with the Queen"s Institute of District Nursing in Scotland." She may also be styled as Dame Susan Gilmour. She died in 1962, aged 91.