Before the Governor’s House was built, Hudson’s Bay Company officers lived in Upper Fort Garry’s Main House which was built in the 1830s. This earlier building gradually fell into disrepair, particularly as a consequence of the huge flood of 1852 and the careless housekeeping of the commanding military officers such as John Crofton and William Caldwell who lived in it between 1846 and 1854. For these reasons, the new house, 48 by 28 feet, two and a half storeys, with a hip roof and gables, was built in 1854.
When the first Lieutenant-Governor of Manitoba, Adams Archibald, arrived on 2 September 1870, he stayed in this house as a guest of the Hudson’s Bay Company’s presiding officer, Donald A. Smith. Though he moved to St. James briefly, the Lieutenant-Governor decided that it was too much trouble to ride to the growing settlement of Winnipeg to conduct business, so he negotiated with the Company to rent and reside in the Governor’s House. It remained the Lieutenant Governor’s official residence for the next twelve years.
The Governor’s House was never entirely satisfactory as an official residence. Archibald’s successor, Alexander Morris, complained in a note to the federal government in 1873: “You can fancy the state of the house when I tell you there was five feet of water in the cellar till middle of July. Year before  till middle of August but I am making the H. B. Coy. put in a proper drain.”
Many adjustments were made to the building during its forty years. Key changes included the addition of a West Wing. At some point, a third storey – of a lighter stud frame construction – was added to the original post-and-beam building. Ironically, by 1885, this newer, upper level was abandoned. The building was still standing in 1887, though it was practically falling apart.