Ford's stories were not always reliable but at least the setting of this one is right: William Rossetti was Secretary to the Inland Revenue, which was based on Somerset House. In the Spring of 1852 Mary Anne Evans and Herbert Spencer
'would walk on the terrace of Somerset House, to which [John] Chapman had a key', and there 'they would pace backwards and forwards for hours, "discussing many things"' (Gordon S. Haight, George Eliot: A Biography (1968), p. 112).