Olivia was mentioned in the 8 January 1843 will of her father, James Dunn, proved 16 January 1843. 6th: I lend to my son in law Peter T Duff one seventh part and at his death I give the same to his Wife, Olivia D. Duff & her Children. [Essex County, Virginia, Will Book 25, pages 49–50.]

In 1843, orators and oratrices James William infant of James Dunn decd who sued by Virginia F. Dunn his mother and next friend, Peter T. Duff and Olivia Duff his wife who was Olivia D. Dunn daughter of the said James Dunn decd, Thomas Parron and Julia D. his wife who was Julia D. Dunn another daughter of James Dunn decd, Charles C. Smith and Emily his wife who was Emily Dunn another daughter of James Dunn decd, William R. Allen who married Drusilla another daughter of James Dunn since dead, Arthur F. Allen an infant of William R. Allen and Drusilla by his father and next friend, Cordelia Dunn another daughter of James Dunn, and Dorinda M. Dunn another daughter of James Dunn an infant who sued by James A. Dunn her next friend, complained that James Dunn died about __ January 1843, testate. For division of his estate. [Essex County, Virginia, Box Chancery No. 40, item 40-D-7.]

1850 U.S. Census, Essex County, Virginia
Sheet 95
Dwelling 573, Family 573
Duff, Peter, 30, male, farmer, value of real estate $1,500, born in Virginia
Duff, Olivia, 28, female, born in Virginia
Duff, Olivia, 12, female, born in Virginia, in school
Duff, Robert, 10, male, born in Virginia, in school
Duff, Peter, 8, male, born in Virginia
Duff, Julia, 6, female, born in Virginia

The estate of James Dunn, deceased, in account with James A. Dunn, admor de bonis non with the Will annexed, from 1851 to 1852 was ordered recorded on 20 December 1852. On 1 July 1852, a balance of $210.41 shows a division in seven equal parts to: Chas C. Smith & wife, Peter T. Duff & wife, John. J. Cauthorn & wife, children of Thos. Parron decd., Cordelia Dunn, A. F. Allen, and James W. Dunn, implying that not all are of legal age. [Essex County, Virginia, Will Book 27, pages 255–256.]

On 13 July 1858, Ann Taylor sold to James A. Dunn, Trustee in a certain Deed from Peter T. Duff for the benefit of Olivia D. Duff his wife and children. For $50 from Arthur Dunn [sic] Trustee as aforesaid “all of her right tittle and interest in the Land of George Taylor her late husband deceased”. Signed Ann Taylor. Ann Taylor personally appeared to acknowledge the deed 18 July 1858. Recorded 7 September 1868. [Essex County, Virginia, Deed Book 52, pages 83-84.]

On 23 February 1859, Smith Young and Margaret R. Young his wife of the first part sold to James A. Dunn, trustee for Olivia Duff, wife of Peter T. Duff of the second part. For $175, the Youngs sell 13 acres, more or less, adjoining the land of James Croxton deceased, it being the same tract Smith Young purchased of John A. Parker and by Parker of George Taylor deceased. Signed Smith Young, Margaret R. Young. Recorded 7 September 1868.
On 13 July 1858, Ann Taylor sold to James A. Dunn Trustee in a certain Deed from Peter T. Duff for the benefit of Olivia D. Duff his wife and children bearing the date of [blank]. For $50 cash, Ann Taylor sells all of her right tittle [sic] and interest in the Land of George Taylor her late her late [sic] husband deceased. Signed Ann Taylor. On 18 July 1858, acknowledged before Elzer Fogg J.P. On 7 September 1868, ordered recorded.
[Essex County, Virginia, Deed Book 52, pages 83–84.]

In 1865, orators and oratrices Charles C. Smith and Emily C. his wife, Peter T. Duff and Olivia Duff his wife, Christopher Newbill and Mary Newbill his wife, Dorinda M. Cauthorn, Arthur F. Allen, James W. Parron, Lucy E. Parron, Cordelia Dunn and Edwin Hundley complained to the Chancery Court. That James W. Dunn died in 1864 intestate unmarried leaving Emily, Olivia, Mary, Dorinda, Cordelia all 1/2 sisters, James A. Dunn and Edwin Hundley 1/2 brothers, and Arthur F. Allen son of a 1/2 sister, James W. Parron and Lucy E. Parron children of another 1/2 sister his only heirs. [Essex County, Virginia, Box Chancery No. 42, item 42-F-10.]

1880 U.S. Census, Essex County, Virginia
Chapel Grove Precinct
Page 205 (26)
Dwelling 236, Family 236
Duff, Peter T., white, male, 64, married, farmer, born in Virginia, both parents born in Virginia
Duff, Olivia A., white, female, 67, married, keeping house, born in Virginia, both parents born in Virginia
Duff, Peter A., white, male, 32, son, single, works on farm, born in Virginia, both parents born in Virginia
Duff, Olivia A., white, female, 21, daughter, single, at home, born in Virginia, both parens born in Virginia