The church used the Ellicott estate to start a college, initially called Mount Saint Clemens, when it opened in 1868. Several additions to the original building were made, and a chapel was constructed. In 1882 it was renamed Saint Mary’s College. In 1893 the chapel was established as a parish church dedicated to “Our Lady of Perpetual Help.” The parish moved the church to a nearby facility in the 1950s. Enrollment at the college declined, and the college was closed in 1972. The State of Maryland purchased a portion of the campus in 1987 and added the land to adjacent Patapsco Valley State Park.

Originally located about 2 miles southeast of Ellicott City along the Patapsco River, Ilchester was settled by the Ellicott family (John, Andrew, and Joseph Ellicott) in the late 18th century. The Baltimore and Ohio Railroad (B&O) was built up the Patapsco River valley through Ilchester in 1830 (see Old Main Line Subdivision), and George Ellicott, Jr. (grandson of Andrew) built a tavern along the railroad. Ilchester did not become a major stop for the B&O, and the tavern was unsuccessful. Ellicott sold the tavern to a Redemptorist church order in 1866.