The need for an interdisciplinary environmental library collection became apparent early in the history of CES. The library started with books and documents shelved in three wooden bookcases in an office in the Center's initial home, the Van Rensselaer Mansion-where Sawyer Library now stands. When the Center moved to Kellogg House in 1978, the library was put on the porch. In 1982, a fund created in memory of CES graduate Matt Cole '80 by his family created a self-supporting endowment for the library, which was then named the Matt Cole Memorial Library. These funds also supported the renovation of the CES living room into a library reading room.

In 1993, as the Center celebrated its 25th anniversary, the Class of 1943 gave the College funds as part of their 50th reunion gift to add a new library wing to Kellogg House in order to expand the space available for the library. The new wing was completed in 1995.