Let's rephrase the question to, "If global CO2 concentration in the atmosphere was constant, would global average temperature stay constant?" Considering even moderately long timescales (e.g., a few thousand years), the answer would be no because even hypothetically if CO2 concentration stayed exactly the same (which itself is basically an impossible hypothetical considering any part of Earth history if considering more than a few hundred to thousand of years), the amount of incoming solar radiation would change because of a variety of changes in Earth's orbital parameters (e.g., eccentricity, obliquity, precession). These orbital induced changes in solar radiation (i.e., insolation) are typically described in the context of Milankovitch cycles and are broadly the main drivers of most (but not all) climatic changes on millennial timescales with CO2 largely acting as a reinforcing feedback on these timescales. On longer timescales (e.g., several million to hundreds of millions of years), the amount of CO2 in the atmosphere represents a primary driver for climate with orbital changes influencing smaller scale variations (both in terms of magnitude and time scale). So broadly, major and long-lived changes in CO2 concentration (that reflect long-term balances between carbon stored in the atmosphere/hydrosphere vs lithosphere/mantle) drive transitions between Icehouse vs Greenhouse climates where as Milankovitch cycle type controls drive things like interglacial-glacial transitions within an Icehouse climate (i.e., what we have now). Both of these are discussed in significantly more detail in a variety of places, like this FAQ entry.