colder than at the bottom of frozen-over fresh water lakes and rivers.

As the surface of sea water begins to freeze (at -1.9 °C for salinity 3.5%) the ice that forms is essentially salt-free, with about the same density as freshwater ice. This ice floats on the surface, and the salt that is "frozen out" adds to the salinity and density of the sea water just below it, in a process known as brine rejection.