The application of waste heat from exhaust gas of ship’s main engines has become widely practiced as early as in the 1930s. Thus the increase of ship’s overall efficiency was improved. Nowadays all newly built ships of the 400 gross tonnage and above must have specified energy efficiency design index, which is a measure for CO2 emissions of the ship and its impact on the environment. Therefore, the design of waste heat recovery systems requires special attention. The use of these systems is one of the basic ways to reduce CO2 emissions and to improve the ship’s energy efficiency. The paper describes the ship’s heating systems designed for the use of waste heat contained in the exhaust gas of self-ignition engines, in which the heat carriers are respectively water vapor, water or thermal oil. Selected results of comparative exergy analysis of simplified steam, water and oil heating systems have been presented. The results indicate that the oil heating system is comparable to the water system in terms of internal exergy losses. However, larger losses of exergy occur in the case of a steam system. In the steam system, a significant loss is caused by the need to cool the condensate to avoid cavitation in boiler feed pumps. This loss can in many cases cause the negative heat balance of ship during sea voyage while using only the exhaust gas boilers.