1. In 3-quart saucepan, stir water, cocoa, and sugar until smooth. Over medium heat, heat mixture to boiling.
2. Stir in milk; heat until tiny bubbles form around edge (do not boil). Remove saucepan from heat; add vanilla.
3. Pour into mugs or cups. Top each serving with marshmallow or whipped cream.