Usually singular: Milch, Butter & co.

Some food words are normally used only in the singular — you measure them instead of counting them:

• die Milch, die Butter
• der Käse, der Reis, der Zucker
• das Obst, das Gemüse

Nobody orders “zwei Milche” — German puts a measure word in front instead: zwei Glas Milch.

Some of these do have a plural for different kinds — die Käse are cheeses, sorts of cheese — but for shopping and eating you need the singular.