An empty folder or file with a label on it within a filing cabinet still takes up space. An empty box has nothing in it, if it is strong enough it may contain a (partial, yeah I know) vacuum. It still takes up space.

Is it OK to delete empty folders?

Generally speaking, it is safe to delete empty folders, although you would make no real space savings as they occupy 0 bytes. Nevertheless, if it is just good house-keeping you are looking for, you may go ahead. ... In case they are from current Program folders, you could still delete them.

How much storage does a folder use?

So, any folder or directory, on a modern drive (>512 MB), has at least an overhead of 4 KB, so as to store the table _(source). However, this means you can store 128 different files (4 * 1024 B / 32 B = 128) on the same folder and the overhead in storing the records is still the same 4KB as before.