Unlike fairies that drop from enemies, these fairies never disappear until collected. The easiest way to collect them is using the hookshot or a boomerang.
The fairies respawn upon exiting/re-entering the supertile, in case more than four are needed for some reason.