Joyo Bank branches are basically closed from May 1 to May 5. Most ATMs will be in operation, but many of them will be operating under “holiday” hours, so they may close early. Holiday service fees will also apply. Also, sometimes it will be impossible to get money out of bank machines from banks other than Joyo (this will probably only be an issue if you travel out of Ibaraki). When in doubt, try the ATMs at 7-11s as they tend to be open the longest (usually 8:00am to 9:00pm). You can also use your Joyo card in postal ATMs (but only from about 9am to 5pm), which can help if you are travelling.
Here is a link to the information (in Japanese) about accessing your money at Joyo Bank branches/ATMs during Golden Week.
If you think you are going to need extra money during the holidays, plan ahead carefully!