I would guess that it's your wireless connection. In fact, my first course of action would be to find some way of finding out what the best wireless channel is to use in your area. Here's a guide. Yes, it's technical but if you want to solve the problem then the only real answer is to get stuck in! )

Other things you can do is move your router:
a) Away from any potential sources of interference.
b) Nearer the middle of your house.
c) Up a bit higher.
d) Away from any metal objects.

There's lots of other small things you can do, Google should point you in the right direction.

Of course, this is all assuming it is the wireless connection that is flakey. It sounds like all your devices are connected wirelessly but if you have got a PC with a wired connection and that drops out too, you'll need to get on to your ISP to get them to test the line etc.

Good Luck!