As someone who has been a hobbyist since high school and used a lot of eBay-bought and make-do equipment through college, I'll throw in my two cents. I used to work with a lot of questionable equipment to save money when I was starting out, but I have eventually come to have to replace most of it. It helped me get started in the hobby, but, if you have the money to spend, I would not skimp if you can help it.
My list of bare-minimum must-haves would be:
- a used scope- 2 channels bare minimum, and maybe 4 if you can find a good deal. Ebay usually has lots of used ones out of schools and labs that usually work just fine. Max frequency depends on what frequencies you actually plan to work at, so if you only plan to use it for audio, or maybe digital stuff with microcontrollers and 74 series logic, then as low as 20 MHz might be fine. I was able to got a 4-channel, 100 MHz scope for a good price of about $100.
- a good bench power supply- If you plan to do RF related projects I recommend a linear power supply, as switching supplies can produce RF noise. If noise isn't an issue, however, a switcher will likely have a higher max current rating for the same cost. A decent one might run well over a hundred dollars, but off-brand Chinese ones can also be had for around $60 on ebay and amazon- if you dare trust them.
-a digital multimeter- Flukes are nice, but truth be told, I've gotten by just fine with the cheap $25 ones you buy at the hardware store. I'll probably take some flak for suggesting that, though. Sometimes the cheap ones aren't auto-ranging, if that is something that will bother you. Do NOT get an analog panel meter one.
-some kind of audio signal generator- I used to use one made from a kit based around the XR-2206 chip, but it isn't that great so I'll leave it to others to suggest where to find a proper one. My school's amateur radio station still sometimes uses old tube-based HP 200CD wein bridges up at their lab, but those are probably being sold as collectors items on eBay these days. In a pinch, a simple twin-T sine-wave oscillator or a 555 timer for a square wave can work.
-I wouldn't recommend a frequency counter just yet unless you specifically plan to do RF work.
I also used to use a simple home-made power supply based on the LM317 variable regulator that I made from a kit, then another that I got for free at a Hamfest. They work fine for low-power stuff, but they have no current limiting control and the transformers in them started to overheat once I started making projects with higher power requirements. I now use one of the aforementioned Chinese $60 supplies. It works for me so far.