My brain isn't at 100% right now, so my suggestions are going to be general. I'm going to recommend
ideal starting options, not the only options. You can always choose cheaper options, or fewer.
Start with some basic colors of contrast paint (Games Workshop/Citadel or Army Painter, and you can probably save money by buying a set and then adding additional colors.)
Add in some basic paints (I'd recommend Reaper, Vallejo, or Army Painter to keep it simple.) Red, yellow, green, blue, black, white, a medium gray, brown, tan, silver, gold. Add a reaper flesh triad (that's what they're called.) (Super low budget option: Black, white, brown, a single fleshtone, red, yellow, green, blue, silver, gold.)
Toss in some washes. I'm partial to Vallejo, but some people prefer others. You can get by with a brown/sepia and a black.
White primer is easier to learn on, although if you're feeling extravagant you can add black for a special type of priming later (zenithal.) Recommendations: If you have an airbrush, Badger Stynlrez. If not, then you'll want rattlecans. If you want something that will work on anything, Army Painter is a good choice. If you are using Citadel Contrast Paints and want to be a corporate shill (

), then their Wraithbone is probably a better choice. You can also get by with generic Krylon primers, but they tend to have a rougher grit and the accelerant they use will react with some materials, and can ruin miniatures made out of them (keep Krylon far way from PVC minis.) (Super budget option: start with white Krylon primer as long as you're sure it won't mess with your minis)
Get some decent brushes. A few good brushes are better than a lot of cheap ones, and you'll have an easier time. Important note that a lot of people don't always get: Smaller brushes aren't always better for details. For details you want a tiny
point, not a tiny brush. With thin acrylics, a tiny brush dries out too fast, while a larger brush gives you more working time (and you only paint with the very tip, so if the tip of a larger brush is as small as the tip of a tiny brush, it works the same.) Get a couple of crappy brushes, too, for drybrushing (drybrushing will ruin a good brush.)
This is a great set to get started (and I know the company.) (Super budget option... this is the first place you want to spend. Good brushes make a huge difference.)
You need good lighting. You want a white bulb that's as close to natural white light as possible. Do it under a standard house bulb, and your colors may look funky under other lighting. Ideally, something around 5000-5500 Kelvin. You can get by with a cheap hobby lamp and a Walmart bulb in that range, although there are higher priced
options that are much better. (Super budget option: a clip light from the hardware store and a generic retail bulb in the 5,000-6,000 K range.)
If you need something to help you see better, do not get a single-lens magnifier. They distort the subject, mess with your depth perception, and can make brush control difficult. You want binocular magnifiers (one for each eye) to maintain proper perception.
I use something like this, which works perfectly with glasses. (Super budget option: a Harbor Freight style binocular magnifying visor.)
If you are wanting to break the ice and get color on miniatures, start by focusing on the contrast paints and supplementing with the traditional paints. On the other hand, if you're wanting to approach it from a skill/art direction, I'd suggest skipping contrast paints early on and just using traditional paints until you have the basics of brush control, shading, and highlighting down (contrast paints are fast and easy, but they teach bad habits - or, rather, they fail to teach good ones.)
/edit - adding one more. Get yourself some The Masters Brush Cleaner and Preserver. This is the super-budget option - it keeps you from having to re-buy brushes, and is the difference between getting brushes every two or three years, or getting brushes every two or three months.
You also want a hobby knife and extra blades, plus a basic set of jeweler's files (you can get some for under ten bucks.) For the knife, if you want the super budget short-term item, get yourself a basic X-Acto (not Exacto) knife with a few extra blades. If you prefer long-term savings and a much better product, then get a Swann-Morton scalpel handle and some 10a blades. They are sharper, stay sharp longer (so you go through fewer blades) and the blades actually cost considerably less than X-Acto. If you use them for several years, the scalpel is cheaper. If you're only worried about the next six months, the X-Acto is fine.