Baby stuff: what works for us

Must haves:
Some place to sleep: You can go Herdman-style and stick the baby in a dresser drawer if you want, but cribs are traditional. We have a pack-n-play which also serves as changing table and a bit of storage.  If I had more money and space, I would have liked a co-sleeper for easy nighttime feeding.
Clothes: I wouldn't have needed to buy anything because we were given and loaned a lot in newborn and 0-3 mon. sizes. Skip newborn sizes if you're expecting a big baby. Newborns are happy to spend most of their time wrapped in a blanket, so a shirt or onesie and diaper are often sufficient. The bare necessities are (in order from "even if you're destitute" to "slightly less necessary"):

• Onesies
• Socks
• Elastic-bottomed gowns (WAY easier for middle-of-the-night changes than the footed sleepers). These usually have foldover cuffs so you can cover Baby's hands for warmth or to keep those sharp little fingernails under wraps.
• Hats, mittens, and other seasonal clothing if needed 
• Footed sleepers (for one-piece dressing when a onesie isn't warm enough and wrapping in a blanket isn't practical). Harder to find with the foldover cuffs.

I would have liked to have a couple of the wrap front shirts at first, while I was still dealing with the umbilical cord stump, but we did fine without. A couple of pairs of sweat pants and a little baby jacket were also nice to have.
Diapers and wipes: We love our cloth diapers, but I will write about that set-up some other time. Short version: especially because water is included in our rent, it's very cost-effective (more expense up front, less over the long term) for us. You have more laundry with a baby anyway, so additional time spent on cleaning is marginal.
Something to contain dirty diapers: A purpose-made diaper pail isn't necessary. Any covered trashcan will work as long as you empty it frequently.
Car seat: Of course. The Graco SnugRide models are cheap, well-rated, and fit in a lot of the car seat-holding strollers.
Blankets: We got LOTS (more than a dozen?) of receiving blankets, more than I thought we'd need, but it turned out about right. My favorites are the large, flannel, single-layer square ones. They're best for swaddling. Blankets also useful as: burp cloths, sun shades, nursing cover, play mat, something to chew on, absorbent layer if baby needs some diaper-free time, something for the baby to "wear" when there's a diaper blow-out disaster (it happened to me).
Burp cloths: or anything absorbent--extra cloth diapers, wash cloths, or whatever. Bar mops are actually a great size.
Diaper bag: Doesn't have to be a "real" diaper bag, of course. Just something that will fit diapers, wipes, a water-proof container in case you have to pack out your dirties, an extra set of clothes (for baby, and maybe a shirt for you if you have a spitter), and a blanket. Plus bottle, formula, etc., if you aren't breastfeeding.
Toiletries and medical: Lotion and shampoo don't have to be special "baby" products, but it's probably wise to start with something unscented and good for sensitive skin. Rectal thermometer. Bulb syringe (the hospital will give you one). I like manicure scissors for cutting tiny fingernails. Diaper rash cream. Baby pain killer (ibuprofen or acetaminophen). Nasal saline (a stuffed-up baby is a miserable baby). Comb or brush if your baby has hair (mine doesn't).
Books: High-contrast black-and-white wordless books are great for infants.
Journal: or just write those firsts on a calendar. Whatever.
Grandma or grandma-substitute: Anybody who will hold the baby but give him/her back whenever you want, do dishes/laundry/clean the bathroom without supervision, won't care if mom looks like something the cat drug in, and can stock your freezer. Totally priceless. Hard to find on Amazon.

Good to have
Baby carrier: Somehow, I have a sling, a stretchy wrap (Moby brand), a mei tai, and an Ergo-type carrier. I like them all for different reasons, but I think the stretchy wrap and sling are most versatile.
Swaddlers and blanket sleepers: You can swaddle with a blanket, but a swaddler is more secure and snugger. We got a few as gifts.
Stroller: makes going for walks easier. Even more important for running errands on foot, for example if you are going to have to carry groceries home.  I wanted something smallish that I could snap the car seat into and that had an adjustable seat that would recline (nearly) flat and could face me. A backwards-facing seat is hard to find and usually expensive, but we decided on the First Years Indigo and LOVE it. Only negative about it is that it doesn't fold up very small. Also the wheels squeak, but silicone lubricant spray clears that right up.
Boppy: or similar nursing pillow. This is the one thing that I regret not taking to the hospital. I haven't used it for nursing in months, but just at first it was so helpful in getting baby and me in the right position. Bed pillows are too squishy to be really useful. One cover for it has been fine.
Bathtub or sink insert: Just makes hanging on to a slippery baby easier. I liked the spongy yellow sink insert, but my baby was too long for it quickly. Plastic tubs are bulky and difficult to store, but handy.
Toys: a couple of rattles are nice, and you can't avoid the deluge of stuffed animals.  Check out Taggies.

Maybes
Swing or bouncy seat: this is a good thing to borrow if you can, because babies don't always like them. But if the baby does like something like this, it's great for calming down, getting some hands-free time, etc. Buy second-hand for sure.
Breast pump: also good to borrow until you really know what you need. If you need a serious breast pump right away (for a NICU baby or for serious latch problems), you can rent a hospital-grade pump from the hospital. A single electric pump is fine for us since all I want is an occasional bottle. Even a manual might have been enough.
Bottles and bottle nipples, bottle brush: these are a must if you are formula-feeding.
Pacifiers: Our baby just never would take a pacifier, and the hospital will give you at least one if you want it. You can always pick one up at Walmart if it's needed.
Rocking chair

For Mom 
Breastfeeding Made Simple: It's a book; very helpful. Or a pre-natal nursing class. Sadly, post-partum nurses don't always give great advice about nursing, so it's good to know what's normal and have some tips ahead of time.
Lanolin: my hospital gave me an itsy-bitsy tube to start with
Nursing pads
Nursing bra: take your best guess at size and just get one or two to start. After a couple weeks, buy a few that actually fit. Any kind of unstructured, stretchy bra is fine for at home or at night.
Dark-colored pajamas with a button front: To take to the hospital so you don't have to wear a hospital gown, can nurse easily, and don't have to worry too much about stains.